Vehicle trailer connection recognition method and apparatus, vehicle, and storage medium

Abstract

The application relates to the technical field of vehicle control, in particular to a vehicle towing connection identification method and device, a vehicle and a storage medium. The method comprises the following steps: determining the type of a current towing load according to an image flow, and determining the first mass center coordinates and the first central axis direction of the towing connector of a towing vehicle and the second mass center coordinates and the second central axis direction of the towing connector of the current towing load according to point cloud data; calculating the position deviation and the rotation angle deviation between the towing connector of the towing vehicle and the towing connector of the current towing load based on the first mass center coordinates, the first central axis direction, the second mass center coordinates and the second central axis direction; and determining the towing connection state of the towing vehicle and the current towing load according to the type of the current towing load, the position deviation and the rotation angle deviation. Thus, the problem that the related art cannot automatically judge whether the actual state of the towing connection is connected in place and has a safety hazard is solved, and the safety of towing driving is improved.

Description

Technical Field

[0001] This application relates to the field of vehicle control technology, and in particular to a method, device, vehicle, and storage medium for identifying vehicle trailer connections. Background Technology

[0002] The load on a towed trailer is greatly affected by crosswinds and road conditions during operation, and it is prone to swaying at high speeds, posing a challenge to driving safety. Therefore, improving the safety of towed trailer driving has received increasing attention.

[0003] In related technologies, cameras and millimeter-wave radar are used to fuse perception of the area behind the vehicle. Visual recognition algorithms and radar point cloud processing algorithms are used to identify the towed load target, and the driver is assisted in completing the hook-up operation based on the recognition results.

[0004] However, this method cannot automatically determine whether the actual status of the trailer connection is properly established, posing a potential safety hazard that urgently needs to be addressed. Summary of the Invention

[0005] This application provides a method, device, vehicle, and storage medium for identifying vehicle trailer connections, in order to solve the problem that related technologies cannot automatically determine whether the actual status of the trailer connection is properly connected, which poses a safety hazard, and improve the safety of trailer driving.

[0006] To achieve the above objectives, the first aspect of this application proposes a method for identifying vehicle trailer connections, comprising the following steps: The type of the current towed load is determined based on the image stream within the first preset perimeter of the tractor vehicle, and the first centroid coordinates and first central axis direction of the towed connector of the tractor vehicle, as well as the second centroid coordinates and second central axis direction of the towed connector of the current towed load, are determined based on the point cloud data within the second preset perimeter of the tractor vehicle. Based on the first centroid coordinates, the first centerline direction, the second centroid coordinates, and the second centerline direction, calculate the positional deviation and rotational angle deviation between the trailer connector of the tractor vehicle and the trailer connector of the current trailer load; The towing connection status between the tractor vehicle and the current towed load is determined based on the type of the current towed load, the position deviation, and the rotation angle deviation.

[0007] According to one embodiment of this application, determining the first centroid coordinates of the trailer connector of the tractor vehicle includes: The point cloud data is clustered to generate the first point cloud cluster corresponding to the trailer connector of the tractor vehicle; The arithmetic mean of the spatial coordinates of all points in the first point cloud cluster is calculated to obtain the coordinates of the first centroid.

[0008] According to one embodiment of this application, determining the first centerline direction of the trailer connector of the tractor vehicle includes: Calculate the first spatial covariance matrix of the first point cloud cluster; The first spatial covariance matrix is ​​decomposed into eigenvalues ​​to obtain a first set of eigenvectors. The direction of the eigenvector corresponding to the largest eigenvalue in the first set of eigenvectors is taken as the direction of the first central axis.

[0009] According to one embodiment of this application, determining the second centroid coordinates of the trailer connector of the current towed load includes: The point cloud data is clustered to generate a second point cloud cluster corresponding to the trailer connector of the current trailer load; The arithmetic mean of the spatial coordinates of all points in the second point cloud cluster is calculated to obtain the coordinates of the second centroid.

[0010] According to one embodiment of this application, determining the second centerline direction of the trailer connector of the current towed load includes: Calculate the second spatial covariance matrix of the second point cloud cluster; The second spatial covariance matrix is ​​decomposed into eigenvalues ​​to obtain a second set of eigenvectors. The direction of the eigenvector corresponding to the largest eigenvalue in the second set of eigenvectors is taken as the direction of the second central axis.

[0011] According to one embodiment of this application, calculating the positional deviation between the trailer connector of the tractor vehicle and the trailer connector of the current towed load includes: Calculate the absolute value of the difference between the second centroid coordinates and the first centroid coordinates in each coordinate axis direction to obtain the longitudinal deviation, lateral deviation and vertical deviation respectively; The positional deviation is obtained based on the longitudinal deviation, the lateral deviation, and the vertical deviation.

[0012] According to one embodiment of this application, calculating the rotation angle deviation between the trailer connector of the tractor vehicle and the trailer connector of the current towed load includes: Calculate the angle between the direction of the first central axis and the direction of the second central axis; The rotation angle deviation is obtained based on the included angle.

[0013] The vehicle towing connection identification method proposed in this application determines the type of the current towed load based on an image stream, and determines the first centroid coordinates and first central axis direction of the towing vehicle's towing connector, as well as the second centroid coordinates and second central axis direction of the current towed load's towing connector based on point cloud data. Based on the first centroid coordinates, first central axis direction, second centroid coordinates, and second central axis direction, the method calculates the positional deviation and rotational angle deviation between the towing vehicle's towing connector and the current towed load's towing connector. Based on the type of the current towed load, positional deviation, and rotational angle deviation, the towing connection status between the towing vehicle and the current towed load is determined. This solves the problem of related technologies being unable to automatically determine whether the actual towing connection is properly in place, which poses a safety hazard, and improves the safety of towing operations.

[0014] To achieve the above objectives, a second aspect of this application provides a vehicle trailer connection identification device, comprising: The first determining module is used to determine the type of the current towed load based on the image stream within a first preset perimeter of the tractor vehicle, and to determine the first centroid coordinates and the first central axis direction of the towed connector of the tractor vehicle, as well as the second centroid coordinates and the second central axis direction of the towed connector of the current towed load, based on the point cloud data within a second preset perimeter of the tractor vehicle. The calculation module is used to calculate the positional deviation and rotational angle deviation between the trailer connector of the tractor vehicle and the trailer connector of the current trailer load based on the first centroid coordinates, the first centerline direction, the second centroid coordinates, and the second centerline direction. The second determining module is used to determine the towing connection status between the tractor vehicle and the current towing load based on the type of the current towing load, the position deviation, and the rotation angle deviation.

[0015] According to one embodiment of this application, the first centroid coordinates of the trailer connector of the tractor vehicle are determined, wherein the first determining module is specifically used for: The point cloud data is clustered to generate the first point cloud cluster corresponding to the trailer connector of the tractor vehicle; The arithmetic mean of the spatial coordinates of all points in the first point cloud cluster is calculated to obtain the coordinates of the first centroid.

[0016] According to one embodiment of this application, a first centerline direction of the trailer connector of the tractor vehicle is determined, wherein the first determining module is specifically used for: Calculate the first spatial covariance matrix of the first point cloud cluster; The first spatial covariance matrix is ​​decomposed into eigenvalues ​​to obtain a first set of eigenvectors. The direction of the eigenvector corresponding to the largest eigenvalue in the first set of eigenvectors is taken as the direction of the first central axis.

[0017] According to one embodiment of this application, the second centroid coordinate of the trailer connector of the current trailer load is determined, wherein the first determining module is specifically used for: The point cloud data is clustered to generate a second point cloud cluster corresponding to the trailer connector of the current trailer load; The arithmetic mean of the spatial coordinates of all points in the second point cloud cluster is calculated to obtain the coordinates of the second centroid.

[0018] According to one embodiment of this application, the second centerline direction of the trailer connector of the current towed load is determined, wherein the first determining module is specifically used for: Calculate the second spatial covariance matrix of the second point cloud cluster; The second spatial covariance matrix is ​​decomposed into eigenvalues ​​to obtain a second set of eigenvectors. The direction of the eigenvector corresponding to the largest eigenvalue in the second set of eigenvectors is taken as the direction of the second central axis.

[0019] According to one embodiment of this application, the calculation module calculates the positional deviation between the trailer connector of the tractor vehicle and the trailer connector of the current towed load. Specifically, the calculation module is used for: Calculate the absolute value of the difference between the second centroid coordinates and the first centroid coordinates in each coordinate axis direction to obtain the longitudinal deviation, lateral deviation and vertical deviation respectively; The positional deviation is obtained based on the longitudinal deviation, the lateral deviation, and the vertical deviation.

[0020] According to one embodiment of this application, the calculation module calculates the rotational angle deviation between the trailer connector of the tractor vehicle and the trailer connector of the current towed load. Specifically, the calculation module is used for: Calculate the angle between the direction of the first central axis and the direction of the second central axis; The rotation angle deviation is obtained based on the included angle.

[0021] The vehicle trailer connection identification device proposed in this application determines the type of the current trailer load based on an image stream, and determines the first centroid coordinates and first central axis direction of the trailer connector of the tractor vehicle, as well as the second centroid coordinates and second central axis direction of the trailer connector of the current trailer load, based on point cloud data. Based on the first centroid coordinates, first central axis direction, second centroid coordinates, and second central axis direction, the device calculates the positional deviation and rotational angle deviation between the trailer connector of the tractor vehicle and the trailer connector of the current trailer load. Based on the type of the current trailer load, the positional deviation, and the rotational angle deviation, the device determines the trailer connection status between the tractor vehicle and the current trailer load. This solves the problem of related technologies being unable to automatically determine whether the actual trailer connection is properly in place, which poses a safety hazard, and improves the safety of trailer driving.

[0022] To achieve the above objectives, a third aspect of this application provides a vehicle comprising: a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor executes the program to implement the vehicle trailer connection identification method as described in the above embodiments.

[0023] To achieve the above objectives, a fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to implement the vehicle trailer connection identification method as described in the above embodiments.

[0024] To achieve the above objectives, a fifth aspect of this application provides a computer program product comprising a computer program that, when executed by a processor, is used to implement the vehicle trailer connection identification method as described in the above embodiments.

[0025] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0026] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 This is a flowchart of a vehicle trailer connection identification method provided according to an embodiment of this application; Figure 2 This is a schematic diagram showing the installation positions of a camera and radar according to one embodiment of this application; Figure 3 This is a block diagram of a vehicle trailer connection identification device provided according to an embodiment of this application; Figure 4 This is a structural schematic diagram of a vehicle provided according to an embodiment of this application. Detailed Implementation

[0027] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0028] The following description, with reference to the accompanying drawings, outlines a vehicle trailer connection identification method, apparatus, vehicle, and storage medium based on embodiments of this application.

[0029] Figure 1 This is a flowchart of a vehicle trailer connection identification method according to an embodiment of this application.

[0030] Before introducing the vehicle towing connection identification method proposed in the embodiments of this application, let's briefly introduce the vehicle towing connection identification system involved in this method. The system mainly includes a sensing module located directly behind the towing vehicle, an electronic control unit as the control core, a warning module for issuing alerts to occupants, and a screen prompt and voice alarm module.

[0031] The perception module is used to detect targets in the area behind the towing vehicle, and this module may include cameras and millimeter-wave radar. For example... Figure 2 As shown, the camera is preferably a wide-angle camera with night vision capabilities, installed directly above the rear windshield inside the towing vehicle, covering an area of ​​0.5m to 5m directly behind the towing vehicle. The millimeter-wave radar is preferably installed in the center of the rear of the towing vehicle, at the same height as the trailer hitch, with the radar detection surface perpendicular to the vehicle's longitudinal axis, primarily covering the area directly behind the towing vehicle. The camera and millimeter-wave radar complement each other in spatial positioning, forming a detection network with no blind spots. The camera and millimeter-wave radar can be connected to the electronic control unit (ECU) via a dedicated high-speed data cable and a CAN (Controller Area Network) bus, respectively, transmitting real-time raw target data (such as image streams, point cloud images, centroids, relative distances, relative rotation angles, and motion trajectories) to the ECU for processing. Specifically, the camera uses image recognition algorithms to identify the target type of the towed load (such as a motorhome or trailer) from the image stream; the millimeter-wave radar uses the frequency changes of emitted and received electromagnetic waves to accurately measure the spatial position and motion parameters of the target behind. The data from the two sensors are fused in the ECU, thereby improving the accuracy and reliability of perception.

[0032] The electronic control unit (ECU) is the brain of the entire system, responsible for data processing, risk assessment, and control decisions. An ECU includes at least a microprocessor, memory, signal input interface, control output interface, and a CAN bus communication module. The microprocessor runs the core control algorithm; the memory stores program code, object recognition information, and historical data; the signal input interface receives data from the sensing module; the control output interface connects to the warning module and the screen prompt and voice alarm module; and the CAN bus module acquires vehicle status information, such as the intent to enter towing mode, parking signals, and vehicle speed signals. The ECU can be connected to all other components of the system via wiring harnesses, enabling power supply and signal interaction.

[0033] The warning module is responsible for providing users with intuitive alerts. It includes visual and auditory components. The visual components, such as an LCD screen, can be integrated into the central control screen or instrument display. The auditory components, such as miniature speakers, can be installed in the door trim panels or headliner. The warning module receives control signals from the electronic control unit via a wiring harness. When the electronic control unit determines that the tow hook or connector is not connected, it drives the LCD screen to display a text warning message and simultaneously drives the speaker to emit an audio alarm, forming a multi-sensory alert.

[0034] The screen prompt and voice alarm module is the system's execution terminal, responsible for outputting prompts and alarm information to users in high-risk situations. It is connected to the control output interface of the electronic control unit through a wiring harness and receives screen prompts and voice alarm commands issued by the electronic control unit. The purpose is to enable users to be aware of the risks in a timely manner through both visual and auditory channels when the tow hook or connector is not connected due to negligence, thereby ensuring the personal and property safety of users and other traffic participants.

[0035] Next, we will explain in detail the vehicle trailer connection identification method in conjunction with the vehicle trailer connection identification system.

[0036] For example, such as Figure 1 As shown, the vehicle trailer connection identification method includes the following steps: In step S101, the type of the current towed load is determined based on the image stream within the first preset perimeter range of the tractor vehicle, and the first centroid coordinates and first central axis direction of the towed vehicle's trailer connector, as well as the second centroid coordinates and second central axis direction of the towed load's trailer connector, are determined based on the point cloud data within the second preset perimeter range of the tractor vehicle.

[0037] It is understood that in this embodiment, the first preset perimeter range refers to the spatial area 0.5m to 5m directly behind the towing vehicle. This range covers the main area where the trailer hitch is located, which can clearly capture the overall appearance features of the towed load while avoiding the introduction of irrelevant interference information from too far away. The second preset perimeter range refers to the spatial area covered by millimeter-wave radar directly behind the towing vehicle. This area complements the first preset perimeter range in space and is used to acquire point cloud data of the trailer hitch and the front structure of the towed load to accurately perceive the position and attitude of the trailer hitch. The towed load is the towed vehicle connected to the towing vehicle through the trailer hitch, including at least two types: RVs and trailer flatbeds. RVs are large trailers with living functions, and trailer flatbeds are flatbed towed structures used to carry goods or small items. The trailer hitch is a general term for the components used to realize the physical connection between the towing vehicle and the towed load, and may include the trailer hook and corresponding electrical connector at the rear of the towing vehicle, and the trailer hook and corresponding electrical connector at the front of the towed load. The centroid coordinates refer to the coordinates of the geometric center of the trailer connector in three-dimensional space, used to characterize the spatial position of the trailer connector. The centerline direction refers to the orientation of the geometric center axis of the trailer connector in three-dimensional space, used to characterize the spatial attitude of the trailer connector. The image stream is a sequence of image data continuously acquired by a camera. Point cloud data is a set of three-dimensional spatial points generated by millimeter-wave radar emitting electromagnetic waves and receiving reflected signals.

[0038] Specifically, when the vehicle is powered on, the system begins a self-test, and the perception module (including the camera and millimeter-wave radar), electronic control unit, and warning module all start and complete initialization. Subsequently, the electronic control unit continuously monitors the CAN bus for any indication of a user's intention to enter towing mode. Once this signal is detected, it sends a command to the perception module to initiate full-power operation. The camera simultaneously captures image streams within a first preset perimeter area behind the towing vehicle (e.g., a space 0.5m to 5m directly behind the towing vehicle), and analyzes and processes this data using the image processing unit built into the electronic control unit to identify the type of the current towed load, which can be categorized at least as a motorhome or a trailer. Simultaneously, the millimeter-wave radar detects point cloud data within a second preset perimeter area behind the towing vehicle, which may include the centroid position, relative distance, relative rotation angle, and trajectory of the target behind. Based on this point cloud data, the electronic control unit extracts the spatial pose parameters of the towing vehicle's trailer connector and the current towed load's trailer connector, respectively. These spatial pose parameters may include the centroid coordinates and the centerline direction. In this configuration, the trailer hitch of the tractor vehicle is located at the rear of the tractor vehicle, while the trailer hitch of the current towed load is located at the front of the current towed load. Therefore, the electronic control unit can obtain the basic information needed to determine the trailer connection status through the simultaneous acquisition and processing of image streams and point cloud data.

[0039] It should be noted that, in order to accurately identify the type of towed load, this application embodiment can employ an image recognition method based on convolutional neural networks. During the dataset creation phase, images of RVs and trailers from different scenes (e.g., sunny days, rainy days, nighttime), different angles (e.g., directly behind, side-rear), and different models are collected, with at least 1000 images for each category. Subsequently, the towed objects in the images are labeled with categories (i.e., labeled as "RV" or "trailer") using annotation tools. Data augmentation operations such as rotation, scaling, flipping, and adding noise are then performed on the labeled data to expand the dataset and improve the model's generalization ability. During the model selection and training phase, lightweight models suitable for embedded devices, such as YOLOv8 (You Only Look Once version 8) or MobileNet-SSD (Mobile Neural Network - Single Shot MultiBox Detector), are prioritized to balance recognition accuracy and computational speed. During training, the model is first pre-trained using publicly available datasets, and then fine-tuned using a self-made towed load dataset. This allows the model to focus on learning the differentiated features of RVs and trailers, with the training objective being to simultaneously perform object detection (locating towed objects) and classification (determining the type). After training, the model is quantized, compressed, and deployed to the vehicle's electronic control unit. Images are captured in real-time by the camera and input into the model, which then outputs the towed load category and its confidence level (e.g., "RV, 98% confidence"), thus achieving automatic and accurate identification of towed load types.

[0040] As one possible implementation, in some embodiments, determining the first centroid coordinates of the trailer connector of the tractor vehicle includes: performing clustering processing on point cloud data to generate a first point cloud cluster corresponding to the trailer connector of the tractor vehicle; and calculating the arithmetic mean of the spatial coordinates of all points in the first point cloud cluster to obtain the first centroid coordinates.

[0041] It is understood that, in the embodiments of this application, a point cloud cluster refers to a set of point clouds that belong to the same object after clustering, and is used to characterize the spatial shape and position of the object.

[0042] Specifically, after receiving point cloud data collected by millimeter-wave radar, the electronic control unit can perform clustering processing on the point cloud data, grouping spatially adjacent points that conform to the geometric characteristics of the trailer connector into a single cluster, generating the first point cloud cluster corresponding to the trailer connector of the tractor vehicle. Subsequently, the electronic control unit calculates the arithmetic mean of the spatial coordinates of all points in the first point cloud cluster, that is, calculates the average of the coordinate values ​​of all points in each coordinate axis direction, and uses the obtained average position as the first centroid coordinate of the trailer connector of the tractor vehicle.

[0043] Similarly, the method for obtaining the coordinates of the second centroid is similar to that for the first centroid.

[0044] As one possible implementation, in some embodiments, determining the second centroid coordinates of the trailer connector of the current trailer load includes: performing clustering processing on the point cloud data to generate a second point cloud cluster corresponding to the trailer connector of the current trailer load; and calculating the arithmetic mean of the spatial coordinates of all points in the second point cloud cluster to obtain the second centroid coordinates.

[0045] Specifically, after receiving point cloud data acquired by millimeter-wave radar, the electronic control unit performs clustering processing on the point cloud data, grouping spatially adjacent points that conform to the geometric characteristics of the trailer connector into a single cluster, generating a second point cloud cluster corresponding to the current trailer load trailer connector. Subsequently, the electronic control unit calculates the arithmetic mean of the spatial coordinates of all points in this second point cloud cluster, that is, it calculates the average of the coordinate values ​​of all points in each coordinate axis direction, and uses the obtained average position as the second centroid coordinate of the current trailer load trailer connector.

[0046] centroid coordinates The calculation formula can be expressed as:

[0047]

[0048]

[0049] in, This represents the number of point clouds within a point cloud cluster.

[0050] Therefore, by clustering point cloud data and calculating the arithmetic mean of the spatial coordinates of all points in the point cloud cluster to obtain the centroid coordinates, the spatial position of the trailer connector can be accurately determined with low computational complexity. Moreover, the more points in the point cloud cluster and the more uniformly they are distributed, the closer the calculated centroid coordinates are to the true geometric center, effectively ensuring the accuracy and stability of the centroid coordinates and providing a reliable data foundation for subsequent deviation calculations.

[0051] As one possible implementation, in some embodiments, determining the first centerline direction of the trailer connector of the tractor vehicle includes: calculating the first spatial covariance matrix of the first point cloud cluster; performing eigenvalue decomposition on the first spatial covariance matrix to obtain a first eigenvector group; and taking the direction of the eigenvector corresponding to the largest eigenvalue in the first eigenvector group as the first centerline direction.

[0052] It is understood that, in this embodiment, the spatial covariance matrix refers to the covariance matrix calculated based on the spatial coordinates of each point in the point cloud cluster. It describes the dispersion of the point cloud cluster along each coordinate axis in three-dimensional space and the correlation between these directions. This matrix is ​​a 3×3 symmetric matrix, and its elements reflect the geometric shape and orientation information of the point cloud cluster in space. The eigenvector group refers to the three eigenvectors obtained after eigenvalue decomposition of the spatial covariance matrix, each corresponding to an eigenvalue. The three eigenvectors are orthogonal to each other and together constitute a coordinate basis describing the spatial orientation of the point cloud cluster.

[0053] In other words, after the electronic control unit obtains the first point cloud cluster corresponding to the trailer connector of the tractor vehicle, it can perform principal component analysis on the first point cloud cluster to solve for the principal direction vector of the point cloud, that is, its longest axis direction.

[0054] Specifically, the spatial covariance matrix of the first point cloud cluster is calculated as the first spatial covariance matrix. This matrix describes the dispersion of the distribution of each point in the first point cloud cluster along each coordinate axis in three-dimensional space. Subsequently, the electronic control unit can perform eigenvalue decomposition on the first spatial covariance matrix (that is, mathematical operation to decompose the spatial covariance matrix into eigenvalues ​​and eigenvectors; the magnitude of the eigenvalue reflects the dispersion of the point cloud cluster in the direction of its corresponding eigenvector; the larger the eigenvalue, the greater the extension range of the point cloud cluster in that direction), obtaining three eigenvalues ​​and their corresponding three eigenvectors, forming the first eigenvector group. Since the direction of the eigenvector corresponding to the largest eigenvalue represents the direction in which the point cloud cluster is most dispersed in space, that is, the main extension direction of the point cloud cluster, the electronic control unit takes the direction of the eigenvector corresponding to the largest eigenvalue in the first eigenvector group as the first central axis direction of the towing vehicle's trailer connector, which is used to represent the angular attitude of the towing vehicle's trailer connector.

[0055] Similarly, the method for obtaining the direction of the second central axis is similar to that of the first central axis.

[0056] As one possible implementation, in some embodiments, determining the second centerline direction of the trailer connector of the current towed load includes: calculating the second spatial covariance matrix of the second point cloud cluster; performing eigenvalue decomposition on the second spatial covariance matrix to obtain a second eigenvector group; and taking the direction of the eigenvector corresponding to the largest eigenvalue in the second eigenvector group as the second centerline direction.

[0057] Specifically, after obtaining the second point cluster corresponding to the trailer connector of the tractor vehicle, the electronic control unit can calculate the spatial covariance matrix of this second point cluster as the second spatial covariance matrix. This matrix describes the dispersion of the distribution of each point in the second point cluster along each coordinate axis in three-dimensional space. Subsequently, the electronic control unit can perform eigenvalue decomposition on the second spatial covariance matrix to obtain three eigenvalues ​​and their corresponding three eigenvectors, forming a second eigenvector group. The electronic control unit takes the direction of the eigenvector corresponding to the largest eigenvalue in the second eigenvector group as the direction of the second centerline of the trailer connector of the tractor vehicle, which is used to characterize the angular attitude of the trailer connector under the current towing load.

[0058] Therefore, by performing principal component analysis on point cloud clusters, the dispersion of the point cloud clusters in three-dimensional space is quantified into eigenvalues ​​and eigenvectors of the spatial covariance matrix. The direction of the eigenvector corresponding to the largest eigenvalue is taken as the central axis direction, which accurately captures the main extension direction of the trailer connector in space. This direction is then used as the geometric central axis of the connector, thus precisely representing the angular attitude of the trailer connector. Furthermore, the eigenvalue decomposition operation of the spatial covariance matrix is ​​mathematically mature and stable, with high computational efficiency, making it suitable for embedded deployment in vehicle electronic control units.

[0059] In step S102, based on the first centroid coordinates, the first centerline direction, the second centroid coordinates, and the second centerline direction, the positional deviation and rotational angle deviation between the trailer connector of the tractor vehicle and the trailer connector of the current towed load are calculated.

[0060] It is understood that, in the embodiments of this application, positional deviation refers to the spatial offset between the towing vehicle's trailer connector and the current towed load's trailer connector. Rotational angle deviation refers to the angular offset in spatial attitude between the towing vehicle's trailer connector and the current towed load's trailer connector, characterizing the degree of matching between the two in orientation.

[0061] In other words, using the center of mass and main direction of the trailer connector of the tractor vehicle as a reference, the electronic control unit can calculate the positional and angular offsets of the trailer connector of the current towed load relative to this reference. Specifically, based on the first center of mass coordinates, the first centerline direction, the second center of mass coordinates, and the second centerline direction, the electronic control unit calculates the positional and rotational angular deviations between the trailer connector of the tractor vehicle and the trailer connector of the current towed load to obtain the deviation information needed to determine their connection status.

[0062] As one possible implementation, in some embodiments, calculating the positional deviation between the trailer connector of the tractor vehicle and the trailer connector of the current towed load includes: calculating the absolute value of the difference between the second centroid coordinate and the first centroid coordinate in each coordinate axis direction to obtain the longitudinal deviation, lateral deviation and vertical deviation respectively; and obtaining the positional deviation based on the longitudinal deviation, lateral deviation and vertical deviation.

[0063] It is understood that, in the embodiments of this application, longitudinal deviation refers to the absolute value of the difference between the second centroid coordinate and the first centroid coordinate in the direction of travel of the tractor (i.e., the longitudinal direction of the vehicle coordinate system, usually the X-axis direction), used to characterize the degree of positional offset of the two trailer connectors in the longitudinal direction. Lateral deviation refers to the absolute value of the difference between the second centroid coordinate and the first centroid coordinate in the vehicle width direction (i.e., the left-right direction of the vehicle coordinate system, usually the Y-axis direction), used to characterize the degree of positional offset of the two trailer connectors in the left-right direction. Vertical deviation refers to the absolute value of the difference between the second centroid coordinate and the first centroid coordinate in the vehicle height direction (i.e., the vertical direction of the vehicle coordinate system, usually the Z-axis direction), used to characterize the degree of positional offset of the two trailer connectors in the vertical direction.

[0064] Specifically, the electronic control unit can calculate the absolute value of the difference between the second center of gravity coordinate and the first center of gravity coordinate in each coordinate axis direction, based on the first and second center of gravity coordinates. This yields the longitudinal deviation in the front-to-back direction, the lateral deviation in the left-to-right direction, and the vertical deviation in the up-down direction. The absolute values ​​of these three differences are collectively used as the positional deviation to characterize the degree of positional offset of the two trailer hitches in three-dimensional space. That is, let the first center of gravity coordinate be (x... 牵 y 牵 , z 牵 The second centroid coordinates are (x... 拖 y 拖 , z 拖 Then, the longitudinal deviation Δx = |x 拖 -x 牵 |, Lateral deviation Δy=|y 拖 -y 牵 |, Vertical deviation Δz=|z 拖 -z 牵 |

[0065] Therefore, by calculating the absolute values ​​of the differences between the second centroid coordinates and the first centroid coordinates along the three coordinate axes, the longitudinal, lateral, and vertical deviations are obtained respectively. This allows the positional offset of the two trailer connectors in three-dimensional space to be decomposed into three mutually orthogonal directional components, achieving a multi-dimensional and refined characterization of the degree of positional offset. This provides comprehensive positional reference information for the accurate determination of the subsequent connection status. Furthermore, the calculation method for the absolute values ​​of the differences is simple and efficient, enabling rapid quantification of positional deviations without being affected by the positive or negative coordinate directions. This makes it suitable for real-time processing scenarios in vehicle-mounted electronic control units, improving the real-time performance and accuracy of trailer connection status determination.

[0066] As one possible implementation, in some embodiments, calculating the rotational angle deviation between the trailer connector of the tractor vehicle and the trailer connector of the current towed load includes: calculating the angle between the first centerline direction and the second centerline direction; and obtaining the rotational angle deviation based on the angle.

[0067] Specifically, electronic control calculates the angle between the first central axis direction and the second central axis direction, that is, it calculates the spatial angle between the two direction vectors. θ This included angle value can be used as a rotation angle deviation to quantitatively characterize the degree of angular offset between the two trailer connectors in spatial attitude. That is:

[0068] in, The direction vector is the direction of the second central axis. It is the direction vector of the first central axis.

[0069] Therefore, by calculating the angle between the first and second centerline directions to obtain the rotation angle deviation, the angular offset of the two trailer connectors in spatial attitude can be intuitively expressed as a single quantitative index. This index directly reflects the degree of orientation consistency between the two; the smaller the angle, the higher the attitude matching degree of the two trailer connectors. The angle calculation is based on the dot product operation of direction vectors, which is mathematically mature and stable, computationally efficient, and suitable for real-time processing in the embedded environment of vehicle electronic control units, providing reliable attitude reference information for accurate determination of subsequent connection status.

[0070] In step S103, the towing connection status between the tractor vehicle and the current towed load is determined based on the type, position deviation, and rotation angle deviation of the current towed load.

[0071] In other words, the electronic control unit can comprehensively determine the towing connection status between the towing vehicle and the current towing load based on the identified type of the current towed load, as well as the calculated positional deviations (including longitudinal, lateral, and vertical deviations) and rotational angle deviations. Specifically, the electronic control unit can compare the longitudinal, lateral, vertical, and rotational angle deviations with their respective preset thresholds. The included angle θ directly reflects the angular deviation between the two towing connectors; if the angle is too large, mechanical docking cannot be completed. This application embodiment pre-sets reference thresholds for various deviations to determine the alignment status: a lateral deviation Δx not exceeding 2mm indicates that the left-right offset is within the mechanically permissible range; a longitudinal deviation Δy not exceeding 2mm indicates that the front-back offset is within the mechanically permissible range; a height deviation Δz not exceeding 1.5mm indicates that the vertical height offset is within the mechanically permissible range; and an angular deviation θ not exceeding 3° indicates that the angles of the two towing connectors match and there is no tilt. When all the above deviations meet their respective threshold conditions, the electronic control unit can determine that the two trailer connectors are aligned, that is, the towing vehicle and the current towed load are connected in place, and output a "connected" signal; if any deviation exceeds the corresponding threshold, it is determined that the connection is not in place, and an "not connected" signal is output, along with the specific deviation direction information to assist the driver in adjusting the position of the trailer connectors of the towed vehicle until all deviations meet the alignment conditions.

[0072] The electronic control unit (ECU) executes corresponding control strategies based on the identified towed load type and the alignment of the tow hooks. When the current towed load is a motorhome, if either the trailer hitch or the connector is not properly connected, the ECU immediately sends an alarm command to the warning module. The warning module then displays a text warning message on the screen and activates a voice alarm. When the current towed load is a trailer flatbed, if the trailer hitch is not properly connected, the ECU also immediately sends an alarm command to the warning module, triggering a screen display and a voice alarm. When all deviations are within the threshold range, the ECU does not output any alarm command to the warning module, and the user can proceed with the trip after successfully completing the towing connection.

[0073] Furthermore, to improve the accuracy of recognition and the system's anti-interference capability, this application also provides various algorithm and parameter optimization strategies. For camera recognition, a multi-frame fusion judgment strategy can be adopted, that is, continuously acquiring the recognition results of 5-10 frames of images. If the recognition results of most frames are consistent, the result is determined as the final output to avoid single-frame misjudgment. Simultaneously, the model output results are further verified by combining the size threshold of the towed object (e.g., RV height not less than 2m, trailer height not exceeding 1m), further reducing the false recognition rate. In addition, the model's dataset is regularly updated via OTA (Over-The-Air) online upgrades to continuously improve the recognition capability for new RV or trailer models. Regarding millimeter-wave radar anti-interference, Doppler velocity filtering is used to filter out surrounding objects, pedestrians, and other dynamic interference targets, and the Region of Interest (ROI) is dynamically updated. When the towed vehicle approaches the towing vehicle, the ROI range is reduced in real time, allowing the system to focus only on the docking area and process only the point cloud data within that area, thereby effectively reducing computational load and improving perception accuracy and system response speed.

[0074] The vehicle towing connection identification method proposed in this application determines the type of the current towed load based on an image stream, and determines the first centroid coordinates and first central axis direction of the towing vehicle's towing connector, as well as the second centroid coordinates and second central axis direction of the current towed load's towing connector based on point cloud data. Based on the first centroid coordinates, first central axis direction, second centroid coordinates, and second central axis direction, the method calculates the positional deviation and rotational angle deviation between the towing vehicle's towing connector and the current towed load's towing connector. Based on the type of the current towed load, positional deviation, and rotational angle deviation, the towing connection status between the towing vehicle and the current towed load is determined. This solves the problem of related technologies being unable to automatically determine whether the actual towing connection is properly in place, which poses a safety hazard, and improves the safety of towing operations.

[0075] Next, the vehicle trailer connection identification device according to the embodiments of this application is described with reference to the accompanying drawings.

[0076] Figure 3 This is a block diagram of a vehicle trailer connection identification device according to an embodiment of this application.

[0077] like Figure 3 As shown, the vehicle trailer connection identification device 10 includes: a first determination module 100, a calculation module 200, and a second determination module 300.

[0078] The first determining module 100 is used to determine the type of the current towed load based on the image stream within the first preset perimeter of the tractor vehicle, and to determine the first centroid coordinates and the first central axis direction of the towed vehicle's tow connector, as well as the second centroid coordinates and the second central axis direction of the tow connector of the current towed load, based on the point cloud data within the second preset perimeter of the tractor vehicle. The calculation module 200 is used to calculate the positional deviation and rotational angle deviation between the trailer connector of the tractor vehicle and the trailer connector of the current trailer load based on the first centroid coordinates, the first center axis direction, the second centroid coordinates, and the second center axis direction. The second determining module 300 is used to determine the towing connection status between the tractor vehicle and the current towing load based on the type, position deviation, and rotation angle deviation of the current towing load.

[0079] Optionally, in some embodiments, the first centroid coordinates of the trailer connector of the tractor vehicle are determined. The first determining module 100 is specifically used for: Clustering is performed on the point cloud data to generate the first point cloud cluster corresponding to the trailer connector of the tractor vehicle; Calculate the arithmetic mean of the spatial coordinates of all points in the first cloud cluster to obtain the coordinates of the first centroid.

[0080] Optionally, in some embodiments, the first centerline direction of the trailer connector of the tractor vehicle is determined, and the first determining module 100 is specifically used for: Calculate the first spatial covariance matrix of the first point cloud cluster; The first spatial covariance matrix is ​​decomposed into eigenvalues ​​to obtain the first set of eigenvectors. The direction of the eigenvector corresponding to the largest eigenvalue in the first set of eigenvectors is taken as the direction of the first central axis.

[0081] Optionally, in some embodiments, the second centroid coordinates of the trailer connector of the current towed load are determined. The first determining module 100 is specifically used for: Cluster the point cloud data to generate the second point cloud cluster corresponding to the tow connector of the current towed load; Calculate the arithmetic mean of the spatial coordinates of all points in the second point cloud cluster to obtain the coordinates of the second centroid.

[0082] Optionally, in some embodiments, the second centerline direction of the trailer connector of the current towed load is determined, and the first determining module 100 is specifically used for: Calculate the second spatial covariance matrix of the second point cloud cluster; The second spatial covariance matrix is ​​decomposed into eigenvalues ​​to obtain the second set of eigenvectors. The direction of the eigenvector corresponding to the largest eigenvalue in the second set of eigenvectors is taken as the direction of the second central axis.

[0083] Optionally, in some embodiments, the positional deviation between the trailer connector of the tractor vehicle and the trailer connector of the current towed load is calculated. The calculation module 200 is specifically used for: Calculate the absolute value of the difference between the coordinates of the second centroid and the coordinates of the first centroid in each coordinate axis direction to obtain the longitudinal deviation, lateral deviation and vertical deviation respectively; The positional deviation is obtained based on the longitudinal deviation, lateral deviation, and vertical deviation.

[0084] Optionally, in some embodiments, the calculation module 200 calculates the rotational angle deviation between the trailer connector of the tractor vehicle and the trailer connector of the current towed load. Specifically, this is used for: Calculate the angle between the direction of the first central axis and the direction of the second central axis; The rotation angle deviation is obtained from the included angle.

[0085] It should be noted that the foregoing explanation of the vehicle towing connection identification method embodiment also applies to the vehicle towing connection identification device of this embodiment, and will not be repeated here.

[0086] The vehicle trailer connection identification device proposed in this application determines the type of the current trailer load based on an image stream, and determines the first centroid coordinates and first central axis direction of the trailer connector of the tractor vehicle, as well as the second centroid coordinates and second central axis direction of the trailer connector of the current trailer load, based on point cloud data. Based on the first centroid coordinates, first central axis direction, second centroid coordinates, and second central axis direction, the device calculates the positional deviation and rotational angle deviation between the trailer connector of the tractor vehicle and the trailer connector of the current trailer load. Based on the type of the current trailer load, the positional deviation, and the rotational angle deviation, the device determines the trailer connection status between the tractor vehicle and the current trailer load. This solves the problem of related technologies being unable to automatically determine whether the actual trailer connection is properly in place, which poses a safety hazard, and improves the safety of trailer driving.

[0087] Figure 4 A schematic diagram of the structure of a vehicle provided in an embodiment of this application. The vehicle may include: The memory 401, the processor 402, and the computer program stored on the memory 401 and capable of running on the processor 402.

[0088] When the processor 402 executes the program, it implements the vehicle trailer connection identification method provided in the above embodiments.

[0089] Furthermore, the vehicle also includes: Communication interface 403 is used for communication between memory 401 and processor 402.

[0090] The memory 401 is used to store computer programs that can run on the processor 402.

[0091] The memory 401 may include high-speed RAM (Random Access Memory) memory, and may also include non-volatile memory, such as at least one disk storage.

[0092] If the memory 401, processor 402, and communication interface 403 are implemented independently, then the communication interface 403, memory 401, and processor 402 can be interconnected via a bus to complete communication between them. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 4 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0093] Optionally, in a specific implementation, if the memory 401, processor 402, and communication interface 403 are integrated on a single chip, then the memory 401, processor 402, and communication interface 403 can communicate with each other through an internal interface.

[0094] Processor 402 may be a CPU (Central Processing Unit), an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of this application.

[0095] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the above-described vehicle trailer connection identification method.

[0096] This application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the vehicle trailer connection identification method described above.

[0097] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0098] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0099] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A method for identifying vehicle trailer connections, characterized in that, Includes the following steps: The type of the current towed load is determined based on the image stream within the first preset perimeter of the tractor vehicle, and the first centroid coordinates and first central axis direction of the towed connector of the tractor vehicle, as well as the second centroid coordinates and second central axis direction of the towed connector of the current towed load, are determined based on the point cloud data within the second preset perimeter of the tractor vehicle. Based on the first centroid coordinates, the first centerline direction, the second centroid coordinates, and the second centerline direction, calculate the positional deviation and rotational angle deviation between the trailer connector of the tractor vehicle and the trailer connector of the current towed load; The towing connection status between the tractor vehicle and the current towed load is determined based on the type of the current towed load, the position deviation, and the rotation angle deviation.

2. The method according to claim 1, characterized in that, Determining the first centroid coordinates of the trailer connector of the tractor vehicle includes: The point cloud data is clustered to generate the first point cloud cluster corresponding to the trailer connector of the tractor vehicle; The arithmetic mean of the spatial coordinates of all points in the first point cloud cluster is calculated to obtain the coordinates of the first centroid.

3. The method according to claim 2, characterized in that, Determining the direction of the first centerline of the trailer connector of the tractor vehicle includes: Calculate the first spatial covariance matrix of the first point cloud cluster; The first spatial covariance matrix is ​​decomposed into eigenvalues ​​to obtain a first set of eigenvectors. The direction of the eigenvector corresponding to the largest eigenvalue in the first set of eigenvectors is taken as the direction of the first central axis.

4. The method according to claim 1, characterized in that, Determine the second centroid coordinates of the trailer connector of the current towed load, including: The point cloud data is clustered to generate a second point cloud cluster corresponding to the trailer connector of the current trailer load; The arithmetic mean of the spatial coordinates of all points in the second point cloud cluster is calculated to obtain the coordinates of the second centroid.

5. The method according to claim 4, characterized in that, Determine the direction of the second centerline of the trailer connection for the current towed load, including: Calculate the second spatial covariance matrix of the second point cloud cluster; The second spatial covariance matrix is ​​decomposed into eigenvalues ​​to obtain a second set of eigenvectors. The direction of the eigenvector corresponding to the largest eigenvalue in the second set of eigenvectors is taken as the direction of the second central axis.

6. The method according to claim 1, characterized in that, Calculating the positional deviation between the trailer connector of the tractor vehicle and the trailer connector of the current towed load includes: Calculate the absolute value of the difference between the second centroid coordinates and the first centroid coordinates in each coordinate axis direction to obtain the longitudinal deviation, lateral deviation and vertical deviation respectively; The positional deviation is obtained based on the longitudinal deviation, the lateral deviation, and the vertical deviation.

7. The method according to claim 1 or 6, characterized in that, Calculating the rotation angle deviation between the trailer connector of the tractor vehicle and the trailer connector of the current towed load includes: Calculate the angle between the direction of the first central axis and the direction of the second central axis; The rotation angle deviation is obtained based on the included angle.

8. A vehicle trailer connection identification device, characterized in that, include: The first determining module is used to determine the type of the current towed load based on the image stream within a first preset perimeter of the tractor vehicle, and to determine the first centroid coordinates and the first central axis direction of the towed connector of the tractor vehicle, as well as the second centroid coordinates and the second central axis direction of the towed connector of the current towed load, based on the point cloud data within a second preset perimeter of the tractor vehicle. The calculation module is used to calculate the positional deviation and rotational angle deviation between the trailer connector of the tractor vehicle and the trailer connector of the current trailer load based on the first centroid coordinates, the first centerline direction, the second centroid coordinates, and the second centerline direction. The second determining module is used to determine the towing connection status between the tractor vehicle and the current towing load based on the type of the current towing load, the position deviation, and the rotation angle deviation.

9. A vehicle, characterized in that, include: A memory, a processor, and a computer program stored in the memory and capable of running on the processor, the processor executing the program to implement the vehicle trailer connection identification method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the vehicle trailer connection identification method as described in any one of claims 1-7.

Images