Flexible pull detection method and system based on monocular vision

Abstract

The invention provides a flexible trailer detection method and system based on monocular vision. The method comprises the steps that monocular vision information and 3D point cloud information of a trailer are obtained respectively; information fusion is carried out on the monocular vision information and the 3D point cloud information to obtain image information, and the image information at least comprises posture, position and motion state information of the flexible pull part; performing feature extraction and target detection on the image information to obtain target information of the flexible pull part; monocular direction estimation of a front pose is carried out on the target information, and pose information of the flexible pull part is obtained; and performing path planning and decision adjustment based on the pose information in combination with the dynamic change of the front vehicle, and generating a control instruction to control vehicle tracking. According to the invention, the high-efficiency detection of the flexible towed object is realized by using the monocular vision technology, and the safety and the operation efficiency of the towed vehicle are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of monocular vision, and in particular to a flexible trailer detection method and system based on monocular vision. Background Art

[0002] With the development of logistics and transportation industries, flexible trailer vehicles play an important role in cargo transportation. However, traditional flexible trailer detection systems have some limitations, such as relying on a large number of sensors, complex installation locations, high costs, and strict equipment requirements. Current flexible trailer detection systems mainly rely on sensors such as lidar and camera arrays to monitor and detect trailer vehicles. Although these sensors can provide a certain degree of accuracy, their installation and maintenance costs are high, and there are certain blind spots and errors in the detection process of flexible trailer vehicles.

[0003] Therefore, the existing flexible trailer detection system usually has the following technical defects:

[0004] (1) High cost: Traditional flexible towed vehicle detection systems rely on multiple sensors, which have complex installation locations and increase installation and maintenance costs;

[0005] (2) Large blind spots and errors: The sensor layout in traditional systems may have blind spots or dead angles, resulting in certain errors in the monitoring and detection of flexible towed objects, reducing the accuracy and reliability of the system.

[0006] (3) Installation location restrictions: The sensor installation location of traditional systems is usually restricted and needs to be installed in a fixed or specific location, which may limit the applicability and flexibility of the system in different scenarios and environments.

[0007] The search found:

[0008] The Chinese invention patent application with application number 202211667015.4 and application date 2022-12-22 is entitled "A Collision Detection Method, Device, Electronic Device and Storage Medium", in which the method includes obtaining the obstacle motion trajectory, establishing a trailer motion model, feature matching, detecting whether obstacles intersect, performing convex optimization processing on intersecting obstacles, marking the collision area of ​​the target obstacle and other steps to predict the collision of the trailer. Its technical features are to obtain point cloud information, determine the obstacle contour and motion trajectory, establish a trailer motion model obtained by a vehicle-mounted radar sensor, consider factors such as the tractor speed and angle, perform feature matching, take the intersecting obstacles as target obstacles, and determine the hinge points, perform convex optimization processing on the target obstacles, and obtain the collision area. This method can be used to formulate driving routes and issue alarms to improve vehicle safety. This method cannot take into account the complex and changeable actual road environment and sudden accidents, and the trailer motion model may have certain simplifications and assumptions, and cannot fully and accurately reflect the actual situation.

[0009] The Chinese invention patent application "Gap Measurement for Vehicle Escort" with application number 202211662662.6 and application date 2017-10-26 describes a variety of methods, controllers and algorithms for identifying specific vehicles (such as the rear of a formation companion) and tracking their tails. This technology can be used in combination with various distance measurement technologies, is suitable for vehicle formation and escort systems, and describes a technology for fusing different vehicle sensor data for at least partially automatic control. The described method is suitable for use in combination with various vehicle control applications, including formation, escort and connected driving applications. The technology also involves the use of a distance measurement module to measure and maintain the desired gap between the leading vehicle and the following vehicle. However, this technology still fails to solve the problems of tracking errors, environmental interference and algorithm optimization for complex vehicle formations or escort situations, and in practical applications, integration and compatibility with other vehicle systems also need to be considered.

[0010] Therefore, how to accurately predict the collision of trailers in complex and changeable actual road environments, take into account unexpected situations, and optimize the accuracy and adaptability of trailer motion models, has become an urgent problem to be solved in this field. At present, no collision detection method or report similar to the present invention has been found, nor has any relevant domestic and foreign information that can fully solve the above problems been collected. Summary of the invention

[0011] In view of the above-mentioned deficiencies in the flexible trailer vehicle detection and control technology in the prior art, an object of the present invention is to provide a flexible trailer detection method and system based on monocular vision.

[0012] To achieve the above object, the present invention is implemented through the following technical solutions:

[0013] According to one aspect of the present invention, a flexible trailer detection method based on monocular vision is provided, comprising:

[0014] Obtain monocular vision information and 3D point cloud information of the trailer vehicle respectively;

[0015] Fusing the monocular vision information and the 3D point cloud information to obtain image information, wherein the image information at least includes: posture, position and motion state information of the flexible trailer part;

[0016] Performing feature extraction and target detection on the image information to obtain target information of the flexible trailer part;

[0017] Performing monocular direction estimation of the forward posture of the target information to obtain the posture information of the flexible trailer part;

[0018] Based on the position information and in combination with the dynamic changes of the vehicle ahead, path planning and decision adjustment are performed to generate control instructions to control vehicle tracking.

[0019] Preferably, the step of respectively acquiring monocular vision information and 3D point cloud information of the trailer vehicle comprises:

[0020] A monocular vision camera is used to capture real-time image data of the trailer vehicle, wherein the image data is used to display the shape, contour and texture information of the flexible trailer part, and obtain preliminary morphology and position information of the flexible trailer part, that is, to obtain monocular vision information;

[0021] A laser radar sensor is used to obtain point cloud data of the trailer vehicle. The point cloud data is used to describe the three-dimensional structure, spatial position and speed information of the flexible trailer part, that is, to obtain 3D point cloud information, which provides auxiliary depth information and spatial positioning reference for the image data.

[0022] Preferably, fusing the monocular vision information and the 3D point cloud information comprises:

[0023] The monocular vision information and the 3D point cloud information are deeply fused and processed to obtain the current state and behavior information for accurately identifying the flexible trailer part, that is, to obtain image information.

[0024] Preferably, the performing feature extraction and target detection on the image information includes:

[0025] Providing a pre-trained deep learning model, using the model to extract features from the image information, identify the trailer part, and estimate the direction of the leading vehicle;

[0026] The trailer part is subjected to contour extraction and target detection, and combined with the estimation result of the front vehicle direction, the flexible trailer part is preliminarily positioned and morphologically analyzed to obtain target information.

[0027] Preferably, the step of performing monocular direction estimation of the forward posture of the target information to obtain the posture information of the flexible trailer part includes:

[0028] A monocular direction estimation model is provided, the target information is used as the input of the model, and the key feature points of the target information are extracted; wherein the monocular direction estimation model estimates the direction of each monomer in the target information, and then forms a constraint optimization between directions by combining the directions of each rigid body with the intersection constraints between rigid bodies, thereby obtaining the direction on the arc of the towing, that is, obtaining the key feature points of the target information;

[0029] Matching the key feature points with the prior pose information of the vehicle ahead, and evaluating the motion state of the target information;

[0030] According to the motion state of the target information, the estimated position information of the flexible trailer part is calculated, including: position coordinates (x, y, z) and posture (roll, pitch, yaw);

[0031] The estimated position and posture information of the flexible towing part is smoothed, and the final position and posture information of the flexible towing part is output.

[0032] Preferably, the path planning and decision adjustment based on the posture information and in combination with the dynamic changes of the vehicle ahead include:

[0033] Obtaining the length, angle and speed parameter information of the flexible trailer part according to the posture information, and adjusting the driving trajectory and speed of the vehicle in real time based on the parameter information so that the flexible trailer part remains stable during driving, and / or issuing an early warning according to the posture state;

[0034] Combined with the current road conditions and the dynamic changes of the vehicle ahead, path planning and evaluation are performed to select the optimal path and adjust control decisions to generate command information to control vehicle tracking.

[0035] Preferably, it also includes:

[0036] The monocular vision information, 3D point cloud information, image information, target information, position information, path information and / or decision information are sent to other terminals in real time for monitoring, management and / or information sharing and feedback.

[0037] According to another aspect of the present invention, a flexible trailer detection system based on monocular vision is provided, comprising: a monocular vision camera, an image processing and analysis module, a decision and control module, and an interactive feedback module; wherein:

[0038] A monocular vision detection module, which is used to obtain monocular vision information and 3D point cloud information of the trailer vehicle respectively;

[0039] An image processing and analysis module, which is used to fuse the monocular visual information and the 3D point cloud information to obtain image information, wherein the image information at least includes: posture, position and motion state information of the flexible trailer part; perform feature extraction and target detection on the image information to obtain target information of the flexible trailer part; perform monocular direction estimation of the forward posture on the target information to obtain posture information of the flexible trailer part;

[0040] A decision and control module performs path planning and decision adjustment based on the position information and in combination with the dynamic changes of the vehicle ahead, and generates control instructions to control vehicle tracking.

[0041] Preferably, the monocular vision detection module comprises: a monocular vision camera and a laser radar sensor; wherein:

[0042] The monocular vision camera is used to capture real-time image data of the trailer vehicle, and the image data is used to display the shape, contour and texture information of the flexible trailer part, and obtain the preliminary form and position information of the flexible trailer part, that is, to obtain monocular vision information;

[0043] The laser radar sensor is used to obtain point cloud data of the trailer vehicle, and the point cloud data is used to describe the three-dimensional structure, spatial position and speed information of the flexible trailer part, that is, to obtain 3D point cloud information, providing auxiliary depth information and spatial positioning reference for the image data.

[0044] Preferably, it also includes any one or more of the following:

[0045] -Interactive feedback module, which is used to share information and / or provide information feedback with other terminals;

[0046] - Status monitoring module, which is used to monitor the key parameters of the vehicle in real time, including: the position information of the flexible trailer part and the driving trajectory information of the trailer vehicle;

[0047] -Data processing and analysis module, used for data analysis and processing of key vehicle parameters, and early warning of potential safety hazards;

[0048] - An adaptive adjustment module, which is used to dynamically adjust the parameters of the image processing and analysis module and / or the path and decision of the decision and control module according to the actual operation of the vehicle;

[0049] - A user operation module, which is respectively connected to the image processing and analysis module and the decision and control module for real-time data interaction, real-time status display, operation instruction input and early warning information prompt;

[0050] -Trailer tracking optimization module, which optimizes the tracking accuracy of the flexible trailer part according to the analysis and processing results obtained in the data processing and analysis module, so that the flexible trailer part remains stable during driving;

[0051] -Remote monitoring module, used to remotely monitor and manage the running status of the vehicle, and to display the tracking status of the flexible trailer part, early warning information and / or statistical data of various information in real time;

[0052] -Communication module, used for information transmission and command issuance between the monocular vision detection module, the flexible trailer vehicle and the remote monitoring module.

[0053] Due to the adoption of the above technical solution, the present invention has at least one of the following beneficial effects compared with the prior art:

[0054] The present invention realizes accurate detection and tracking of the flexible trailer part by combining a monocular vision camera and high-precision laser radar data assistance, thereby improving the safety and stability of the trailer vehicle in complex road environments.

[0055] The present invention realizes stable control of the flexible trailer vehicle through the coordinated work of the image processing and analysis module and the decision and control module, and improves the integration and compatibility, reliability and efficiency of the trailer system.

[0056] The present invention utilizes interactive feedback technology to enhance the interactive experience between the user and the trailer system, optimizes the adaptability of the trailer motion model, and enables the user to more comprehensively understand and control the flexible trailer vehicle.

[0057] The present invention has broad application prospects in practical applications and can be applied to various scenarios requiring towing transportation, such as logistics, agriculture, military and other fields, providing efficient and safe towing solutions for related industries.

[0058] Of course, any product implementing the present invention does not necessarily need to achieve all the advantages mentioned above at the same time. But in general, the present invention uses advanced vision and radar technology, combined with efficient image processing and control algorithms, to achieve accurate detection and stable control of the flexible trailer part, providing strong support for the development of related industries. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0060] Figure 1 The present invention is a flowchart of a flexible trailer detection method based on monocular vision in a preferred embodiment of the present invention.

[0061] Figure 2 It is a structural block diagram of a flexible trailer detection system based on monocular vision in a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0062] The following is a detailed description of the embodiments of the present invention: This embodiment is implemented on the premise of the technical solution of the present invention, and a detailed implementation method and a specific operation process are given. It should be pointed out that for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention.

[0063] An embodiment of the present invention provides a flexible trailer detection method based on monocular vision, which uses high-precision laser radar technology as an auxiliary to provide accurate reference for monocular vision technology, thereby improving the safety and stability of flexible trailer vehicles in complex road environments, achieving efficient and accurate detection and intelligent control of flexible trailer vehicles, and ensuring the safety and operating efficiency of trailer vehicles through information interaction with drivers or other related systems through interactive feedback technology. It is particularly suitable for flexible trailer vehicle detection and management under complex road environments and variable working conditions. Through data processing and analysis, trailer tracking optimization and remote monitoring, the tracking accuracy of flexible trailer vehicles can be effectively improved, the safety hazards caused by inaccurate tracking can be reduced, and the driving safety and operating efficiency of flexible trailer vehicles can be improved.

[0064] Specifically, Figure 1 As shown, the flexible trailer detection method based on monocular vision provided in this embodiment may include the following operations:

[0065] S1, obtains monocular vision information and 3D point cloud information of the trailer vehicle respectively;

[0066] S2, fusing the monocular vision information and the 3D point cloud information to obtain image information, wherein the image information at least includes: posture, position and motion state information of the flexible trailer part;

[0067] S3, extracting features and detecting targets from the image information to obtain target information of the flexible trailer part;

[0068] S4, performing monocular direction estimation of the forward posture of the target information to obtain the posture information of the flexible trailer part;

[0069] S5, based on the posture information and combined with the dynamic changes of the vehicle in front, path planning and decision adjustment are performed to generate control instructions to control vehicle tracking.

[0070] In some preferred embodiments, the above S1 may further include:

[0071] S11, using a monocular vision camera to capture real-time image data of the trailer vehicle, the image data is used to display the shape, contour and texture information of the flexible trailer part, and obtain preliminary shape and position information of the flexible trailer part, that is, obtain monocular vision information;

[0072] S12, using a laser radar sensor to obtain point cloud data of the trailer vehicle, the point cloud data is used to describe the three-dimensional structure, spatial position and speed information of the flexible trailer part, that is, to obtain 3D point cloud information, providing auxiliary depth information and spatial positioning reference for the image data.

[0073] In some preferred embodiments, the above S2 may further include:

[0074] Multimodal fusion technology is used to deeply fuse and process (integrate) monocular vision information and 3D point cloud information to improve recognition accuracy and obtain the current state and behavior information for accurately identifying the flexible trailer part, that is, image information.

[0075] In some preferred implementations, the above S3 may further include:

[0076] S31, providing a pre-trained deep learning model, and using the model to extract features from image information, identify trailer parts, and estimate the direction of the leading vehicle;

[0077] S32, performing contour extraction and target detection on the trailer part, and combining the estimation result of the front vehicle direction, performing preliminary positioning and morphological analysis on the flexible trailer part to obtain target information.

[0078] In this preferred embodiment, the morphological analysis specifically includes:

[0079] S321, extracting edge and contour features of the flexible trailer part to evaluate its shape and size;

[0080] S322, analyzing the geometric shape, proportion and deformation characteristics of the flexible trailer part to identify possible abnormalities or damage;

[0081] S323, combining the target detection result, evaluating the motion characteristics of the trailer, such as acceleration, speed change, moving posture direction, etc.

[0082] In some preferred embodiments, the above S4 may further include:

[0083] S41, providing a monocular direction estimation model, taking the target information as the input of the model, and extracting the key feature points of the target information; wherein, the traditional monocular estimation only estimates the direction of a certain rigid body, and the monocular direction estimation model used in this step estimates the direction of each monomer of the target information (i.e., the flexible towing part), and then forms a constraint optimization between directions by combining the directions of each rigid body with the intersection constraints between the rigid bodies, thereby obtaining the direction on the arc of the towing, that is, a plurality of rigid bodies with their own directions combined with the articulation between them form an arc direction, that is, obtaining the key feature points of the target information; further preferably:

[0084] Estimating the directions of each monomer of the target information (i.e., the flexible towing part) may further include the following operations:

[0085] The direction of each unit in the flexible trailer is estimated; here, "unit" refers to each rigid body module in the trailer system; the monocular direction estimation model can estimate the direction of the rigid body through the visual sensor;

[0086] By combining the directions of the respective rigid bodies with the intersection constraints between the rigid bodies, the constraint optimization between the directions is formed, thereby obtaining the direction of the towing arc, which can further include the following operations:

[0087] Combined with intersection constraints: Consider the connection relationship between each rigid body; the monomers of the flexible towing system are connected by hinged structures, which means that there is a certain constraint relationship between them; mathematically, this relationship can be described by establishing a direction constraint equation, that is, the direction of each rigid body depends not only on its own characteristics, but also on the connection relationship between it and other rigid bodies;

[0088] Direction constraint optimization: After combining the independent directions of each rigid body and the intersection constraints between them, the system will perform direction constraint optimization; the purpose of this step is to solve the overall direction of the flexible trailer system while satisfying the directions and intersection constraints of each monomer; in this way, the directional characteristics of the trailer part in arc can be obtained.

[0089] To obtain the key feature points of the target information, the following operations can be further performed

[0090] Extraction of key feature points: Through direction estimation and constraint optimization, the overall arc direction of the flexible trailer can be determined. These direction information can be regarded as the key feature points of the target information. The acquisition of key feature points is very important for the subsequent target tracking and positioning process because they describe the motion characteristics and posture of the flexible trailer in space.

[0091] S42, matching the key feature points with the prior pose information of the vehicle ahead, using an optical flow method or a feature point tracking algorithm and evaluating the motion state of the target information; wherein the prior pose information of the vehicle ahead is directly detected in 3D by the monocular of the vehicle;

[0092] S43, according to the motion state of the target information, the estimated position information of the flexible trailer part is calculated through triangulation and geometric transformation, including: position coordinates (x, y, z)) and attitude (roll, pitch, yaw);

[0093] S44, using optimization algorithms such as Kalman filtering or particle filtering to smooth the estimated posture information of the flexible trailer part to reduce the impact of noise and improve estimation accuracy, and output the final posture information of the flexible trailer part to provide a basis for subsequent path planning and decision adjustments.

[0094] In this preferred embodiment, the optimization of posture information is achieved: the obtained posture information is optimized by applying algorithms such as Kalman filtering or particle filtering to reduce noise and uncertainty and improve the accuracy of estimation; dynamic environment adaptation is also achieved: the posture estimation is adjusted in real time based on the speed, acceleration and road condition information of the vehicle in front to ensure stable tracking in complex and dynamic environments.

[0095] In some preferred implementations, the above S5 may further include:

[0096] S51, obtaining the length, angle and speed parameter information of the flexible trailer part according to the posture information, and adjusting the driving track and speed of the vehicle in real time based on the parameter information, so that the flexible trailer part remains stable during driving; and further issuing an early warning according to the posture state;

[0097] S52, combining the current road conditions and the dynamic changes of the preceding vehicle, performs path planning and evaluation, selects the optimal path and adjusts the control decision, and generates command information to control vehicle tracking.

[0098] In some preferred embodiments, the above method may further include:

[0099] S6, sending monocular vision information, 3D point cloud information, image information, target information, position information, path information and / or decision information to other terminals in real time for monitoring, management and / or information sharing and feedback.

[0100] The above-mentioned embodiment of the present invention provides a flexible trailer detection method based on monocular vision. When tracking and making decisions on the flexible trailer, the monocular vision camera is first used to capture real-time image data to reveal details such as the shape, contour and texture of the flexible trailer. At the same time, with the help of point cloud information generated by sensors such as high-precision lidar, the three-dimensional structure and spatial position of the flexible trailer are accurately depicted. Through the deep fusion and processing of information, the system can accurately identify the current state and behavior of the flexible trailer.

[0101] When a flexible trailer vehicle is driving, it needs to respond to dynamic changes such as acceleration, deceleration, lane changes, and changes in road conditions. Relying on monocular vision data, the flexible trailer tracking is adjusted in real time to ensure stable and safe driving. Combined with the orientation estimation in the monocular direction estimation, the tracking path is accurately adjusted according to the changes in the posture of the vehicle in front, and the dynamics of the vehicle in front are flexibly adapted to ensure safe and stable driving; at the same time, multiple candidate tracking paths are evaluated, considering factors including tracking accuracy, safety, stability of the flexible trailer part, and driving efficiency. During the evaluation process, external factors such as road conditions and traffic environment are also considered to formulate a more comprehensive and reasonable decision-making strategy.

[0102] Based on the evaluation results, the system will select the optimal tracking path and formulate corresponding decision strategies, including adjusting the acceleration and deceleration of the control vehicle, coordinating the vehicle's steering action, etc. These decisions are designed to ensure that the flexible trailer vehicle maintains high accuracy, high safety and high convenience in the process of tracking the vehicle in front.

[0103] An embodiment of the present invention provides a flexible trailer detection system based on monocular vision, which uses a high-precision laser radar as an auxiliary to provide an accurate reference for a monocular vision camera, thereby improving the safety and stability of flexible trailer vehicles in complex road environments, achieving efficient and accurate detection and intelligent control of flexible trailer vehicles, and ensuring the safety and operating efficiency of trailer vehicles through information interaction with drivers or other related systems through an interactive feedback module. It is particularly suitable for flexible trailer vehicle detection and management under complex road environments and variable working conditions. Through the data processing and analysis module, the trailer tracking optimization module and the remote monitoring module, the tracking accuracy of flexible trailer vehicles can be effectively improved, the safety hazards caused by inaccurate tracking can be reduced, and the driving safety and operating efficiency of flexible trailer vehicles can be improved.

[0104] Specifically, Figure 2 As shown, the flexible trailer detection system based on monocular vision provided by this embodiment may include:

[0105] Monocular vision camera, image processing and analysis module, decision and control module and interactive feedback module; among which:

[0106] A monocular vision detection module, which is used to obtain monocular vision information and 3D point cloud information of the trailer vehicle respectively;

[0107] An image processing and analysis module is used to fuse monocular vision information and 3D point cloud information to obtain image information, which at least includes: posture, position and motion state information of the flexible trailer; perform feature extraction and target detection on the image information to obtain target information of the flexible trailer; perform monocular direction estimation of the forward posture of the target information to obtain the posture information of the flexible trailer;

[0108] The decision-making and control module performs path planning and decision adjustments based on the posture information and the dynamic changes of the vehicle in front, and generates control instructions to control vehicle tracking.

[0109] In some preferred embodiments, the monocular vision detection module may further include: a monocular vision camera and a laser radar sensor; wherein:

[0110] A monocular vision camera is used to capture real-time image data of the trailer vehicle. The image data is used to display the shape, contour and texture information of the flexible trailer part, and obtain the preliminary shape and position information of the flexible trailer part, that is, to obtain monocular vision information;

[0111] The laser radar sensor is used to obtain point cloud data of the trailer vehicle. The point cloud data is used to describe the three-dimensional structure, spatial position and speed information of the flexible trailer part, that is, to obtain 3D point cloud information, providing auxiliary depth information and spatial positioning reference for the image data.

[0112] In some preferred embodiments, the above system may further include any one or more of the following modules:

[0113] -Interactive feedback module, which is used to share information and / or provide information feedback with other terminals;

[0114] - Status monitoring module, which is used to monitor the key parameters of the vehicle in real time, including: the position information of the flexible trailer part and the driving trajectory information of the trailer vehicle;

[0115] -Data processing and analysis module, which is used to analyze and process the key parameters of the vehicle and to warn of possible safety hazards. The data analysis and processing mainly include: Anomaly detection: using anomaly detection algorithms to identify abnormal conditions in vehicle operation, such as sudden braking, sudden acceleration, and deviation from the track; Early warning mechanism: according to the analysis results, a threshold trigger mechanism is set to warn of possible safety hazards in real time and send alarm information to the driver or monitoring system;

[0116] - An adaptive adjustment module, which is used to dynamically adjust the parameters of the image processing and analysis module and / or the path and decision of the decision and control module according to the actual operation of the vehicle;

[0117] - A user operation module, which is respectively connected to the image processing and analysis module and the decision and control module for real-time data interaction, real-time status display, operation instruction input, and early warning information prompt;

[0118] - The trailer tracking optimization module optimizes the tracking accuracy of the flexible trailer part through advanced algorithms and strategies based on the analysis and processing results obtained in the data processing and analysis module (including the real-time posture information, driving trajectory, status monitoring data and safety assessment results of the flexible trailer part, which are used to guide subsequent tracking optimization), so that the flexible trailer part remains stable during driving; wherein, the advanced algorithms and strategies include: Kalman filtering: used to update the posture estimation of the flexible trailer part in real time, and provide more accurate posture information through noise modeling; model predictive control: adjust the control input according to the predicted future state to achieve smooth tracking of the flexible trailer; optimize the tracking in the trailer part through the above-mentioned whole process;

[0119] -Remote monitoring module, used to remotely monitor and manage the running status of the vehicle, and to display the tracking status of the flexible trailer part, early warning information and / or statistical data of various information in real time;

[0120] -Communication module, used for information transmission and command issuance between the monocular vision detection module, the flexible trailer vehicle and the remote monitoring module.

[0121] It should be noted that the steps in the method provided by the present invention can be implemented by using the corresponding components in the system, and those skilled in the art can refer to the technical solution of the system to implement the step flow of the method, and can also refer to the technical solution of the method to implement the composition of the system, that is, the embodiments in the system and the embodiments in the method can be understood as preferred examples of each other, which will not be elaborated here.

[0122] The working contents of each module of the flexible trailer detection system based on monocular vision provided by the above embodiment of the present invention are further described below.

[0123] The monocular vision camera is used to capture real-time image data of the trailer vehicle, with special attention paid to the deformation, position and motion state of the flexible trailer part, and to provide preliminary shape and position information of the trailer part.

[0124] The data from the high-precision LiDAR sensor is used as an aid to obtain the precise distance, speed and angle information of the trailer by emitting a laser beam and receiving the reflected signal, providing depth information and spatial positioning reference for the monocular vision camera. The LiDAR sensor is used to enhance the detection accuracy of the vehicle ahead. It is tightly integrated with other modules to achieve accurate measurement of key information such as the distance, speed, and direction of the vehicle ahead. This module complements the monocular vision camera to build an efficient and accurate vehicle ahead detection system.

[0125] The image processing and analysis module receives data from the monocular vision camera and high-precision lidar, and performs image processing and feature extraction through deep learning algorithms or monocular direction estimation. By performing feature extraction, target detection, and 3D pose estimation on the image, it can accurately identify and locate the flexible trailer part, thereby achieving efficient and accurate detection of the flexible trailer part.

[0126] The decision and control module performs path planning and decision adjustments based on the output results of the image processing and analysis module and the dynamic changes of the vehicle in front, and generates corresponding control instructions to perform corresponding control operations on the trailer vehicle, including but not limited to issuing warning signals, adjusting the driving speed, adjusting the posture of the trailer vehicle to stabilize the flexible trailer part, or triggering emergency braking and other safety measures.

[0127] The introduction of high-precision LiDAR data provides important auxiliary information for the monocular vision camera. Through the depth information and spatial positioning reference obtained by LiDAR, the image processing and analysis module can more accurately identify and locate the flexible trailer part, thereby improving the accuracy and reliability of the entire detection system. At the same time, the decision-making and control module can adjust the vehicle driving status in time according to real-time data and path planning to ensure the stable driving of the flexible trailer vehicle in complex road environments.

[0128] The interactive feedback module is responsible for interacting with the user, displaying real-time status information, receiving user operation instructions, and providing early warning information prompts to ensure that the user has full control and safe use of the flexible trailer vehicle. The existence of the interactive feedback module enables users to understand the vehicle status in real time, receive early warning information, and make necessary operational adjustments, further enhancing the security of the system and user experience. It specifically includes the following functions:

[0129] Feedback the test results, status information and control instructions of the flexible trailer vehicle to the user in real time through interface display, sound prompts, etc., so that the user can understand the operating status of the trailer vehicle in time and take corresponding measures;

[0130] Receive input instructions from users or other operators, such as adjusting detection parameters and control strategies, to meet detection requirements in different scenarios;

[0131] Exchange data and transmit instructions with other related systems or equipment to achieve information sharing and collaborative work. For example, cooperate with the navigation system to optimize the driving path and avoid unnecessary vibration or impact on the flexible trailer part.

[0132] The status monitoring module of the flexible trailer vehicle is used to monitor the key parameters such as the trajectory of the flexible trailer part and the position of the trailer vehicle in real time, and to warn of possible safety hazards through data analysis and processing.

[0133] The adaptive adjustment module can dynamically adjust the algorithm parameters of the image processing and analysis module or the decision strategy of the decision and control module according to the actual operation of the flexible trailer vehicle to improve the detection accuracy and response speed.

[0134] Through the design of the above structure and function, the flexible trailer detection system based on monocular vision can realize efficient and accurate detection and intelligent control of flexible trailer vehicles. Through the information interaction with the driver or other related systems through the interactive feedback module, the safety and operation efficiency of the trailer vehicle can be ensured. It is particularly suitable for the detection and management of flexible trailer vehicles in complex road environments and changeable working conditions.

[0135] The user operation module is connected to the flexible trailer vehicle and interacts with the monocular vision detection module, image processing and analysis module, and decision-making and control module in real time through wireless communication. At the same time, the user operation module provides functions such as real-time status display, operation command input, and early warning information prompts to ensure the user's comprehensive control and safe use of the flexible trailer vehicle.

[0136] The data processing and analysis module is responsible for receiving and processing the data from the monocular vision detection module, and accurately identifying and analyzing the real-time status of the flexible trailer part;

[0137] The trailer tracking optimization module optimizes the tracking accuracy of the flexible trailer part according to the analysis and processing results obtained in the data processing and analysis module, ensuring efficient and stable tracking of the flexible trailer part during driving;

[0138] The communication module is responsible for information transmission and command issuance between the monocular vision detection module, the flexible trailer vehicle and the remote monitoring module to ensure the real-time nature of the data and the accuracy of the commands;

[0139] The remote monitoring module is used to remotely monitor and manage the operating status of the flexible trailer vehicle, and to display the tracking status, warning information and statistical data of the flexible trailer part in real time, providing comprehensive monitoring and decision support for managers.

[0140] Through the above design and implementation, the tracking accuracy of flexible trailer vehicles can be effectively improved, the safety hazards caused by inaccurate tracking can be reduced, and the driving safety and operating efficiency of flexible trailer vehicles can be improved.

[0141] The image processing and analysis module and the decision and control module realize the control decision link and undertake the key task of issuing instructions. The monocular vision camera accurately captures the posture information of the front vehicle, and then makes a detailed judgment on the current state of the flexible trailer and the relative position of the front vehicle. Based on this information, a safe and efficient driving path and action strategy are formulated to maintain the stable connection of the flexible trailer and ensure the stability and safety of the vehicle.

[0142] The above-mentioned embodiment of the present invention is not only responsible for monitoring the changes in the vehicle's surrounding environment, but also pays special attention to the problem of multi-target vehicle management. Through monocular vision technology, the visual information of multiple targets such as flexible trailer vehicles and surrounding vehicles and obstacles is captured and processed in real time. In complex traffic environments, the system can accurately distinguish and detect each target, analyze its dynamic characteristics and interactive relationships, and provide comprehensive and accurate data support for path planning and decision-making.

[0143] The working process of the above-mentioned flexible trailer detection system based on monocular vision includes an initialization phase, a flexible trailer detection phase, and a decision and control phase. Among them:

[0144] During the initialization phase:

[0145] Perform initial configuration and calibration of the monocular vision camera and high-precision LiDAR to ensure that the camera can accurately capture real-time image data of the flexible trailer, the LiDAR can accurately obtain the depth information and spatial positioning data of the trailer, and that all modules can work normally and coordinate with each other.

[0146] The image processing and analysis module is parameterized to meet the flexible trailer detection requirements in different scenarios. The decision and control module also needs to be initially configured to ensure that it can perform path planning and decision adjustments based on the output results of the image processing and analysis module and the dynamic changes of the vehicle in front.

[0147] Finally, the above is tested to test the functions of the interactive feedback module to ensure that it can normally display real-time status information, receive user operation instructions, and provide early warning information prompts.

[0148] During the flexible towing detection phase:

[0149] First, the monocular vision camera starts and captures real-time image data of the flexible trailer part, providing preliminary trailer shape and position information. At the same time, the high-precision laser radar starts working, emitting laser beams and receiving reflected signals to obtain the precise depth, speed and angle information of the trailer part. This information provides important precise depth information and spatial positioning references for the image processing and analysis module, which helps to further improve the recognition and positioning accuracy of the flexible trailer part. The image processing and analysis module receives data from the monocular vision camera and laser radar, and uses a pre-trained deep learning model to perform feature extraction, trailer recognition and leading vehicle direction estimation. By performing contour extraction and morphological analysis on the trailer part in the image, combined with specific direction estimation and prediction, preliminary positioning and morphological analysis of the flexible trailer part, efficient image processing and feature extraction are performed to achieve accurate recognition and positioning of the flexible trailer part.

[0150] During the trailer control phase:

[0151] The decision and control module performs path planning and decision adjustments based on the output results of the image processing and analysis module, combined with the current road conditions and the dynamic changes of the preceding vehicle. By generating corresponding control instructions, the flexible trailer vehicle can be stably controlled. In addition, the decision and control module continuously monitors the status of the trailer part and makes real-time adjustments as needed to ensure the safety and stability of the trailer vehicle in complex road environments. Specifically, the decision and control module obtains parameters such as the length, angle and speed of the trailer part based on the posture information of the flexible trailer part, and adjusts the vehicle's driving trajectory and speed to ensure that the trailer part remains stable during driving. At the same time, the decision and control module also monitors the status of the trailer part in real time. Once an abnormal situation or potential risk is found, it will immediately take corresponding measures to make adjustments or issue early warning information.

[0152] During the entire control process, the interactive feedback module maintains real-time interaction with the user. It displays the real-time status information of the flexible trailer, such as length, angle, speed, etc., as well as the vehicle's driving trajectory and path planning information. The user can receive this information through the interactive feedback module and make operational adjustments as needed. In addition, the interactive feedback module also provides an early warning information prompt function to ensure that the user can take timely measures when encountering potential risks. Receive the user's operating instructions and provide necessary early warning information prompts to ensure that the user has full control and safe use of the flexible trailer vehicle.

[0153] It is worth noting that in the flexible trailer detection stage, the image processing and analysis module may use advanced algorithms and technologies to improve the recognition and positioning accuracy of the flexible trailer. At the same time, the decision-making and control module may consider multiple factors when controlling the flexible trailer vehicle, such as road conditions, traffic flow, wind resistance, etc., to ensure that the trailer vehicle can travel safely and stably in various scenarios.

[0154] Through the above implementation modes, the method and system provided by the above embodiments of the present invention can realize accurate detection and stable control of the flexible trailer part, thereby improving the safety and stability of the trailer vehicle in a complex road environment.

[0155] An embodiment of the present invention also provides a computer detection terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it can be used to execute any method of the above-mentioned embodiments of the present invention, or to run any system of the above-mentioned embodiments of the present invention.

[0156] Optionally, the memory is used to store programs; the memory may include volatile memory (English: volatile memory), such as random-access memory (English: random-access memory, abbreviated: RAM), such as static random-access memory (English: static random-access memory, abbreviated: SRAM), double data rate synchronous dynamic random access memory (English: Double Data Rate Synchronous Dynamic Random Access Memory, abbreviated: DDR SDRAM), etc.; the memory may also include non-volatile memory (English: non-volatile memory), such as flash memory (English: flash memory). The memory is used to store computer programs (such as applications, functional modules, etc. that implement the above method), computer instructions, etc., and the above computer programs, computer instructions, etc. can be partitioned and stored in one or more memories. And the above computer programs, computer instructions, data, etc. can be called by the processor.

[0157] The above-mentioned computer programs, computer instructions, etc. may be stored in one or more memories in partitions, and the above-mentioned computer programs, computer instructions, data, etc. may be called by a processor.

[0158] The processor is used to execute the computer program stored in the memory to implement the various steps of the method or various modules of the system involved in the above embodiments. For details, please refer to the relevant descriptions in the above method and system embodiments.

[0159] The processor and the memory may be independent structures or integrated structures. When the processor and the memory are independent structures, the memory and the processor may be coupled and connected via a bus.

[0160] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it can be used to execute any method of the above embodiments of the present invention, or to run any system of the above embodiments of the present invention.

[0161] The flexible trailer detection method and system based on monocular vision provided by the above embodiment of the present invention is intended to break through the technical bottleneck of the traditional trailer detection system in flexible object detection. The system cleverly integrates monocular vision detection, image processing and analysis, decision-making and control, and interactive feedback functions to build an efficient and accurate flexible trailer detection system. The detection and tracking of flexible trailer objects are realized through deep learning algorithms or monocular direction estimation technology, and the direction detection of a single trailer is realized for multiple trailers. Through the monocular direction estimation technology, the detection and tracking of flexible trailer objects are effectively performed, and the single trailer direction detection is realized in the case of multiple trailers, thereby realizing the flexible direction judgment for the fully flexible trailer. According to the detection results, corresponding control operations are performed, and seamless connection is made with other terminals (systems or vehicle-mounted equipment) to realize information interconnection and intelligent collaboration. The above embodiment of the present invention uses monocular vision technology to realize efficient detection of flexible trailer objects, thereby improving the safety and operation efficiency of trailer vehicles.

[0162] All matters not covered in the above embodiments of the present invention are well known in the art.

[0163] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art may make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. A flexible trailer detection method based on monocular vision, characterized in that: include: Obtain monocular vision information and 3D point cloud information of the trailer vehicle respectively; Fusing the monocular vision information and the 3D point cloud information to obtain image information, wherein the image information at least includes: posture, position and motion state information of the flexible trailer part; Performing feature extraction and target detection on the image information to obtain target information of the flexible trailer part; Performing monocular direction estimation of the forward posture of the target information to obtain the posture information of the flexible trailer part; Based on the position information and in combination with the dynamic changes of the vehicle ahead, path planning and decision adjustment are performed to generate control instructions to control vehicle tracking.

2. The flexible trailer detection method based on monocular vision according to claim 1 is characterized in that: The step of respectively acquiring monocular vision information and 3D point cloud information of the trailer vehicle includes: A monocular vision camera is used to capture real-time image data of the trailer vehicle, wherein the image data is used to display the shape, contour and texture information of the flexible trailer part, and obtain preliminary morphology and position information of the flexible trailer part, that is, to obtain monocular vision information; A laser radar sensor is used to obtain point cloud data of the trailer vehicle. The point cloud data is used to describe the three-dimensional structure, spatial position and speed information of the flexible trailer part, that is, to obtain 3D point cloud information, which provides auxiliary depth information and spatial positioning reference for the image data.

3. The flexible trailer detection method based on monocular vision according to claim 1 is characterized in that: The fusing the monocular vision information and the 3D point cloud information comprises: The monocular vision information and the 3D point cloud information are deeply fused and processed to obtain the current state and behavior information for accurately identifying the flexible trailer part, that is, to obtain image information.

4. The flexible trailer detection method based on monocular vision according to claim 1 is characterized in that: The performing feature extraction and target detection on the image information includes: Providing a pre-trained deep learning model, using the model to extract features from the image information, identify the trailer part, and estimate the direction of the leading vehicle; The trailer part is subjected to contour extraction and target detection, and combined with the estimation result of the front vehicle direction, the flexible trailer part is preliminarily positioned and morphologically analyzed to obtain target information.

5. The flexible trailer detection method based on monocular vision according to claim 1 is characterized in that: The step of performing monocular direction estimation of the front posture of the target information to obtain the posture information of the flexible trailer part includes: A monocular direction estimation model is provided, the target information is used as the input of the model, and the key feature points of the target information are extracted; wherein the monocular direction estimation model estimates the direction of each monomer in the target information, and then forms a constraint optimization between directions by combining the directions of each rigid body with the intersection constraints between rigid bodies, thereby obtaining the direction on the arc of the towing, that is, obtaining the key feature points of the target information; Matching the key feature points with the prior pose information of the vehicle ahead, and evaluating the motion state of the target information; Calculate the estimated position and posture information of the flexible trailer part according to the motion state of the target information, including: position coordinates and posture; The estimated position and posture information of the flexible towing part is smoothed, and the final position and posture information of the flexible towing part is output.

6. The flexible trailer detection method based on monocular vision according to claim 1 is characterized in that: The path planning and decision adjustment based on the posture information and in combination with the dynamic changes of the vehicle ahead include: Obtaining the length, angle and speed parameter information of the flexible trailer part according to the posture information, and adjusting the driving trajectory and speed of the vehicle in real time based on the parameter information so that the flexible trailer part remains stable during driving, and / or issuing an early warning according to the posture state; Combined with the current road conditions and the dynamic changes of the vehicle ahead, path planning and evaluation are performed to select the optimal path and adjust control decisions to generate command information to control vehicle tracking.

7. The flexible trailer detection method based on monocular vision according to any one of claims 1 to 6, characterized in that: Also includes: The monocular vision information, 3D point cloud information, image information, target information, position information, path information and / or decision information are sent to other terminals in real time for monitoring, management and / or information sharing and feedback.

8. A flexible trailer detection system based on monocular vision, characterized in that: include: Monocular vision camera, image processing and analysis module, decision and control module and interactive feedback module; among which: A monocular vision detection module, which is used to obtain monocular vision information and 3D point cloud information of the trailer vehicle respectively; An image processing and analysis module, which is used to fuse the monocular visual information and the 3D point cloud information to obtain image information, wherein the image information at least includes: posture, position and motion state information of the flexible trailer part; perform feature extraction and target detection on the image information to obtain target information of the flexible trailer part; perform monocular direction estimation of the forward posture on the target information to obtain posture information of the flexible trailer part; A decision and control module performs path planning and decision adjustment based on the position information and in combination with the dynamic changes of the vehicle ahead, and generates control instructions to control vehicle tracking.

9. The monocular vision-based flexible trailer detection system according to claim 8, characterized in that: The monocular vision detection module includes: a monocular vision camera and a laser radar sensor; wherein: The monocular vision camera is used to capture real-time image data of the trailer vehicle, and the image data is used to display the shape, contour and texture information of the flexible trailer part, and obtain the preliminary form and position information of the flexible trailer part, that is, to obtain monocular vision information; The laser radar sensor is used to obtain point cloud data of the trailer vehicle, and the point cloud data is used to describe the three-dimensional structure, spatial position and speed information of the flexible trailer part, that is, to obtain 3D point cloud information, providing auxiliary depth information and spatial positioning reference for the image data.

10. The flexible trailer detection system based on monocular vision according to claim 8 or 9, characterized in that: Also includes any one or more of the following: -Interactive feedback module, which is used to share information and / or provide information feedback with other terminals; - Status monitoring module, which is used to monitor the key parameters of the vehicle in real time, including: the position information of the flexible trailer part and the driving trajectory information of the trailer vehicle; -Data processing and analysis module, used for data analysis and processing of key vehicle parameters, and early warning of potential safety hazards; - An adaptive adjustment module, which is used to dynamically adjust the parameters of the image processing and analysis module and / or the path and decision of the decision and control module according to the actual operation of the vehicle; - A user operation module, which is respectively connected to the image processing and analysis module and the decision and control module for real-time data interaction, real-time status display, operation instruction input and early warning information prompt; -Trailer tracking optimization module, which optimizes the tracking accuracy of the flexible trailer part according to the analysis and processing results obtained in the data processing and analysis module, so that the flexible trailer part remains stable during driving; -Remote monitoring module, used to remotely monitor and manage the running status of the vehicle, and to display the tracking status of the flexible trailer part, early warning information and / or statistical data of various information in real time; -Communication module, used for information transmission and command issuance between the monocular vision detection module, the flexible trailer vehicle and the remote monitoring module.

Images