A method and system for detecting the end of a semi-trailer based on a CMS system
By setting ROI regions in the CMS system, performing edge and line detection, calculating the coordinates of the rear of the vehicle, and adjusting the field of view in real time, the problem of incomplete observation of the rear of semi-trailers was solved, and driving safety was improved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ZHEJIANG AUTOMOTIVE ELECTRONICS CO LTD
- Filing Date
- 2024-08-03
- Publication Date
- 2026-07-14
Smart Images
Figure CN119151970B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of CMS electronic exterior rearview mirror technology, specifically relating to a method and system for detecting the rear of a semi-trailer based on a CMS system. Background Technology
[0002] In recent years, the automotive industry has become increasingly electronic and intelligent. With the rapid development of technology, related technologies in the automotive field have made great progress. CMS (Camera Monitor System) is one such example. Unlike traditional optical rearview mirrors, CMS electronic rearview mirror systems use a combination of cameras and displays. The images captured by the external cameras are processed and displayed on the display screen inside the cabin. Its advantages include a wide field of view, less susceptibility to environmental influences, small size, and low wind resistance. Especially in the commercial vehicle sector, harsh environments and obstructed visibility are important factors in accidents. CMS systems can greatly improve these problems and reduce the probability of accidents.
[0003] The CMS system's images allow observation of the vehicle's posture and surrounding driving environment. The CMS's field of view must meet regulatory requirements and can be temporarily adjusted according to changes in the scene. Compared to ordinary passenger cars, semi-trailers are longer, and the cab and cargo box are separate. Therefore, in scenarios such as turning, the angle between the cargo box and the cab is larger, causing the rear of the vehicle to exceed or leave the image's field of view. This is not conducive to the driver's observation of the environment behind the vehicle. The existing CMS system's field of view adjustment method is not accurate and cannot guarantee that the rear of the semi-trailer is always within the optimal field of view. It can only be adjusted based on the driver's experience.
[0004] Therefore, a method for detecting the rear of a semi-trailer is needed to promptly observe changes in the position of the rear, adjust the field of view of the CMS system, keep the rear of the vehicle in the optimal observation area, and avoid accidents caused by obstructed vision. Summary of the Invention
[0005] In view of this, the present invention proposes a method and system for detecting the rear of a semi-trailer based on a CMS system, which is used to solve the problem that existing CMS systems cannot keep the rear of the semi-trailer within the optimal field of vision at all times.
[0006] In a first aspect, this invention discloses a method for detecting the rear of a semi-trailer based on a CMS system, the method comprising:
[0007] The CMS system continuously collects images of the rear of the vehicle and sets ROI regions within these images.
[0008] Perform vertical edge detection on the ROI region and extract candidate vertical edge lines of the vehicle rear;
[0009] Edge detection is performed on the rear image of the vehicle to obtain an edge image. Line detection is then performed on the edge image to extract the bottom edge line of the vehicle body.
[0010] The coordinates of the current rear point are calculated based on the candidate vertical edge line of the rear of the vehicle and the bottom edge line of the vehicle body;
[0011] The CMS system adjusts its field of view in real time based on the current coordinates of the rear of the vehicle, ensuring that the rear of the vehicle is always within the preset optimal display range.
[0012] Based on the above technical solutions, preferably, setting the ROI region in the rear image specifically includes:
[0013] For the initial frame of the rear image, the position coordinates of the rear point on the image are obtained according to the calibration module of the CMS system;
[0014] Using the rear of the vehicle as a reference point, set up a Region of Interest (ROI) with a width of w and a height of h. With the top left corner of the ROI as the origin, make the coordinates of the rear of the vehicle in the ROI (w / 2, 3h / 4).
[0015] For non-initial frame rear-end images, the rear-end point detected in the previous frame is used as a reference point. A Region of Interest (ROI) with width w and height h is set, and the top left corner of the ROI is taken as the origin of the coordinate system, so that the coordinates of the rear-end point in the ROI are (w / 2, 3h / 4).
[0016] Based on the above technical solutions, preferably, the step of performing vertical edge detection on the ROI region and extracting candidate vertical edge lines of the vehicle rear specifically includes:
[0017] Extracting the vertical edge image of the ROI region using the Sobel operator;
[0018] In the vertical edge image, the cumulative vertical edge value of each column of pixels is used to obtain a sequence of cumulative vertical edge values;
[0019] The vertical edge cumulative value sequence is smoothed using Gaussian filtering, and the peak value of the vertical edge cumulative value sequence and the corresponding x-coordinate of the peak value are found.
[0020] The x-coordinates corresponding to each peak are combined into an x-coordinate array;
[0021] The straight line perpendicular to the horizontal axis at the x-coordinate corresponding to each peak is taken as the candidate vertical edge line of the rear of the car.
[0022] Based on the above technical solutions, preferably, the step of performing straight line detection on the edge image and extracting the bottom edge line of the vehicle body specifically includes:
[0023] For the initial frame rear image, Hough transform is used to detect lines in the extracted edge image to extract multiple first candidate lines;
[0024] The CMS system's calibration module calculates the projection coordinates of the ground projection point corresponding to the camera on the image, and the straight line formed by the rear point of the vehicle in the initial frame rear image and the projection coordinates is used as the initial vehicle body reference line L.
[0025] Calculate the angle between each first candidate line and the initial vehicle body reference line L, and filter out the first candidate lines whose angle with the initial vehicle body reference line L is greater than a preset angle threshold.
[0026] Calculate the rear point (u0, v0) and projected coordinates (u...) respectively. c ,v c The distance to each first candidate line is used to select the first candidate line with the smallest sum of distances to the two points as the bottom edge line of the vehicle body.
[0027] Based on the above technical solutions, preferably, the step of performing line detection on the edge image and extracting the bottom edge line of the vehicle body further includes:
[0028] For a non-initial frame rear-end image, calculate the intersection point P of the bottom edge line of the vehicle body and the image edge of the previous frame rear-end image, and calculate the horizontal distance d between the rear-end point of the previous frame and the intersection point P. w ;
[0029] Using the bottom edge of the car body in the previous frame's rear image as the center line, extend horizontally to both sides with a width d to obtain the extended area, where d = 2d. w ;
[0030] Canny edge and Hough line detection are performed within the extended region to extract multiple second candidate lines;
[0031] Calculate the distances from the rear point and intersection point P of the previous frame's rear image to each of the second candidate lines, and select the second candidate line with the smallest sum of distances to the two points as the bottom edge line of the vehicle body.
[0032] Based on the above technical solutions, preferably, the step of calculating the current rear point coordinates based on the candidate vertical edge line of the rear of the vehicle and the bottom edge line of the vehicle body specifically includes:
[0033] Substitute the x-coordinate of each candidate vehicle rear vertical edge line into the equation of the bottom edge line of the vehicle body to obtain the corresponding y-coordinate.
[0034] The horizontal coordinates and corresponding vertical coordinates of the vertical edge lines of each candidate vehicle's rear are used as candidate vehicle rear points.
[0035] Calculate the Euclidean distance between each candidate vehicle rear point and the vehicle rear point in the previous frame image;
[0036] The candidate rear point with the smallest Euclidean distance is selected as the coordinate of the rear point in the current frame.
[0037] Based on the above technical solutions, preferably, the CMS system's field of view is adjusted in real time according to the current coordinates of the rear of the vehicle, so that the rear of the vehicle is always within the preset optimal display range:
[0038] The horizontal visible field of view is cropped based on the coordinates of the vehicle's rear point in the previous frame, ensuring that the horizontal width from the rear point to the edge of the image containing the vehicle body does not exceed a preset threshold proportion of the entire field of view. A second aspect of this invention discloses a semi-trailer rear detection system based on a CMS system, the system comprising:
[0039] Data acquisition module: Used to continuously acquire the current rear-end image of the vehicle through the CMS system;
[0040] Rear edge detection module: used to set ROI regions in the rear image, perform vertical edge detection on the ROI regions, and extract candidate vertical edge lines of the rear of the vehicle;
[0041] Vehicle body edge detection module: used to perform edge detection on the rear image of the vehicle to obtain an edge image, perform line detection on the edge image, and extract the bottom edge line of the vehicle body;
[0042] Rear end point calculation module: used to calculate the current rear end point coordinates based on the candidate vertical edge line of the rear end and the bottom edge line of the vehicle body;
[0043] View adjustment module: Used to adjust the view of the CMS system in real time according to the current coordinates of the rear of the vehicle, so that the rear of the vehicle is always within the preset optimal display range.
[0044] A third aspect of the present invention discloses an electronic device comprising: at least one processor, at least one memory, a communication interface, and a bus;
[0045] The processor, memory, and communication interface communicate with each other through the bus.
[0046] The memory stores program instructions that can be executed by the processor, which invokes the program instructions to implement the method as described in the first aspect of the present invention.
[0047] In a fourth aspect, the present invention discloses a computer-readable storage medium storing computer instructions that cause a computer to perform the method described in the first aspect of the present invention.
[0048] The present invention has the following advantages over the prior art:
[0049] 1) Based on the existing CMS system, this invention extracts candidate vertical edge lines of the rear of the vehicle and bottom edge lines of the vehicle body from the ROI region of the rear image. The current coordinates of the rear point are calculated based on the candidate vertical edge lines of the rear of the vehicle and the bottom edge lines of the vehicle body. This allows for real-time adjustment of the field of view of the CMS system camera, ensuring that the rear of the vehicle is always within the optimal field of view, facilitating observation of the environment behind the vehicle and improving the driving safety of semi-trailers.
[0050] 2) This invention sets the ROI region for rear vehicle detection based on the rear vehicle coordinates of the previous frame. The vertical edge image of the ROI region is extracted using the Sobel operator. By performing straight line detection on the edge image of the rear vehicle image, the bottom edge line of the vehicle body is extracted. The abscissa of each candidate vertical edge line of the rear vehicle is substituted into the straight line equation of the bottom edge line of the vehicle body to obtain the corresponding ordinate, thereby determining the candidate rear vehicle point. Finally, the Euclidean distance between each candidate rear vehicle point and the rear vehicle point of the previous frame is calculated. The candidate rear vehicle point with the smallest Euclidean distance is selected as the coordinate of the rear vehicle point in the current frame. This fully utilizes the temporal continuity of the video frames captured by the camera, which helps to stabilize tracking in dynamic scenes and avoids incorrect positioning caused by drastic image changes or detection noise.
[0051] 3) This invention extracts multiple candidate straight lines by measuring straight lines, making full use of the geometric features of the semi-trailer to filter out the bottom edge line of the vehicle body. Specifically, for the initial frame rear image, the first candidate straight line whose angle with the initial vehicle body reference line L is greater than a preset angle threshold is filtered out; the first candidate straight line with the smallest sum of distances from the rear point and the projection center point to each of the first candidate straight lines is selected as the bottom edge line of the vehicle body; for non-initial frame rear images, multiple second candidate straight lines are extracted in the extended area, and the second candidate straight line with the smallest sum of distances from the rear point and the intersection point P of the previous frame rear image to each of the second candidate straight lines is selected as the bottom edge line of the vehicle body. That is, this invention can quickly eliminate candidate lines that are far from the vehicle body or irrelevant in the presence of noise and interference, and reliably identify the bottom edge line of the vehicle body; at the same time, the correlation between two frames of images is used for detection, which makes the identification of the rear point more stable and helps to maintain the reliability of tracking in high-speed or complex scenes.
[0052] 4) In the process of calculating the coordinates of the bottom edge line and the rear point of the vehicle body, the present invention selects the optimal solution through distance calculation, which can quickly obtain the results with less computation, is suitable for the needs of real-time application environment, and improves the efficiency of the algorithm. Attached Figure Description
[0053] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0054] Figure 1 This is a flowchart of the semi-trailer rear detection method based on the CMS system of the present invention;
[0055] Figure 2 A schematic diagram of the ROI region set up in this invention;
[0056] Figure 3 The horizontal distance d between the rear viewpoint of the vehicle and the intersection point P in the previous frame's rear view image is... w A schematic diagram;
[0057] Figure 4 This is a schematic diagram of the expanded area obtained by extending the bottom edge of the vehicle body of the previous frame's rear image with a width d parallel to both sides.
[0058] Figure 5 This is a schematic diagram illustrating how the CMS system adjusts its field of view in real time based on the current coordinates of the vehicle's rear point. Detailed Implementation
[0059] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.
[0060] Please see Figure 1 This invention discloses a method for detecting the rear of a semi-trailer based on a CMS system, the method comprising:
[0061] S1. Continuously collect the current rear-end image through the CMS system and set the ROI region in the rear-end image.
[0062] Step S1 specifically includes the following sub-steps:
[0063] S11. For the initial frame image of the rear of the vehicle, obtain the position coordinates of the rear point on the image based on the calibration module of the existing semi-trailer CMS system.
[0064] In an embodiment of the present invention, the camera of the CMS system continuously acquires the current rear image of the vehicle. Based on the calibration module of the existing semi-trailer CMS system, the coordinates of the rear point in the rear image are calibrated. After calibration, the position (u0, v0) of the rear of the vehicle on the image can be obtained.
[0065] S12. Using the rear of the vehicle as a reference point, set a ROI region with a width of w and a height of h.
[0066] In embodiments of the present invention, the specific requirements for setting the ROI region are as follows: taking the upper left corner of the ROI region as the origin of the coordinate system, the coordinates of the reference point in the ROI region are (w / 2, 3h / 4), such as... Figure 2 The diagram shown is a schematic of the ROI region set in this invention.
[0067] S13. For non-initial frame rear-end images, use the rear-end point detected in the previous frame as a reference point and set a ROI region with width w and height h.
[0068] Similarly, the specific requirements for setting the ROI region are: take the upper left corner of the ROI region as the origin of the coordinate system, and make the coordinates of the reference point in the ROI region (w / 2, 3h / 4).
[0069] S2. Perform vertical edge detection on the ROI region and extract candidate vertical edge lines of the rear of the vehicle.
[0070] Step S2 specifically includes the following sub-steps:
[0071] S21. Extract the vertical edge image of the ROI region using the Sobel operator.
[0072] Extract the Sobel vertical edges within the ROI region, where the Sobel operator has a size of 3×3. Within the ROI region, the vertical partial derivative G at position (i,j) is... y The formula for calculating (i,j) is:
[0073] G y (i,j)=[f(i-1,j+1)+2f(i,j+1)+f(i+1,j+1)]-[f(i-1,j-1)+2f(i,j+1)+f(i+1,j+1)]
[0074] f(i,j) represents the pixel value at position (i,j) in the ROI region.
[0075] S22. In the vertical edge image, along the horizontal axis x, the vertical edge cumulative value of each column of pixels is accumulated to obtain the vertical edge cumulative value sequence.
[0076] S23. Use Gaussian filtering to smooth the vertical edge cumulative value sequence, find the peak value of the vertical edge cumulative value sequence and the corresponding horizontal coordinate of the peak value, and form an horizontal coordinate array by combining the horizontal coordinates corresponding to each peak value.
[0077] In embodiments of the present invention, the specific search process is as follows:
[0078] (1) Iterate through each element in the sequence;
[0079] (2) Determine if the current element is the peak value: If the current element is greater than its left and right adjacent elements, then it is the peak value, and record the peak value and its corresponding x-coordinate value peak_x. k If the element is not a peak value, skip it and proceed to the next element.
[0080] (3) Filter elements whose peak values are lower than the preset peak threshold;
[0081] (4) Calculate peak_x k The lateral distance to the rear of the car in the previous frame is used to filter out elements whose distance exceeds a preset distance threshold.
[0082] (5) After traversing the entire sequence, obtain an array (peax_x0, peax_x1, ..., peak_x) containing the x-coordinates corresponding to multiple peaks. k , ...,peak_x K ), where k = 1, 2, ..., K, and K is the total number of peaks.
[0083] S24. Take the straight line perpendicular to the horizontal axis at the horizontal coordinate corresponding to each peak as the candidate vertical edge line of the rear of the vehicle.
[0084] For each peak value, the corresponding horizontal coordinate is represented by the line x = peak_x perpendicular to the horizontal axis. k This is a candidate vertical edge line of the rear of the car.
[0085] S3. Perform edge detection on the rear image of the vehicle to obtain an edge image, perform line detection on the edge image, and extract the bottom edge line of the vehicle body.
[0086] Step S3 specifically includes the following sub-steps:
[0087] S31. For the initial frame image of the rear of the vehicle, the bottom edge line of the vehicle body.
[0088] S311. For the initial frame rear image, perform Canny edge detection on the entire rear image, and use Hough transform to perform line detection on the extracted edge image to extract multiple first candidate lines.
[0089] S312. Calculate the projection coordinates of the ground projection point corresponding to the camera on the image according to the calibration module of the CMS system, and take the straight line formed by the rear point of the vehicle in the initial frame rear image and the projection coordinates as the initial vehicle body reference line L.
[0090] Specifically, based on the calibration parameters, the projected coordinates (u) of the ground projection point corresponding to the camera on the image are calculated. c ,v c Connect the calibrated rear point (u0, v0) with the projected coordinates (u... c ,v c The two points are used to form a straight line, which is then used as the initial vehicle body reference line L.
[0091] S313. Calculate the angle between each first candidate line and the initial vehicle body reference line L, and filter out the first candidate lines whose angle with the initial vehicle body reference line L is greater than a preset angle threshold.
[0092] S314. Calculate the distances from the rear point and the projected coordinates to each of the first candidate lines, and select the first candidate line with the smallest sum of distances to the two points as the bottom edge line of the vehicle body.
[0093] Specifically, calculate the rear point (u0, v0) and the projected coordinates (u...) separately. c ,v c The distances to each straight line are used to select the first candidate straight line with the smallest sum of distances to the two points as the bottom edge line of the vehicle body. The equation of the bottom edge line of the vehicle body is expressed as y = kx + b.
[0094] S32. For the rear image of the vehicle that is not the initial frame, extract the bottom edge line of the vehicle body.
[0095] S321. For a non-initial frame rear image, calculate the intersection point P of the bottom edge line of the vehicle body and the image edge of the previous frame rear image, and calculate the horizontal distance d between the rear point of the previous frame rear image and the intersection point P. w .
[0096] like Figure 3 The figure shows the horizontal distance d between the rear end of the vehicle and the intersection point P in the previous frame's rear-end image. w The diagram shows that P is the intersection of the bottom edge of the vehicle body in the previous frame's rear image and the edge of the rear image.
[0097] S322. Using the bottom edge of the vehicle body in the previous frame's rear image as the center line, extend parallel to both sides to obtain the extended area.
[0098] like Figure 4 The diagram shows the expanded area obtained by extending the bottom edge of the car body parallel to both sides with a width d as the center line from the previous frame's rear image, where d = 2d. wThe dotted line in the middle is the bottom edge line of the car body in the previous frame's rear image.
[0099] S323. Perform Canny edge and Hough line detection within the extended area to extract multiple second candidate lines.
[0100] Since the vehicle body movement does not change significantly between adjacent frames in consecutive frames, this invention uses the detection result of the bottom edge line of the vehicle body in the previous frame's rear image to constrain the detection area of the current frame, calculates the intersection point P of the straight line in the previous frame and the image edge, and calculates the horizontal distance d between the rear point and P. w Using the straight line of the previous frame as the center, extend the width d to both sides in parallel, and extract the bottom edge line of the vehicle body of the current frame within the extended area.
[0101] S324. Calculate the distances from the rear point and intersection point P of the previous frame's rear image to each of the second candidate lines, and select the second candidate line with the smallest sum of distances to the two points as the bottom edge line of the vehicle body.
[0102] Let the formula for a certain second candidate line be Ax + By + C = 0, and the distance from the intersection point P(x0, y0) to the second candidate line be D:
[0103]
[0104] Within the extended region, the second candidate line closest to the detection result of the previous frame's rear image is the bottom edge line of the vehicle body in the current frame, and the equation of the bottom edge line of the vehicle body is expressed as y = kx + b.
[0105] This invention extracts multiple candidate straight lines by measuring straight lines, fully utilizing the geometric features of the semi-trailer to filter out the bottom edge line of the vehicle body. Specifically, for the initial frame rear image, the first candidate straight line whose angle with the initial vehicle body reference line L is greater than a preset angle threshold is filtered out. The first candidate straight line with the smallest sum of distances from the rear point and the projection center point to each of the first candidate straight lines is selected as the bottom edge line of the vehicle body. For rear images that are not the initial frame, multiple second candidate straight lines are extracted in the extended region. The second candidate straight line with the smallest sum of distances from the rear point and the intersection point P of the previous frame rear image to each of the second candidate straight lines is selected as the bottom edge line of the vehicle body. This allows for the rapid elimination of candidate lines that are far from or irrelevant to the vehicle body in the presence of noise and interference, reliably identifying the bottom edge line of the vehicle body. At the same time, the correlation between two frames is used for detection, making the identification of the rear point more stable and helping to maintain the reliability of tracking in high-speed or complex scenes.
[0106] S4. Calculate the current rear point coordinates based on the candidate rear vertical edge line and the bottom edge line of the vehicle body.
[0107] The vertical edge line of the rear of the vehicle and the bottom edge line of the vehicle together constitute the geometric features of the semi-trailer. This invention uses these geometric features to calculate the coordinates of the current rear point.
[0108] Specifically, the horizontal coordinates (peax_x0, pearx_x1, ..., peak_x) of each candidate car's rear end perpendicular to the edge line are respectively... k , ...,peak_x K Substituting the equation of the straight line at the bottom edge of the vehicle body, we obtain the corresponding ordinates (peax_y0, peax_y1, ..., peak_y). k , ...,peak_y K ).
[0109] The x-coordinate and y-coordinate of the vertical edge line of each candidate vehicle's rear are used as candidate vehicle rear points (peax_x0, pearx_y0), (peax_x1, pearx_y1), ..., (peak_x0, pearx_y0). k ,peak_y k ), ..., (peak_x K ,peak_y K ).
[0110] Calculate the Euclidean distance between each candidate vehicle rear point and the vehicle rear point in the previous frame image;
[0111] The candidate rear point with the smallest Euclidean distance is selected as the coordinate of the rear point in the current frame.
[0112] This invention sets the ROI region for rear vehicle detection based on the rear vehicle coordinates of the previous frame. It extracts the vertical edge image of the ROI region using the Sobel operator, performs straight line detection on the edge image of the rear vehicle image, and extracts the bottom edge line of the vehicle body. The abscissa of each candidate vertical edge line of the rear vehicle is substituted into the equation of the bottom edge line of the vehicle body to obtain the corresponding ordinate, thus determining the candidate rear vehicle point. Finally, the Euclidean distance between each candidate rear vehicle point and the rear vehicle point of the previous frame is calculated, and the candidate rear vehicle point with the smallest Euclidean distance is selected as the coordinate of the rear vehicle point in the current frame. This fully utilizes the temporal continuity of the video frames captured by the camera, which helps with stable tracking in dynamic scenes and avoids incorrect positioning due to drastic image changes or detection noise.
[0113] Furthermore, in the process of calculating the coordinates of the bottom edge line and the rear point of the vehicle body, the present invention selects the optimal solution through distance calculation, which can quickly obtain the results with less computation, making it suitable for the needs of real-time application environments and improving the efficiency of the algorithm.
[0114] S5. Adjust the CMS system's field of view in real time based on the current coordinates of the rear of the vehicle, so that the rear of the vehicle is always within the preset optimal display range.
[0115] The horizontal field of view is cropped based on the coordinates of the rear point of the vehicle in the previous frame, so that the horizontal width from the rear point of the vehicle to the edge of the image where the vehicle body is located does not exceed the proportion of the entire field of view.
[0116] Specifically, such as Figure 5 The diagram illustrates how the CMS system adjusts its field of view in real time based on the current coordinates of the vehicle's rear point. By cropping the horizontal view, the horizontal width W from the rear point to the edge of the image containing the vehicle body can be adjusted. r The area should not exceed 80% of the total field of view W to reduce accidents caused by obstruction of the field of view.
[0117] This invention is based on the existing CMS system. It extracts candidate vertical edge lines of the rear of the vehicle and bottom edge lines of the vehicle body from the ROI region of the rear image. It calculates the current coordinates of the rear point based on the candidate vertical edge lines of the rear of the vehicle and the bottom edge lines of the vehicle body. This allows for real-time adjustment of the field of view of the CMS system camera, ensuring that the rear of the vehicle is always within the optimal field of view, facilitating observation of the environment behind the vehicle and improving the driving safety of semi-trailers.
[0118] Corresponding to the above method embodiments, the present invention also discloses a semi-trailer rear detection system based on a CMS system, the system comprising:
[0119] Data acquisition module: Used to continuously acquire the current rear-end image of the vehicle through the CMS system;
[0120] Rear edge detection module: used to set ROI regions in the rear image, perform vertical edge detection on the ROI regions, and extract candidate vertical edge lines of the rear of the vehicle;
[0121] Vehicle body edge detection module: used to perform edge detection on the rear image of the vehicle to obtain an edge image, perform line detection on the edge image, and extract the bottom edge line of the vehicle body;
[0122] Rear end point calculation module: used to calculate the current rear end point coordinates based on the candidate vertical edge line of the rear end and the bottom edge line of the vehicle body;
[0123] View adjustment module: Used to adjust the view of the CMS system in real time according to the current coordinates of the rear of the vehicle, so that the rear of the vehicle is always within the preset optimal display range.
[0124] The above system embodiments and method embodiments are one-to-one correspondences. For a brief description of the system embodiments, please refer to the method embodiments.
[0125] The present invention also discloses an electronic device, comprising: at least one processor, at least one memory, a communication interface, and a bus; wherein the processor, memory, and communication interface communicate with each other through the bus; the memory stores program instructions executable by the processor, and the processor calls the program instructions to implement the aforementioned method of the present invention.
[0126] The present invention also discloses a computer-readable storage medium that stores computer instructions, which cause the computer to implement all or part of the steps of the method described in the embodiments of the present invention. The storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.
[0127] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, meaning they can be distributed across multiple network units. Those skilled in the art can select some or all of the modules to achieve the purpose of this embodiment without any inventive effort, based on actual needs.
[0128] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A method for detecting the rear of a semi-trailer based on a CMS system, characterized in that, The method includes: The CMS system continuously acquires images of the vehicle's rear end and sets ROI regions within these images. Specifically, setting ROI regions involves: for the initial frame of the rear end image, obtaining the coordinates of the rear end point on the image using the CMS system's calibration module; using the rear end point as a reference point, setting an ROI region with width *w* and height *h*, and using the top-left corner of the ROI region as the origin, such that the coordinates of the rear end point within the ROI region are (w / 2, 3h / 4); for non-initial frame rear end images, using the rear end point detected in the previous frame as a reference point, setting an ROI region with width *w* and height *h*, and using the top-left corner of the ROI region as the origin, such that the coordinates of the rear end point within the ROI region are (w / 2, 3h / 4). Vertical edge detection is performed on the ROI region to extract candidate rear vertical edge lines. Specifically, this includes: extracting the vertical edge image of the ROI region using the Sobel operator; accumulating the vertical edge value of each column of pixels in the vertical edge image to obtain a vertical edge cumulative value sequence; smoothing the vertical edge cumulative value sequence using Gaussian filtering; finding the peak value of the vertical edge cumulative value sequence and the corresponding horizontal coordinate; forming an horizontal coordinate array from the horizontal coordinates of each peak value; and using the straight line perpendicular to the horizontal axis at the horizontal coordinate of each peak value as a candidate rear vertical edge line. Edge detection is performed on the rear image to obtain an edge image. Line detection is then performed on the edge image to extract the bottom edge line of the vehicle body. Specifically, the line detection and extraction of the bottom edge line of the vehicle body includes: for the initial frame rear image, using Hough transform to perform line detection in the extracted edge image to extract multiple first candidate lines; calculating the projection coordinates of the ground projection point corresponding to the camera on the image according to the calibration module of the CMS system, and using the line formed by the rear point in the initial frame rear image and the projection coordinates as the initial vehicle body reference line L; calculating the angle between each first candidate line and the initial vehicle body reference line L, and filtering out first candidate lines whose angle with the initial vehicle body reference line L is greater than a preset angle threshold; calculating the distances from the rear point and the projection coordinates to each first candidate line, and selecting the first candidate line with the smallest sum of distances to the two points as the bottom edge line of the vehicle body; the line detection and extraction of the bottom edge line of the vehicle body also includes: for non-initial frame rear images, calculating the intersection point P of the bottom edge line of the vehicle body in the previous frame rear image and the image edge, and calculating the horizontal distance d between the rear point in the previous frame rear image and the intersection point P. w Using the bottom edge of the car body in the previous frame's rear image as the center line, extend the line parallel to both sides with a width d to obtain the extended area, where d = 2d. w Canny edge and Hough line detection are performed within the extended area to extract multiple second candidate lines; the distances from the rear point and intersection point P of the previous rear image to each second candidate line are calculated respectively, and the second candidate line with the smallest sum of distances to the two points is selected as the bottom edge line of the vehicle body. The coordinates of the current rear point are calculated based on the candidate vertical edge line of the rear of the vehicle and the bottom edge line of the vehicle body; The CMS system adjusts its field of view in real time based on the current coordinates of the rear of the vehicle, ensuring that the rear of the vehicle is always within the preset optimal display range.
2. The semi-trailer rear detection method based on a CMS system according to claim 1, characterized in that, The calculation of the current rear point coordinates based on the candidate vertical edge line of the rear of the vehicle and the bottom edge line of the vehicle body specifically includes: Substitute the x-coordinate of each candidate vehicle rear vertical edge line into the equation of the bottom edge line of the vehicle body to obtain the corresponding y-coordinate. The horizontal coordinates and corresponding vertical coordinates of the vertical edge lines of each candidate vehicle's rear are used as candidate vehicle rear points. Calculate the Euclidean distance between each candidate vehicle rear point and the vehicle rear point in the previous frame image; The candidate rear point with the smallest Euclidean distance is selected as the coordinate of the rear point in the current frame.
3. The semi-trailer rear detection method based on a CMS system according to claim 2, characterized in that, The step of adjusting the CMS system's field of view in real time based on the current coordinates of the rear of the vehicle, so that the rear of the vehicle is always within the preset optimal display range, specifically includes: The horizontal field of view is cropped based on the coordinates of the rear point of the vehicle in the previous frame, so that the horizontal width from the rear point of the vehicle to the edge of the image where the vehicle body is located does not exceed the proportion of the entire field of view.
4. A semi-trailer rear detection system based on a CMS system, characterized in that, The system includes: Data acquisition module: Used to continuously acquire the current rear-end image of the vehicle through the CMS system; Rear End Edge Detection Module: Used to set a Region of Interest (ROI) in the rear end image, perform vertical edge detection on the ROI, and extract candidate vertical edge lines of the rear end. Setting the ROI in the rear end image specifically includes: for the initial frame rear end image, obtaining the position coordinates of the rear end point on the image according to the calibration module of the CMS system; using the rear end point as a reference point, setting an ROI with a width of w and a height of h, with the upper left corner of the ROI as the origin, so that the coordinates of the rear end point in the ROI are (w / 2, 3h / 4); for rear end images not from the initial frame, using the rear end point detected in the previous frame as a reference point, setting an ROI with a width of w and a height of h, with the upper left corner of the ROI as the origin, so that the coordinates of the rear end point in the ROI are (w / 2, 3h / 4); The top left corner is the origin, and the coordinates of the rear of the vehicle in the ROI region are (w / 2, 3h / 4). The process of performing vertical edge detection on the ROI region and extracting candidate vertical edge lines for the rear of the vehicle specifically includes: extracting the vertical edge image of the ROI region using the Sobel operator; obtaining a sequence of cumulative vertical edge values for each column of pixels in the vertical edge image; smoothing the sequence of cumulative vertical edge values using Gaussian filtering; finding the peak value of the sequence and the corresponding horizontal coordinate; forming an array of horizontal coordinates corresponding to each peak value; and using the line perpendicular to the horizontal axis at each peak value's horizontal coordinate as a candidate vertical edge line for the rear of the vehicle. Vehicle body edge detection module: used to perform edge detection on the rear image of the vehicle to obtain an edge image, and to perform line detection on the edge image to extract the bottom edge line of the vehicle body; the specific steps of performing line detection on the edge image to extract the bottom edge line of the vehicle body include: for the initial frame rear image, using Hough transform to perform line detection in the extracted edge image to extract multiple first candidate lines; calculating the projection coordinates of the ground projection point corresponding to the camera on the image according to the calibration module of the CMS system, and using the line formed by the rear point in the initial frame rear image and the projection coordinates as the initial vehicle body reference line L. The process involves: calculating the angle between each first candidate line and the initial vehicle body reference line L, filtering out first candidate lines whose angle with the initial vehicle body reference line L is greater than a preset angle threshold; calculating the distance from the rear point and the projected coordinates to each first candidate line, selecting the first candidate line with the smallest sum of distances to the two points as the bottom edge line of the vehicle body; and further, for non-initial frame rear images, calculating the intersection point P of the bottom edge line of the vehicle body in the previous frame rear image and the image edge, and calculating the horizontal distance d between the rear point and the intersection point P in the previous frame rear image. w Using the bottom edge of the car body in the previous frame's rear image as the center line, extend the line parallel to both sides with a width d to obtain the extended area, where d = 2d. w Canny edge and Hough line detection are performed within the extended area to extract multiple second candidate lines; the distances from the rear point and intersection point P of the previous rear image to each second candidate line are calculated respectively, and the second candidate line with the smallest sum of distances to the two points is selected as the bottom edge line of the vehicle body. Rear end point calculation module: used to calculate the current rear end point coordinates based on the candidate vertical edge line of the rear end and the bottom edge line of the vehicle body; View adjustment module: Used to adjust the view of the CMS system in real time according to the current coordinates of the rear of the vehicle, so that the rear of the vehicle is always within the preset optimal display range.
5. An electronic device, characterized in that, include: At least one processor, at least one memory, a communication interface, and a bus; The processor, memory, and communication interface communicate with each other through the bus. The memory stores program instructions that can be executed by the processor, and the processor invokes the program instructions to implement the method as described in any one of claims 1 to 3.
6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that are executed by a processor to cause the computer to perform the method as described in any one of claims 1 to 3.