Semitrailer tail detection method and system based on CMS system

Abstract

The invention discloses a semitrailer tail detection method and system based on a CMS system, and the method comprises the steps: continuously collecting a current tail image through the CMS system, and setting an ROI region in the tail image; performing vertical edge detection on the ROI region, and extracting candidate vertical edge lines of the vehicle tail; carrying out edge detection on the vehicle tail image, carrying out straight line detection on an edge image, and extracting a vehicle body bottom edge line; calculating coordinates of a current vehicle tail point according to the candidate vehicle tail vertical edge line and the vehicle body bottom edge line; and the visual field of the CMS system is adjusted in real time according to the current vehicle tail point coordinates, so that the vehicle tail is always in a preset optimal display range. According to the method, on the basis of an existing CMS system of the semitrailer, the candidate vertical edge line of the tail of the semitrailer and the edge line of the bottom of the semitrailer body are extracted from the tail image, and the coordinates of the current tail point are calculated, so that the visual field of a camera of the CMS system is adjusted in an auxiliary mode in real time, and the tail of the semitrailer is always in the optimal visual field.

Description

Technical Field

[0001] The invention belongs to the technical field of CMS electronic exterior rearview mirrors, and in particular relates to a method and system for detecting the rear end of a semitrailer based on a CMS system. Background Art

[0002] In recent years, the development of the automotive field has become more and more electronic and intelligent. With the rapid development of science and technology, relevant technologies in the automotive field have made great progress. The CMS (Camera Monitor System, electronic exterior rearview mirror) system is one of them. Different from traditional optical rearview mirrors, the CMS electronic exterior rearview mirror system uses a combination of cameras and display screens. The images collected by the external camera are processed and displayed on the display screen in the cabin. Its advantages include wide field of view, little impact from the environment, small size, low wind resistance, etc. Especially in the field of commercial vehicles, harsh environment and obstructed vision are important factors in the occurrence of accidents. The CMS system can greatly improve these problems and reduce the probability of accidents.

[0003] The vehicle's posture and the surrounding driving environment can be observed through the images of the CMS system. The field of view of CMS must meet regulatory requirements and can be temporarily adjusted as the scene changes. Compared with ordinary passenger cars, the semi-trailer has a longer body and the front and compartment are separated. Therefore, in scenes such as turning, the angle between the compartment and the front is larger, causing the rear of the vehicle to exceed or leave the field of view of the image, which is not conducive to the driver's observation of the environment behind the vehicle. The field of view adjustment method of the CMS system in the prior art is not accurate and cannot ensure that the rear of the semi-trailer is always in the best field of view. The body can only be adjusted based on the driver's experience.

[0004] Therefore, a method for detecting the rear end of a semi-trailer is needed to timely observe the position changes of the rear end of the vehicle, adjust the field of view of the CMS system, keep the rear end of the vehicle in the best observation area, and avoid accidents caused by obstruction of the field of view. Summary of the invention

[0005] In view of this, the present invention proposes a method and system for detecting the rear end of a semi-trailer based on a CMS system, which is used to solve the problem that the existing CMS system cannot always keep the rear end of the semi-trailer within the optimal field of view.

[0006] In a first aspect, the present invention discloses a method for detecting the rear end of a semitrailer based on a CMS system, the method comprising:

[0007] The current rear image of the vehicle is continuously collected through the CMS system, and the ROI area is set in the rear image of the vehicle;

[0008] Performing vertical edge detection on the ROI area to extract candidate vertical edge lines of the rear of the vehicle;

[0009] Performing edge detection on the rear image to obtain an edge image, performing straight line detection on the edge image to extract the edge line of the bottom of the vehicle body;

[0010] Calculating the coordinates of the current vehicle rear point according to the candidate vehicle rear vertical edge line and the vehicle body bottom edge line;

[0011] The field of view of the CMS system is adjusted in real time according to the current coordinates of the rear point of the vehicle, so that the rear of the vehicle is always within the preset optimal display range.

[0012] On the basis of the above technical solution, preferably, the step of setting the ROI area in the rear image specifically includes:

[0013] For the initial frame vehicle rear image, the position coordinates of the vehicle rear point on the image are obtained according to the calibration module of the CMS system;

[0014] Taking the rear point of the vehicle as the reference point, set the ROI area with a width of w and a height of h, and take the upper left corner of the ROI area as the coordinate origin, so that the coordinates of the rear point of the vehicle in the ROI area are (w / 2,3h / 4);

[0015] For the rear image of the non-initial frame, use the rear point detected in the previous frame as the reference point, set the ROI area with a width of w and a height of h, and take the upper left corner of the ROI area as the coordinate origin, so that the coordinates of the rear point in the ROI area are (w / 2, 3h / 4).

[0016] On the basis of the above technical solution, preferably, the performing vertical edge detection on the ROI area to extract the candidate rear vertical edge line specifically includes:

[0017] Extract the vertical edge image of the ROI area using the Sobel operator;

[0018] In the vertical edge image, the vertical edge cumulative values ​​of each column of pixels are accumulated to obtain a vertical edge cumulative value sequence;

[0019] Use Gaussian filtering to smooth the vertical edge cumulative value sequence, and find the peak value of the vertical edge cumulative value sequence and the horizontal coordinate corresponding to the peak value;

[0020] The abscissas corresponding to each peak value are formed into an abscissa array;

[0021] The straight line perpendicular to the horizontal axis at the horizontal coordinate corresponding to each peak value is taken as a candidate vertical edge line of the rear end of the vehicle.

[0022] On the basis of the above technical solution, preferably, the performing straight line detection on the edge image to extract the edge line of the bottom of the vehicle body specifically includes:

[0023] For the rear image of the vehicle in the initial frame, Hough transform is used to perform line detection in the extracted edge image to extract multiple first candidate lines;

[0024] The projection coordinates of the ground projection point corresponding to the camera on the image are calculated according to the calibration module of the CMS system, and the straight line formed by the rear point in the rear image of the initial frame and the projection coordinates is used as the initial vehicle body reference line L;

[0025] Calculate the angles between each first candidate straight line and the initial vehicle body reference line L respectively, and filter out the first candidate straight lines whose angles with the initial vehicle body reference line L are greater than a preset angle threshold;

[0026] Calculate the rear point (u0, v0) and the projection coordinates (u c ,v c ) to each first candidate straight line, and select the first candidate straight line with the smallest sum of distances to the two points as the bottom edge line of the vehicle body.

[0027] On the basis of the above technical solution, preferably, the performing straight line detection on the edge image to extract the edge line of the bottom of the vehicle body further comprises:

[0028] For the non-initial frame rear image, calculate the intersection point P between the bottom edge line of the vehicle body and the image edge of the previous frame rear image, and calculate the horizontal distance d between the rear point of the previous frame rear image and the intersection point P w ;

[0029] The bottom edge line of the vehicle body in the previous frame of the rear image is taken as the center line, and the width d is expanded parallel to both sides to obtain the expanded area, where d = 2d w ;

[0030] Perform canny edge and Hough line detection in the expanded area to extract multiple second candidate lines;

[0031] The distances from the rear point and the intersection point P of the previous frame of the rear image to each second candidate straight line are calculated respectively, and the second candidate straight line with the smallest sum of distances to the two points is selected as the bottom edge line of the vehicle body.

[0032] On the basis of the above technical solution, preferably, the calculating the coordinates of the current vehicle rear point according to the candidate vehicle rear vertical edge line and the vehicle body bottom edge line specifically includes:

[0033] Substitute the horizontal coordinate of each candidate vertical edge line of the rear of the vehicle into the linear equation of the edge line of the bottom of the vehicle body to obtain the corresponding vertical coordinate;

[0034] The horizontal coordinate and the corresponding vertical coordinate of each candidate rear vertical edge line are used as candidate rear points;

[0035] Calculate the Euclidean distance between each candidate vehicle rear point and the vehicle rear point of the previous frame of vehicle rear image;

[0036] The candidate vehicle tail point with the smallest Euclidean distance is selected as the vehicle tail point coordinate of the current frame.

[0037] On the basis of the above technical solution, preferably, the field of view of the CMS system is adjusted in real time according to the current coordinates of the rear point of the vehicle, so that the rear of the vehicle is always within the preset optimal display range:

[0038] The horizontal visual field is clipped according to the coordinates of the rear point of the previous frame, so that the ratio of the horizontal width from the rear point to the edge of the image where the vehicle body is located to the width of the entire field of view does not exceed a preset ratio threshold. In a second aspect of the present invention, a semi-trailer rear detection system based on a CMS system is disclosed, the system comprising:

[0039] Data acquisition module: used to continuously collect the current rear image through the CMS system;

[0040] The vehicle rear edge detection module is used to set the ROI area in the vehicle rear image, perform vertical edge detection on the ROI area, and extract the candidate vertical edge line of the vehicle rear;

[0041] The vehicle body edge detection module is used to perform edge detection on the vehicle rear image to obtain an edge image, perform straight line detection on the edge image, and extract the edge line of the bottom of the vehicle body;

[0042] A vehicle rear point calculation module: used to calculate the coordinates of the current vehicle rear point according to the candidate vehicle rear vertical edge line and the vehicle body bottom edge line;

[0043] Field of view adjustment module: used to adjust the field of view of the CMS system in real time according to the current coordinates of the rear point of the vehicle, so that the rear of the vehicle is always within the preset optimal display range.

[0044] In a third aspect of the present invention, an electronic device is disclosed, comprising: at least one processor, at least one memory, a communication interface and a bus;

[0045] Wherein, the processor, memory, and communication interface communicate with each other via the bus;

[0046] The memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to implement the method according to the first aspect of the present invention.

[0047] According to a fourth aspect of the present invention, a computer-readable storage medium is disclosed, wherein the computer-readable storage medium stores computer instructions, and the computer instructions enable a computer to implement the method described in the first aspect of the present invention.

[0048] Compared with the prior art, the present invention has the following beneficial effects:

[0049] 1) Based on the existing CMS system, the present invention extracts candidate rear vertical edge lines and vehicle body bottom edge lines from the ROI area of ​​the rear image, and calculates the current rear point coordinates according to the candidate rear vertical edge lines and vehicle body bottom edge lines, thereby assisting in real-time adjustment of the field of view of the CMS system camera, so that the rear of the vehicle is always within the optimal field of view, facilitating observation of the environment behind the vehicle, and improving the driving safety of the semi-trailer.

[0050] 2) The present invention sets a ROI area for vehicle rear detection according to the coordinates of the vehicle rear in the previous frame, extracts the vertical edge image of the ROI area by using the Sobel operator, extracts the edge line of the bottom of the vehicle body by performing straight line detection on the edge image of the vehicle rear image, substitutes the horizontal coordinate of each candidate vehicle rear vertical edge line into the straight line equation of the bottom edge line of the vehicle body, obtains the corresponding vertical coordinate, thereby determining the candidate vehicle rear point, and finally calculates the Euclidean distance between each candidate vehicle rear point and the vehicle rear point of the vehicle rear image in the previous frame, and selects the candidate vehicle rear point with the smallest Euclidean distance as the coordinate of the vehicle rear point in the current frame, which makes full use of the temporal continuity of the video frames collected by the camera, helps to stabilize tracking in dynamic scenes, and avoids incorrect positioning due to drastic image changes or detection noise.

[0051] 3) The present invention extracts multiple candidate straight lines through straight line measurement, and fully utilizes the geometric features of the semi-trailer to screen out the bottom edge line of the vehicle body, wherein, for the initial frame rear image, the first candidate straight line whose angle with the initial vehicle body reference line L is greater than the preset angle threshold is filtered out; the first candidate straight line with the smallest sum of distances from the rear point and the projection center point to each first candidate straight line is selected as the bottom edge line of the vehicle body; for the non-initial frame rear image, multiple second candidate straight lines are extracted in the extended area, and the second candidate straight line with the smallest sum of distances from the rear point and the intersection P of the previous frame rear image to each second candidate straight line is selected as the bottom edge line of the vehicle body; that is, the present invention can quickly exclude candidate lines that are far away from the vehicle body or irrelevant in the presence of noise and interference, and reliably identify the bottom edge line of the vehicle body; at the same time, the correlation between the two frames of images is used for detection, so that the recognition of the rear point can be more stable, which helps to maintain the reliability of tracking in high-speed or complex scenes.

[0052] 4) In the process of calculating the bottom edge line of the vehicle body and the coordinates of the rear point of the vehicle, the present invention selects the optimal solution through distance calculation, and can quickly obtain the result with less calculation amount, which is suitable for the needs of real-time application environment and improves the efficiency of the algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0054] Figure 1 This is a flow chart of a method for detecting the rear end of a semi-trailer based on a CMS system of the present invention;

[0055] Figure 2 A schematic diagram of the ROI area set for the present invention;

[0056] Figure 3 is the horizontal distance d between the rear point of the previous frame and the intersection point P w Schematic diagram of

[0057] Figure 4 is a schematic diagram of an expanded area obtained by expanding the width d to both sides in parallel with the bottom edge line of the vehicle body of the previous frame of the rear image as the center line;

[0058] Figure 5 A schematic diagram of adjusting the field of view of the CMS system in real time according to the current coordinates of the rear point of the vehicle. DETAILED DESCRIPTION

[0059] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0060] See also Figure 1 The present invention discloses a method for detecting the rear end of a semitrailer based on a CMS system, the method comprising:

[0061] S1. Continuously collect the current rear image of the vehicle through the CMS system, and set the ROI area in the rear image of the vehicle.

[0062] Step S1 specifically includes the following sub-steps:

[0063] S11 . For the initial frame vehicle rear image, the position coordinates of the vehicle rear point on the image are obtained according to the calibration module of the existing semi-trailer CMS system.

[0064] In an embodiment of the present invention, the camera of the CMS system continuously collects the current rear image, and after the calibration module of the existing semi-trailer CMS system is used to calibrate the coordinates of the rear point in the rear image, the position (u0, v0) of the rear on the image can be obtained.

[0065] S12, taking the rear point of the vehicle as a reference point, setting a ROI area with a width of w and a height of h.

[0066] In the embodiment of the present invention, the specific requirements for setting the ROI area are: taking the upper left corner of the ROI area as the coordinate origin, making the coordinates of the reference point in the ROI area (w / 2, 3h / 4), such as Figure 2 Shown is a schematic diagram of the ROI area set in the present invention.

[0067] S13. For a non-initial frame vehicle rear image, use the vehicle rear point detected in the previous frame as a reference point and set a ROI region with a width of w and a height of h.

[0068] Similarly, the specific requirements for setting the ROI area are: taking the upper left corner of the ROI area as the coordinate origin, the coordinates of the reference point in the ROI area are (w / 2, 3h / 4).

[0069] S2. Perform vertical edge detection on the ROI area to extract candidate vertical edge lines of the rear of the vehicle.

[0070] Step S2 specifically includes the following sub-steps:

[0071] S21. Extract the vertical edge image of the ROI region using the Sobel operator.

[0072] Extract the Sobel vertical edge in the ROI area, where the size of the Sobel operator is 3×3. In the ROI area, the vertical partial derivative G at position (i, j) is y The calculation formula for (i,j) is:

[0073] G y (i,j)=[f(i-1,j+1)+2f(i,j+1)+f(i+1,j+1)]-[f(i-1,j-1)+2f(i,j+1)+f(i+1,j+1)]

[0074] f(i,j) represents the pixel value at position (i,j) in the ROI area.

[0075] S22. In the vertical edge image, vertical edge cumulative values ​​of each column of pixels are accumulated along the horizontal axis x direction to obtain a vertical edge cumulative value sequence.

[0076] S23, using Gaussian filtering to smooth the vertical edge cumulative value sequence, finding the peak value of the vertical edge cumulative value sequence and the horizontal coordinate corresponding to the peak value, and forming the horizontal coordinates corresponding to each peak value into a horizontal coordinate array.

[0077] In the embodiment of the present invention, the specific search process is as follows:

[0078] (1) Traverse each element in the sequence;

[0079] (2) Determine whether the current element is a peak: If the current element is larger than its left and right adjacent elements, it is a peak. Record the peak value and its corresponding horizontal coordinate value peak_x k , skip if it is not a peak value and traverse the next element;

[0080] (3) filtering elements whose peak values ​​are lower than the threshold value through a preset peak threshold;

[0081] (4) Calculate peak_x k The horizontal distance from the rear point of the previous frame is filtered out through a preset distance threshold, with the elements whose distance is higher than the distance threshold.

[0082] (5) Traverse the entire sequence and obtain an array containing the horizontal coordinates corresponding to multiple peaks (peax_x0, peax_x1, ..., peak_x k , ..., peak_x K ), where k = 1, 2, …, K, and K is the total number of peaks.

[0083] S24. Take the straight line perpendicular to the horizontal axis at the horizontal coordinate corresponding to each peak value as a candidate vertical edge line of the rear end of the vehicle.

[0084] For each peak value corresponding to the horizontal axis, the corresponding straight line x=peak_x k It is a candidate vertical edge line of the rear of the vehicle.

[0085] S3, performing edge detection on the rear image to obtain an edge image, performing straight line detection on the edge image, and extracting an edge line of the bottom of the vehicle body.

[0086] Step S3 specifically includes the following sub-steps:

[0087] S31, for the initial frame rear image, the bottom edge line of the vehicle body.

[0088] S311 . For the vehicle rear image in the initial frame, perform canny edge detection on the entire vehicle rear image, use Hough transform to perform straight line detection in the extracted edge image, and extract multiple first candidate straight lines.

[0089] S312, calculating the projection coordinates of the ground projection point corresponding to the camera on the image according to the calibration module of the CMS system, and taking the straight line formed by the rear point in the rear image of the initial frame and the projection coordinates as the initial vehicle body reference line L.

[0090] Specifically, according to the calibration parameters, the projection coordinates (u c ,v c ), connect the calibrated rear point (u0,v0) and the projection coordinate (u c ,v c ) two points and use the straight line formed as the initial vehicle body reference line L.

[0091] S313, respectively calculating the angles between each first candidate straight line and the initial vehicle body reference line L, and filtering out the first candidate straight lines whose angles with the initial vehicle body reference line L are greater than a preset angle threshold.

[0092] S314, respectively calculating the distances from the vehicle rear point and the projection coordinates to each first candidate straight line, and selecting the first candidate straight line with the smallest sum of distances from the two points as the bottom edge line of the vehicle body.

[0093] Specifically, calculate the rear point (u0, v0) and the projection coordinates (u c ,v c ) to each straight line, and select the first candidate straight line with the smallest sum of distances to the two points as the bottom edge line of the vehicle body. The straight line equation of the bottom edge line of the vehicle body is expressed as y=kx+b.

[0094] S32: For the non-initial frame rear end image, extract the bottom edge line of the vehicle body.

[0095] S321, for a non-initial frame rear image, calculate the intersection point P between the bottom edge line of the vehicle body and the image edge of the previous frame rear image, and calculate the horizontal distance d between the rear point of the previous frame rear image and the intersection point P w .

[0096] like Figure 3 The horizontal distance d between the rear point of the previous frame of the rear image and the intersection point P is shown. w , wherein P is the intersection point of the bottom edge line of the vehicle body of the previous frame of the rear image and the edge of the rear image.

[0097] S322, taking the bottom edge line of the vehicle body of the previous frame of the rear image as the center line, and expanding in parallel to both sides to obtain an expansion area.

[0098] like Figure 4 The figure shows a schematic diagram of the expansion area obtained by expanding the width d to both sides in parallel with the bottom edge line of the vehicle body of the previous frame of the rear image as the center line, where d = 2d w, the dotted line in the middle is the bottom edge line of the vehicle body in the previous frame of the rear image.

[0099] S323, performing canny edge and Hough line detection in the expanded area to extract multiple second candidate lines.

[0100] Since the vehicle body movement between adjacent frames does not change much in continuous frames, the present invention uses the detection result of the bottom edge line of the vehicle body of the previous frame of the rear image to constrain the detection area of ​​the current frame, calculates the intersection point P of the previous frame straight line and the edge of the image, and calculates the horizontal distance d between the rear point and P. w , taking the straight line of the previous frame as the center, expand the width d parallel to both sides, and extract the bottom edge line of the vehicle body of the current frame in the expanded area.

[0101] S324, respectively calculating the distances from the vehicle rear point and the intersection point P of the previous frame of the vehicle rear image to each second candidate straight line, and selecting the second candidate straight line with the smallest sum of distances from the two points as the bottom edge line of the vehicle body.

[0102] Assume that the formula of a second candidate line is Ax+By+C=0, and the distance from the intersection point P(x0,y0) to the second candidate line is D:

[0103]

[0104] In the expanded area, the second candidate straight line closest to the detection result of the previous frame of the rear image is the bottom edge line of the vehicle body in the current frame, and the straight line equation of the bottom edge line of the vehicle body is expressed as y=kx+b.

[0105] The present invention extracts multiple candidate straight lines through straight line measurement, and fully utilizes the geometric features of the semi-trailer to screen out the bottom edge line of the vehicle body, wherein, for the initial frame rear image, the first candidate straight line whose angle with the initial vehicle body reference line L is greater than a preset angle threshold is filtered out; the first candidate straight line whose sum of distances from the rear point and the projection center point to each first candidate straight line is the smallest is selected as the bottom edge line of the vehicle body; for the non-initial frame rear image, multiple second candidate straight lines are extracted in the extended area, and the second candidate straight line whose sum of distances from the rear point and the intersection point P of the previous frame rear image to each second candidate straight line is the smallest is selected as the bottom edge line of the vehicle body, and in the presence of noise and interference, the candidate lines that are far away from the vehicle body or irrelevant can be quickly excluded, and the bottom edge line of the vehicle body can be reliably identified; at the same time, the correlation between the two frames of images is used for detection, so that the recognition of the rear point can be more stable, which helps to maintain the reliability of tracking in high-speed or complex scenes.

[0106] S4. Calculate the coordinates of the current vehicle rear point according to the candidate vehicle rear vertical edge line and the vehicle body bottom edge line.

[0107] The vertical edge line of the rear end and the edge line of the bottom of the vehicle body together constitute the body geometric features of the semi-trailer. The present invention uses these body geometric features to calculate the coordinates of the current rear end point.

[0108] Specifically, the horizontal coordinates of each candidate rear vertical edge line (peax_x0, peax_x1, ..., peak_x k , ..., peak_x K ) into the straight line equation of the bottom edge of the vehicle body, and obtain the corresponding ordinates (peax_y0, peax_y1, ..., peak_y k , ..., peak_y K ).

[0109] The horizontal coordinate and the corresponding vertical coordinate of each candidate rear vertical edge line are used as candidate rear points (peax_x0, peax_y0), (peax_x1, peax_y1), ..., (peak_x k , peak_y k ),...,(peak_x K , peak_y K ).

[0110] Calculate the Euclidean distance between each candidate vehicle rear point and the vehicle rear point of the previous frame of vehicle rear image;

[0111] The candidate vehicle tail point with the smallest Euclidean distance is selected as the vehicle tail point coordinate of the current frame.

[0112] The invention sets a ROI area for vehicle rear detection according to the vehicle rear coordinates of the previous frame, extracts a vertical edge image of the ROI area by using a Sobel operator, extracts the edge line of the bottom of the vehicle body by performing straight line detection on the edge image of the vehicle rear image, substitutes the horizontal coordinates of each candidate vehicle rear vertical edge line into the straight line equation of the bottom edge line of the vehicle body respectively, obtains the corresponding vertical coordinate, thereby determining the candidate vehicle rear point, and finally calculates the Euclidean distance between each candidate vehicle rear point and the vehicle rear point of the vehicle rear image of the previous frame, and selects the candidate vehicle rear point with the smallest Euclidean distance as the vehicle rear point coordinate of the current frame, which makes full use of the temporal continuity of the video frames collected by the camera, is conducive to stable tracking in dynamic scenes, and avoids erroneous positioning caused by drastic image changes or detection noise.

[0113] In addition, in the process of calculating the bottom edge line of the vehicle body and the coordinates of the rear point of the vehicle, the present invention selects the optimal solution through distance calculation, and can quickly obtain results with less calculation amount, which is suitable for the needs of real-time application environment and improves the efficiency of the algorithm.

[0114] S5. Adjust the field of view of the CMS system in real time according to the current coordinates of the rear point of the vehicle, so that the rear of the vehicle is always within the preset optimal display range.

[0115] The visible field of view in the horizontal direction is cropped according to the coordinates of the vehicle rear point of the previous frame, so that the ratio of the horizontal width from the vehicle rear point to the edge of the image where the vehicle body is located to the entire field of view width does not exceed a preset ratio threshold.

[0116] Specifically, Figure 5 The figure shows a schematic diagram of adjusting the field of view of the CMS system in real time according to the current coordinates of the rear point of the vehicle. By cutting the visible field of view in the horizontal direction, the horizontal width W from the rear point of the vehicle to the edge of the image where the vehicle body is located can be reduced. r The proportion does not exceed 80% of the entire field of view width W, reducing accidents caused by obstructed vision.

[0117] The present invention is based on the existing CMS system, extracts candidate vertical edge lines of the rear end of the vehicle and the bottom edge lines of the vehicle body from the ROI area of ​​the rear end image, and calculates the coordinates of the current rear end point according to the candidate vertical edge lines of the rear end of the vehicle and the bottom edge lines of the vehicle body, thereby assisting in real-time adjustment of the field of view of the CMS system camera, so that the rear end of the vehicle is always within the optimal field of view, which is convenient for observing the environment behind the vehicle and improving the driving safety of the semi-trailer.

[0118] Corresponding to the above method embodiment, the present invention further discloses a semi-trailer rear end detection system based on a CMS system, the system comprising:

[0119] Data acquisition module: used to continuously collect the current rear image through the CMS system;

[0120] The vehicle rear edge detection module is used to set the ROI area in the vehicle rear image, perform vertical edge detection on the ROI area, and extract the candidate vertical edge line of the vehicle rear;

[0121] The vehicle body edge detection module is used to perform edge detection on the vehicle rear image to obtain an edge image, perform straight line detection on the edge image, and extract the edge line of the bottom of the vehicle body;

[0122] A vehicle rear point calculation module: used to calculate the coordinates of the current vehicle rear point according to the candidate vehicle rear vertical edge line and the vehicle body bottom edge line;

[0123] Field of view adjustment module: used to adjust the field of view of the CMS system in real time according to the current coordinates of the rear point of the vehicle, so that the rear of the vehicle is always within the preset optimal display range.

[0124] The above system embodiments and method embodiments correspond one to one, and for a brief description of the system embodiments, please refer to the method embodiments.

[0125] The present invention also discloses an electronic device, comprising: at least one processor, at least one memory, a communication interface and a bus; wherein the processor, memory, and communication interface communicate with each other through the bus; the memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to implement the aforementioned method of the present invention.

[0126] The present invention also discloses a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, wherein the computer instructions enable the computer to implement all or part of the steps of the method described in the embodiment of the present invention. The storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk, and other media that can store program codes.

[0127] The system embodiment described above is merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be distributed to multiple network units. A person skilled in the art may select some or all of the modules according to actual needs to achieve the purpose of the solution of this embodiment without creative effort.

[0128] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for detecting the rear end of a semi-trailer based on a CMS system, characterized in that: The method comprises: The current rear image of the vehicle is continuously collected through the CMS system, and the ROI area is set in the rear image of the vehicle; Performing vertical edge detection on the ROI area to extract candidate vertical edge lines of the rear of the vehicle; Performing edge detection on the rear image to obtain an edge image, performing straight line detection on the edge image to extract the edge line of the bottom of the vehicle body; Calculating the coordinates of the current vehicle rear point according to the candidate vehicle rear vertical edge line and the vehicle body bottom edge line; The field of view of the CMS system is adjusted in real time according to the current coordinates of the rear point of the vehicle, so that the rear of the vehicle is always within the preset optimal display range.

2. The method for detecting the rear end of a semitrailer based on a CMS system according to claim 1, characterized in that: The step of setting the ROI region in the rear image specifically includes: For the initial frame vehicle rear image, the position coordinates of the vehicle rear point on the image are obtained according to the calibration module of the CMS system; Taking the rear point of the vehicle as the reference point, set the ROI area with a width of w and a height of h, and take the upper left corner of the ROI area as the coordinate origin, so that the coordinates of the rear point of the vehicle in the ROI area are (w / 2,3h / 4); For the rear image of the non-initial frame, the rear point detected in the previous frame is used as the reference point, and the ROI area with a width of w and a height of h is set. The upper left corner of the ROI area is used as the coordinate origin, so that the coordinates of the rear point in the ROI area are (w / 2, 3h / 4).

3. The method for detecting the rear end of a semitrailer based on a CMS system according to claim 2, characterized in that: The performing vertical edge detection on the ROI region to extract candidate rear vertical edge lines specifically includes: Extract the vertical edge image of the ROI area using the Sobel operator; In the vertical edge image, the vertical edge cumulative values ​​of each column of pixels are accumulated to obtain a vertical edge cumulative value sequence; Use Gaussian filtering to smooth the vertical edge cumulative value sequence, and find the peak value of the vertical edge cumulative value sequence and the horizontal coordinate corresponding to the peak value; The abscissas corresponding to each peak value are formed into an abscissa array; The straight line perpendicular to the horizontal axis at the horizontal coordinate corresponding to each peak value is taken as a candidate vertical edge line of the rear end of the vehicle.

4. The method for detecting the rear end of a semitrailer based on a CMS system according to claim 1, characterized in that: The performing straight line detection on the edge image to extract the bottom edge line of the vehicle body specifically includes: For the rear image of the vehicle in the initial frame, Hough transform is used to perform line detection in the extracted edge image to extract multiple first candidate lines; The projection coordinates of the ground projection point corresponding to the camera on the image are calculated according to the calibration module of the CMS system, and the straight line formed by the rear point in the rear image of the initial frame and the projection coordinates is used as the initial vehicle body reference line L; Calculate the angles between each first candidate straight line and the initial vehicle body reference line L respectively, and filter out the first candidate straight lines whose angles with the initial vehicle body reference line L are greater than a preset angle threshold; The distances from the rear point and the projection coordinates to each first candidate straight line are calculated respectively, and the first candidate straight line with the smallest sum of distances to the two points is selected as the bottom edge line of the vehicle body.

5. The method for detecting the rear end of a semitrailer based on a CMS system according to claim 4, characterized in that: The performing straight line detection on the edge image to extract the bottom edge line of the vehicle body further comprises: For the non-initial frame rear image, calculate the intersection point P between the bottom edge line of the vehicle body and the image edge of the previous frame rear image, and calculate the horizontal distance d between the rear point of the previous frame rear image and the intersection point P w ; The bottom edge line of the vehicle body in the previous frame of the rear image is taken as the center line, and the width d is expanded parallel to both sides to obtain the expanded area, where d = 2d w ; Perform canny edge and Hough line detection in the expanded area to extract multiple second candidate lines; The distances from the rear point and the intersection point P of the previous frame of the rear image to each second candidate straight line are calculated respectively, and the second candidate straight line with the smallest sum of distances to the two points is selected as the bottom edge line of the vehicle body.

6. The method for detecting the rear end of a semitrailer based on a CMS system according to claim 3, characterized in that: The calculating of the current vehicle rear point coordinates according to the candidate vehicle rear vertical edge line and the vehicle body bottom edge line specifically includes: Substitute the horizontal coordinate of each candidate vertical edge line of the rear of the vehicle into the linear equation of the edge line of the bottom of the vehicle body to obtain the corresponding vertical coordinate; The horizontal coordinate and the corresponding vertical coordinate of each candidate rear vertical edge line are used as candidate rear points; Calculate the Euclidean distance between each candidate vehicle rear point and the vehicle rear point of the previous frame of vehicle rear image; The candidate vehicle tail point with the smallest Euclidean distance is selected as the vehicle tail point coordinate of the current frame.

7. The method for detecting the rear end of a semitrailer based on a CMS system according to claim 6, characterized in that: The real-time adjustment of the field of view of the CMS system according to the current coordinates of the rear point of the vehicle so that the rear of the vehicle is always within the preset optimal display range specifically includes: The visible field of view in the horizontal direction is cropped according to the coordinates of the vehicle rear point of the previous frame, so that the ratio of the horizontal width from the vehicle rear point to the edge of the image where the vehicle body is located to the entire field of view width does not exceed a preset ratio threshold.

8. A semi-trailer rear end detection system based on a CMS system, characterized in that: The system comprises: Data acquisition module: used to continuously collect the current rear image through the CMS system; The vehicle rear edge detection module is used to set the ROI area in the vehicle rear image, perform vertical edge detection on the ROI area, and extract the candidate vertical edge line of the vehicle rear; The vehicle body edge detection module is used to perform edge detection on the vehicle rear image to obtain an edge image, perform straight line detection on the edge image, and extract the edge line of the bottom of the vehicle body; A vehicle rear point calculation module: used to calculate the coordinates of the current vehicle rear point according to the candidate vehicle rear vertical edge line and the vehicle body bottom edge line; Field of view adjustment module: used to adjust the field of view of the CMS system in real time according to the current coordinates of the rear point of the vehicle, so that the rear of the vehicle is always within the preset optimal display range.

9. An electronic device, characterized in that: include: at least one processor, at least one memory, a communication interface, and a bus; Wherein, the processor, memory, and communication interface communicate with each other via the bus; The memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are executed by a processor to enable a computer to implement the method according to any one of claims 1 to 7.

Images