Method and system for automatically measuring width of trailer

Abstract

The invention relates to the technical field of trailer width measurement, and particularly discloses a trailer width automatic measurement method and system, and the method comprises the steps: judging whether a trailer width detection condition is satisfied after a vehicle is ignited, and outputting a default trailer width if the trailer width detection condition is not satisfied; if yes, performing image denoising on the plurality of left and right rearview images, inputting the denoised left and right rearview images into a pre-trained neural network for segmentation, and obtaining a left lower edge detection point and a right lower edge detection point of the trailer; performing straight line fitting on the left lower edge detection point and the right lower edge detection point of the trailer to obtain a left lower edge straight line and a right lower edge straight line of the trailer; and calculating the width of the trailer based on the left-side lower edge straight line and the right-side lower edge straight line of the trailer. According to the method, the automatic driving system can be suitable for trailers with different widths, and the situation that the vehicle frequently presses the line and even collides with other vehicles due to the fact that the vehicle with the wrong width is used can be prevented.

Description

Technical Field

[0001] The invention relates to the technical field of trailer width measurement, and in particular to a method and system for automatically measuring trailer width. Background Art

[0002] Truck autonomous driving can save fuel, relieve driver fatigue, and improve driving comfort and safety, and has great commercial prospects in road transportation. However, due to the difference in structure between trucks and passenger cars, the autonomous driving algorithm suitable for trucks needs to have specific functions.

[0003] The schematic diagram of the truck is as follows: Figure 1 As shown in the figure, its structure is divided into two parts: tractor 1 and trailer 2. In the actual operation of the truck, trailers of different sizes often need to be replaced according to the different goods being transported. The common trailer width is 2.5m to 3m. The standard lane width of my country's highways is 3.75m. For a trailer with a width of 3m, when the vehicle is completely centered, there is only 37.5cm of remaining space on the left and right sides. Therefore, different trailer widths have a great impact on auxiliary driving functions such as lane keeping and lane departure warning in the autonomous driving system, as well as modules such as planning, decision-making and control. Therefore, realizing automatic perception of the width of the trailer is very important for the commercialization of the truck autonomous driving system. Summary of the invention

[0004] In view of the above problems, the first object of the present invention is to provide a method for automatically measuring the width of a trailer, which enables the automatic driving system to be applicable to trailers of different widths and can prevent the vehicle from frequently crossing the line or even colliding with other vehicles due to the use of the wrong vehicle width.

[0005] The second object of the present invention is to provide the above-mentioned trailer width automatic measurement system, which only reuses the most basic sensor in the automatic driving system: the rearview camera, and does not need to add additional sensors. It is low-cost and simple to implement.

[0006] The first technical solution adopted by the present invention is: a method for automatically measuring the width of a trailer, comprising the following steps:

[0007] S100: After the vehicle is ignited, determine whether the trailer width detection condition is met. If so, execute step S200; if not, output the default trailer width;

[0008] S200: Acquire a plurality of left rear view images and right rear view images, and obtain a left lower edge straight line and a right lower edge straight line of the trailer based on the plurality of left rear view images and right rear view images;

[0009] S300: Calculating the width of the trailer based on the left lower edge straight line and the right lower edge straight line;

[0010] Wherein, step S200 includes:

[0011] Performing image denoising on the plurality of left rear view images and right rear view images to obtain denoised left rear view images and right rear view images; and inputting the denoised left rear view images and right rear view images into a pre-trained neural network for segmentation to obtain left lower edge detection points and right lower edge detection points of the trailer; performing straight line fitting on the left lower edge detection points and right lower edge detection points to obtain left lower edge straight lines and right lower edge straight lines of the trailer.

[0012] Preferably, the trailer width detection condition in step S100 includes that the vehicle is in a straight-moving state and the rearview camera meets working conditions.

[0013] Preferably, the default trailer width in step S100 is 2.88 m.

[0014] Preferably, the image denoising in step S200 includes multi-frame weighted denoising and edge enhancement operations.

[0015] Preferably, the neural network in step S200 includes one of a CNN network, a Transformer network, a U-Net, an E-Net, a PSPNet and a SegNet.

[0016] Preferably, the neural network in step S200 is trained in the following manner:

[0017] Obtain rear view image data with the lower edge of the trailer marked;

[0018] Performing a data enhancement operation on the rear view image data with the lower edge of the trailer marked, thereby obtaining rear view image enhancement data with the lower edge of the trailer marked;

[0019] The neural network is trained based on the rear view image enhancement data of the marked lower edge of the trailer to obtain a trained neural network.

[0020] Preferably, the data enhancement operation includes one or more of geometric transformation, color transformation and noise addition operations.

[0021] Preferably, step S200 includes:

[0022] The left lower edge detection point and the right lower edge detection point are subjected to straight line fitting by using the least square method or the Hough transform method to obtain the left lower edge straight line and the right lower edge straight line of the trailer.

[0023] Preferably, step S300 includes:

[0024] The left lower edge straight line and the right lower edge straight line of the trailer are sampled respectively to obtain multiple weighted lateral positions of the left lower edge straight line and the right lower edge straight line to the ground, that is, to obtain the lateral projection positions of the left lower edge straight line and the right lower edge straight line;

[0025] The trailer width is calculated based on the lateral projection positions of the left lower edge straight line and the right lower edge straight line.

[0026] The second technical solution adopted by the present invention is: a trailer width automatic measurement system, including a judgment module, a detection module and a calculation module;

[0027] The judgment module is used to judge whether the trailer width detection condition is met after the vehicle is ignited. If so, the detection module is called; if not, a default trailer width is output;

[0028] The detection module is used to obtain a plurality of left rear view images and a plurality of right rear view images, and obtain a left lower edge straight line and a right lower edge straight line of the trailer based on the plurality of left rear view images and the right rear view images;

[0029] The calculation module is used to calculate the trailer width based on the left lower edge straight line and the right lower edge straight line;

[0030] The detection module is used to perform the following operations:

[0031] Performing image denoising on the plurality of left rear view images and right rear view images to obtain denoised left rear view images and right rear view images; and inputting the denoised left rear view images and right rear view images into a pre-trained neural network for segmentation to obtain left lower edge detection points and right lower edge detection points of the trailer; performing straight line fitting on the left lower edge detection points and right lower edge detection points to obtain left lower edge straight lines and right lower edge straight lines of the trailer.

[0032] Beneficial effects of the above technical solution:

[0033] (1) The present invention discloses an automatic trailer width measurement method, which enables the automatic driving system to be applicable to trailers of different widths; it can provide more accurate vehicle information for auxiliary driving functions such as lane keeping and lane departure warning, and downstream planning, decision-making and control modules, to prevent the frequent crossing of the line or even collision with other vehicles due to the use of the wrong vehicle width.

[0034] (2) Compared with the current system for measuring trailer width based on laser radar, the automatic trailer width measurement system disclosed in the present invention only reuses the most basic sensor (rearview camera) in the automatic driving system to detect the trailer width, which can avoid the additional economic cost brought by adding sensors such as laser radar. Compared with the higher cost of laser radar, the automatic trailer width measurement system disclosed in the present invention has the advantages of full automation of detection, high accuracy, low cost and simple implementation.

[0035] (3) The present invention realizes the function of automatically detecting the width of the trailer, so that the automatic driving system is suitable for trailers of different widths; and no additional sensors are required for automatic detection of the trailer width, which is low-cost; at the same time, the automatic trailer width measurement method only performs detection within a short period of time after the vehicle is ignited, which has low overhead and is easy to implement and deploy. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 A schematic diagram of a truck structure in the background technology of the present invention;

[0037] Figure 2 A schematic flow chart of a method for automatically measuring trailer width provided by one embodiment of the present invention;

[0038] Figure 3 A schematic diagram of a trailer edge detection process provided by an embodiment of the present invention;

[0039] Among them, 1-tractor; 2-trailer. DETAILED DESCRIPTION

[0040] The following detailed description of the embodiments of the present invention is further described in detail in conjunction with the accompanying drawings and examples. The detailed description of the following embodiments and the accompanying drawings are used to exemplarily illustrate the principles of the present invention, but cannot be used to limit the scope of the present invention, that is, the present invention is not limited to the preferred embodiments described, and the scope of the present invention is defined by the claims.

[0041] In the description of the present invention, it should be noted that, unless otherwise specified, “plurality” means two or more than two; the terms “first”, “second”, etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance; for ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances.

[0042] Embodiment 1

[0043] like Figure 2 As shown, one embodiment of the present invention provides a method for automatically measuring the width of a trailer, comprising the following steps:

[0044] S100: After the vehicle is ignited, determine whether the trailer width detection condition is met. If so, execute step S200; if not, output the default trailer width;

[0045] Trucks usually replace trailers with the engine off, so the detection is only started after the vehicle is ignited. After the vehicle is ignited, it is determined whether the trailer width detection conditions are met. The trailer width detection conditions include but are not limited to the vehicle being in a straight-ahead state and the rearview camera meeting the working conditions (for example, the rearview camera is not blocked and the lighting conditions are insufficient); if the trailer width detection conditions are met, the trailer width detection operation is performed; otherwise, the trailer width detection operation is skipped and the default trailer width is directly output, which is 2.88m; the detection is exited after a stable detection value is obtained.

[0046] S200: Acquire a plurality of left rear view images and right rear view images, and obtain a left lower edge straight line and a right lower edge straight line of the trailer based on the plurality of left rear view images and right rear view images;

[0047] The rearview cameras are respectively installed at the left rearview mirror and the right rearview mirror of the truck, and a plurality of left rearview images and right rearview images are acquired based on the rearview cameras or other sensors;

[0048] like Figure 3 As shown, obtaining the left lower edge straight line and the right lower edge straight line of the trailer based on the plurality of left rear view images and the right rear view images includes the following sub-steps:

[0049] S210: performing image denoising on the plurality of left rear view images and right rear view images to obtain denoised left rear view images and right rear view images;

[0050] Input a plurality of left and right rear view images. Here, in order to reduce the amount of calculation, the left rear view images and the right rear view images can be input alternately. Perform multi-frame weighted denoising and edge enhancement operations on the input plurality of left rear view images and right rear view images to achieve image denoising.

[0051] Among them, considering that the position of the trailer is relatively fixed in multiple frames of images, the pixels of several left rear view images or right rear view images can be weighted averaged to achieve multi-frame weighted denoising and obtain more accurate denoised pixel values; multi-frame weighted denoising is better for removing complex noise and processing image sequences with dynamic scenes;

[0052] After completing multi-frame weighted denoising, several left rear view images or right rear view images are fused and adjusted, and a sharpening filter is used to enhance the edge of the image to achieve edge enhancement operation, thereby obtaining denoised left rear view images and right rear view images, making the images look clearer and more natural.

[0053] S220: Input the denoised left rear view image and the right rear view image into a pre-trained neural network for segmentation to obtain the left lower edge detection point and the right lower edge detection point of the trailer; perform straight line fitting on the left lower edge detection point and the right lower edge detection point of the trailer to obtain the left lower edge straight line and the right lower edge straight line of the trailer.

[0054] Neural networks include but are not limited to CNN networks, Transformer networks, U-Net, E-Net, PSPNet, SegNet, etc.; Neural network construction includes: encoder, decoder and output layer design;

[0055] The encoder design includes: using methods including but not limited to multi-layer convolution and pooling;

[0056] The decoder design includes: upsampling the low-resolution feature map obtained by the encoder by methods such as but not limited to deconvolution and bilinear difference to restore it to a size close to the original image, and also fusing the feature maps of different levels in the encoder with the feature maps of the corresponding level of the decoder, so that the decoder can simultaneously use low-level detail information and high-level semantic information to enrich the segmentation features;

[0057] The output layer design includes: according to the number of categories of the segmentation task, using a Softmax or Sigmoid activation function in the output layer to map the feature vector of each pixel into a category probability distribution, thereby determining the category to which each pixel belongs.

[0058] Neural network training: Obtain rear-view images of different trailers from internal road test data (covering different scenes, angles, lighting conditions, etc.), and use annotation to mark the lower edge of the trailer in the rear-view image to obtain rear-view image data with the lower edge of the trailer marked;

[0059] The rear-view image data of the marked lower edge of the trailer is enhanced, including: performing geometric transformation (random rotation, scaling, flipping, cropping, etc., to increase data diversity and improve the model's recognition ability of objects of different shapes), color transformation (adjusting image brightness, contrast, hue, saturation, etc., to simulate images under different lighting conditions and enhance the robustness of the model), adding noise, etc., to the rear-view image data of the marked lower edge of the trailer, thereby obtaining enhanced rear-view image data of the marked lower edge of the trailer;

[0060] The neural network is trained based on the rear view image enhancement data of the marked lower edge of the trailer to obtain a trained neural network; during training, the parameters of the model are given initial values ​​by random initialization or pre-trained model initialization; and the model can converge stably in the later stage of training by adjusting the training batch, learning rate and optimization method.

[0061] The denoised left rear view image and the right rear view image are input into the neural network for segmentation to obtain the left lower edge detection point and the right lower edge detection point of the trailer;

[0062] In order to obtain a more robust edge, the obtained lower edge detection points on the left side and the lower edge detection points on the right side of the trailer are fitted with a straight line by using the least square method, Hough transform and other methods to obtain the lower edge straight line on the left side and the lower edge straight line on the right side of the trailer.

[0063] S300: Calculate the width of the trailer based on the left lower edge straight line and the right lower edge straight line of the trailer, and output it;

[0064] Project the detected straight lines of the left and right lower edges of the trailer onto the top view to calculate the accurate trailer width, including:

[0065] The left lower edge straight line and the right lower edge straight line of the trailer are sampled respectively to obtain multiple weighted lateral positions of the left lower edge straight line and the right lower edge straight line of the trailer to the ground, that is, to obtain the lateral projection positions of the left lower edge straight line and the right lower edge straight line of the trailer;

[0066] The trailer width is calculated based on the lateral projection positions of the left lower edge straight line and the right lower edge straight line of the trailer, so as to obtain the accurate trailer width; when a stable trailer width detection value is obtained, the trailer width is output and the detection is ended.

[0067] Embodiment 2

[0068] An embodiment of the present invention provides a trailer width automatic measurement system, comprising a judgment module, a detection module and a calculation module;

[0069] The judgment module is used to judge whether the trailer width detection condition is met after the vehicle is ignited. If so, the detection module is called; if not, a default trailer width is output;

[0070] The detection module is used to obtain a plurality of left rear view images and a plurality of right rear view images, and obtain a left lower edge straight line and a right lower edge straight line of the trailer based on the plurality of left rear view images and the right rear view images;

[0071] The calculation module is used to calculate the trailer width based on the left lower edge straight line and the right lower edge straight line, and output it;

[0072] The detection module is used to perform the following operations:

[0073] Performing image denoising on the plurality of left rear view images and right rear view images to obtain denoised left rear view images and right rear view images; and inputting the denoised left rear view images and right rear view images into a pre-trained neural network for segmentation to obtain left lower edge detection points and right lower edge detection points of the trailer; performing straight line fitting on the left lower edge detection points and right lower edge detection points to obtain left lower edge straight lines and right lower edge straight lines of the trailer.

[0074] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in the embodiments of the present invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0075] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0076] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0077] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0078] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, ROM, RAM, magnetic disks, or optical disks.

[0079] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for automatically measuring trailer width, characterized in that: The following steps are involved: S100: After the vehicle is ignited, determine whether the trailer width detection condition is met. If so, execute step S200; if not, output the default trailer width; S200: Acquire a plurality of left rear view images and right rear view images, and obtain a left lower edge straight line and a right lower edge straight line of the trailer based on the plurality of left rear view images and right rear view images; S300: Calculating the width of the trailer based on the left lower edge straight line and the right lower edge straight line; Wherein, step S200 includes: Performing image denoising on the plurality of left rear view images and right rear view images to obtain denoised left rear view images and right rear view images; and inputting the denoised left rear view images and right rear view images into a pre-trained neural network for segmentation to obtain left lower edge detection points and right lower edge detection points of the trailer; performing straight line fitting on the left lower edge detection points and right lower edge detection points to obtain left lower edge straight lines and right lower edge straight lines of the trailer.

2. The method for automatically measuring trailer width according to claim 1, characterized in that: The trailer width detection conditions in step S100 include that the vehicle is in a straight-moving state and the rearview camera meets working conditions.

3. The method for automatically measuring trailer width according to claim 1, characterized in that: The default trailer width in step S100 is 2.88 m.

4. The method for automatically measuring trailer width according to claim 1, characterized in that: The image denoising in step S200 includes multi-frame weighted denoising and edge enhancement operations.

5. The method for automatically measuring trailer width according to claim 1, characterized in that: The neural network in step S200 includes one of a CNN network, a Transformer network, a U-Net, an E-Net, a PSPNet and a SegNet.

6. The method for automatically measuring trailer width according to claim 1, characterized in that: The neural network in step S200 is trained in the following manner: Obtain rear view image data with the lower edge of the trailer marked; Performing a data enhancement operation on the rear view image data with the lower edge of the trailer marked, thereby obtaining rear view image enhancement data with the lower edge of the trailer marked; The neural network is trained based on the rear view image enhancement data of the marked lower edge of the trailer to obtain a trained neural network.

7. The method for automatically measuring trailer width according to claim 6, characterized in that: The data enhancement operation includes one or more of a geometric transformation, a color transformation, and a noise addition operation.

8. The method for automatically measuring trailer width according to claim 1, characterized in that: Step S200 includes: The left lower edge detection point and the right lower edge detection point are subjected to straight line fitting by using the least square method or the Hough transform method to obtain the left lower edge straight line and the right lower edge straight line of the trailer.

9. The method for automatically measuring trailer width according to claim 1, characterized in that: Step S300 includes: The left lower edge straight line and the right lower edge straight line of the trailer are sampled respectively to obtain multiple weighted lateral positions of the left lower edge straight line and the right lower edge straight line to the ground, that is, to obtain the lateral projection positions of the left lower edge straight line and the right lower edge straight line; The trailer width is calculated based on the lateral projection positions of the left lower edge straight line and the right lower edge straight line.

10. A trailer width automatic measurement system, characterized in that: It includes a judgment module, a detection module and a calculation module; The judgment module is used to judge whether the trailer width detection condition is met after the vehicle is ignited. If so, the detection module is called; if not, a default trailer width is output; The detection module is used to obtain a plurality of left rear view images and a plurality of right rear view images, and obtain a left lower edge straight line and a right lower edge straight line of the trailer based on the plurality of left rear view images and the right rear view images; The calculation module is used to calculate the trailer width based on the left lower edge straight line and the right lower edge straight line; The detection module is used to perform the following operations: Performing image denoising on the plurality of left rear view images and right rear view images to obtain denoised left rear view images and right rear view images; and inputting the denoised left rear view images and right rear view images into a pre-trained neural network for segmentation to obtain left lower edge detection points and right lower edge detection points of the trailer; performing straight line fitting on the left lower edge detection points and right lower edge detection points to obtain left lower edge straight lines and right lower edge straight lines of the trailer.

Images