Method, device and equipment for determining folding angle of semi-trailer train and semi-trailer train

Abstract

The invention provides a method, a device and equipment for determining a folding angle of a semi-trailer train and the semi-trailer train. According to the method, at least one camera is installed on a first part of the semi-trailer train and faces a second part, the first part is a tractor, the second part is a trailer, or the first part is the trailer, and the second part is the tractor. When it is detected that the vehicle is in a turning state, obtaining a current frame image shot by each camera, and then carrying out key point detection processing according to each current frame image to obtain a key point position; and calculating a target folding angle according to the position of each key point and the obtained straight movement reference point position. According to the scheme, the folding angle is determined through the image shot by the camera, the joint of the tractor and the trailer does not need to be transformed, and cost is effectively reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of vehicles, in particular to a folding angle determination method, device and equipment of a semi-trailer train and the semi-trailer train. BACKGROUND

[0002] The semi-trailer train is composed of a tractor and a trailer, the tractor and the trailer are connected through a pin shaft, and the tractor pulls the trailer to move. The angle formed by the tractor and the trailer is called a folding angle or a hinged angle, and the folding angle is required in trajectory prediction, obstacle avoidance and other technologies, so it is necessary to determine the folding angle of the semi-trailer train.

[0003] In the prior art, the folding angle is usually determined by modifying the connection between the tractor and the trailer to install a grating encoder or install a magnet and a Hall sensor, and converting the data detected by the grating encoder or the Hall sensor to obtain the folding angle.

[0004] In summary, the existing folding angle determination method of the semi-trailer train needs to modify the semi-trailer train, resulting in high cost. SUMMARY

[0005] The folding angle determination method, device and equipment of the semi-trailer train provided by the embodiments of the present application solve the problem of high cost caused by the need to modify the semi-trailer train in the existing folding angle determination method of the semi-trailer train.

[0006] In a first aspect, the present application provides a folding angle determination method of a semi-trailer train, at least one camera is installed on a first part of the semi-trailer train and faces a second part, the first part is a tractor and the second part is a trailer, or the first part is a trailer and the second part is a tractor, and the method comprises:

[0007] When the vehicle is in a turning state, a current frame image captured by each camera is obtained, the turning state is a left turning state or a right turning state;

[0008] According to each current frame image, key point detection processing is performed to obtain a key point position;

[0009] According to each key point position and the obtained straight reference point position, a target folding angle is calculated.

[0010] In a possible implementation, if the number of cameras is greater than 1, the at least one camera is installed on the side surface of the first part, and the key point detection processing according to each current frame image to obtain the key point position comprises:

[0011] According to the turning state, a target image is selected from all current frame images;

[0012] Based on the target image, key point detection processing is performed to obtain the key point locations.

[0013] In one possible implementation, one or more of the at least one camera are mounted on the left side of the first portion, and the remaining camera is mounted on the right side of the first portion.

[0014] In one possible implementation, the step of performing key point detection processing based on the target image to obtain the key point locations includes:

[0015] The target image is subjected to distortion correction and viewpoint transformation processing to obtain a first bird's-eye view image;

[0016] Based on whether the folding angle, the first part, the turning state, and the first preset error angle have been determined in the previous frame of the target image, the detection area in the first bird's-eye view image is determined;

[0017] The detection area is processed using a contour extraction algorithm to obtain an initial set of contour points;

[0018] The target contour point set is obtained by filtering all initial contour point sets according to the preset filtering strategy.

[0019] The position of each point in each target contour point set in the first bird's-eye view image is used as the key point position.

[0020] In one possible implementation, determining the detection area in the first bird's-eye view image based on whether the folding angle, the first portion, the turning state, and the first preset error angle have been determined in the previous frame of the target image includes:

[0021] Based on the first part and the turning state, the target side is determined, which is either the left or the right side;

[0022] If the folding angle of the previous frame of the target image is not determined, then the target side half region in the first bird's-eye view image is taken as the detection region;

[0023] If the folding angle of the previous frame of the target image has been determined, then a first angle and a second angle are generated based on the folding angle and the first preset error angle.

[0024] The detection area is generated based on the first angle, the second angle, and the midpoint of the bottom edge of the first bird's-eye view image.

[0025] In one possible implementation, determining the target side based on the first portion and the turning state includes:

[0026] If the first part is a trailer and the turning state is a left turn, then the target side is determined to be the right side;

[0027] If the first part is a trailer and the turning state is a right turn, then the target side is determined to be the left side;

[0028] If the first part is a tractor and the turning state is a left turn, then the target side is determined to be the left side;

[0029] If the first part is a tractor and the turning state is a right turn, then the target side is determined to be the right side.

[0030] In one possible implementation, calculating the target folding angle based on the location of each key point and the obtained straight reference point location includes:

[0031] Based on the location of each key point, generate the turning line corresponding to each target contour point set;

[0032] Generate a straight line based on the position of the straight reference point;

[0033] Based on each turning line and the straight line, generate multiple initial folding angles;

[0034] The target fold angle is generated based on whether the fold angle has been determined in the previous frame of the target image, and each initial fold angle.

[0035] In one possible implementation, generating the target fold angle based on whether the fold angle has been determined in the previous frame of the target image, and each initial fold angle, includes:

[0036] If the folding angle of the previous frame of the target image is not determined, then the average value of all initial folding angles is taken as the target folding angle;

[0037] If the folding angle of the previous frame of the target image has been determined, then an error angle range is generated based on the folding angle and the second preset error angle.

[0038] The target folding angle is generated based on the error angle range and each initial folding angle.

[0039] In one possible implementation, the step of performing key point detection processing based on the target image to obtain the key point locations includes:

[0040] The target image is detected using a vehicle body edge key point detection model to obtain initial edge points. The vehicle body edge key point detection model is a pre-trained deep learning model for obtaining edges from images. The vehicle body edge key point detection model includes a backbone network, a neck network, a curve feature extraction layer, a feature fusion layer, and a key point regression head.

[0041] The target image is subjected to distortion correction and viewpoint transformation processing to obtain a second bird's-eye view image, and there is a correspondence between the points in the target image and the second bird's-eye view image;

[0042] The position of each point corresponding to the initial edge point in the second bird's-eye view is taken as the key point position.

[0043] In one possible implementation, calculating the target folding angle based on the location of each key point and the obtained straight reference point location includes:

[0044] Generate a turning line based on the locations of all key points;

[0045] Generate a straight line based on the position of the straight reference point;

[0046] The target folding angle is determined based on the turning line and the straight line.

[0047] In one possible implementation, the first part is a trailer, and the key point detection processing based on the target image to obtain the key point positions includes:

[0048] Within a preset detection area of ​​the target image, a matching process is performed based on a preset rearview mirror image to obtain the rearview mirror area;

[0049] The location of the center point of the rearview mirror area is taken as the location of the key point.

[0050] In one possible implementation, the straight-ahead reference point position includes a left straight-ahead point position and a right straight-ahead point position. The step of calculating the target folding angle based on each key point position and the acquired straight-ahead reference point position includes:

[0051] Based on the turning state, the target straight-ahead point position is determined from the left straight-ahead point position and the right straight-ahead point position;

[0052] Calculate the turning vector based on the key point locations and preset camera intrinsic parameters;

[0053] Calculate the straight-line vector based on the target straight-line point position and the preset camera intrinsic parameters;

[0054] The target folding angle is calculated based on the steering vector and the straight-line vector.

[0055] In one possible implementation, before acquiring the current frame image captured by each camera when the vehicle is detected to be turning, the method further includes:

[0056] Obtain the left and right images of the vertical line;

[0057] The left and right images of the straight line are respectively taken as target straight line images, and the following processing is performed:

[0058] Based on whether the target straight image is a straight left image, the target region is determined from the first preset region and the second preset region;

[0059] Contour extraction is performed on the straight-line image of the target to obtain the vehicle body contour points;

[0060] The first rearview mirror contour point is determined based on the target area and the vehicle body contour points;

[0061] Within the target area of ​​the target straight image, a matching process is performed based on a preset rearview mirror image to determine the contour points of the second rearview mirror.

[0062] Based on the first rearview mirror contour points and the second rearview mirror contour points, Kalman filtering is performed to obtain the target rearview mirror contour points.

[0063] If the target straight image is a straight left image, then the position of the center point of the target rearview mirror contour point is taken as the position of the left straight point;

[0064] If the target straight image is a straight right image, then the position of the center point of the target rearview mirror contour point is taken as the position of the right straight point.

[0065] In one possible implementation, the method further includes:

[0066] If the left turn signal is detected to be on, it is determined that the vehicle is in a left turn state.

[0067] If the right turn signal is detected to be on, it indicates that the vehicle is in a right turn state.

[0068] In one possible implementation, if the number of cameras is one, the first part is a trailer, and the camera is mounted at the front of the trailer, the step of performing key point detection processing based on each current frame image to obtain the key point positions includes:

[0069] Perform foreground segmentation processing on the current frame image to obtain the first tractor image;

[0070] The first tractor image is processed by feature point extraction to obtain multiple feature points and a feature vector for each feature point, wherein the feature vector includes the neighborhood gradient.

[0071] For each feature point, the position of the feature point in the first tractor image is taken as the key point position, and the feature vector of the feature point is taken as the feature vector of the key point position.

[0072] In one possible implementation, calculating the target folding angle based on the location of each key point and the obtained straight reference point location includes:

[0073] Based on the position of each straight reference point, obtain the feature vector of each straight reference point position;

[0074] Based on the feature vectors of each key point and the feature vectors of each straight reference point, a matching process is performed to obtain initial position pairs;

[0075] Based on the key point position and straight reference point position in each initial position pair, as well as the neighborhood gradient in the feature vector of the key point position and straight reference point position, a filtering process is performed to obtain the target position pair;

[0076] Generate a homography matrix based on all target location pairs;

[0077] The target folding angle is determined based on the homography matrix.

[0078] In one possible implementation, the method further includes:

[0079] Acquire the initial image captured by the camera in real time;

[0080] The initial image is segmented to obtain the second tractor image;

[0081] The image of the second tractor is subjected to contour extraction processing to obtain an initial contour;

[0082] The contour with the largest area among all initial contours is taken as the target contour;

[0083] The position of the center point of the bounding rectangle of the target contour in the initial image is used as the turning state detection position.

[0084] Based on the turning state detection location and the preset straight-ahead area, it is determined whether the vehicle is in a turning state.

[0085] In one possible implementation, the method further includes:

[0086] The target fold angle is processed by Kalman filtering to obtain the updated target fold angle.

[0087] Secondly, embodiments of this application provide a device for determining the folding angle of a semi-trailer train, comprising:

[0088] The acquisition module is used to acquire the current frame image captured by each camera when the vehicle is detected to be turning, wherein the turning state is either a left turn or a right turn.

[0089] The processing module is used to perform key point detection processing on each current frame image to obtain the key point positions;

[0090] The angle determination module is used to calculate the target folding angle based on the position of each key point and the position of the obtained straight reference point.

[0091] Thirdly, embodiments of this application provide an electronic device, including:

[0092] Processor, memory, communication interface;

[0093] The memory is used to store the executable instructions of the processor;

[0094] The processor is configured to execute the method for determining the folding angle of a semi-trailer train as described in any of the first aspects by executing the executable instructions.

[0095] Fourthly, embodiments of this application provide a semi-trailer train, including at least one camera and a controller;

[0096] The at least one camera is mounted on the first part of the semi-trailer and faces the second part, wherein the first part is the tractor and the second part is the trailer, or the first part is the trailer and the second part is the tractor;

[0097] The controller is used to execute the method for determining the folding angle of a semi-trailer as described in any of the first aspects above.

[0098] Fifthly, embodiments of this application provide a readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method for determining the folding angle of a semi-trailer train as described in any of the first aspects.

[0099] In a sixth aspect, embodiments of this application provide a computer program product, including a computer program, which, when executed by a processor, is used to implement the method for determining the folding angle of a semi-trailer train as described in any of the first aspects.

[0100] The method, apparatus, equipment, and semi-trailer for determining the folding angle of a semi-trailer provided in this application embodiment involve installing at least one camera on the first part of the semi-trailer, facing the second part, where the first part is the tractor and the second part is the trailer, or vice versa. When the vehicle is detected to be turning, the current frame image captured by each camera is acquired. Based on each current frame image, key point detection processing is performed to obtain the key point positions. Then, based on each key point position and the acquired straight-ahead reference point position, the target folding angle is calculated. This solution determines the folding angle using images captured by cameras, eliminating the need for modifications to the connection between the tractor and trailer, effectively reducing costs. Attached Figure Description

[0101] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0102] Figure 1 A schematic diagram of the folding angle provided in this application;

[0103] Figure 2 A flowchart illustrating an embodiment of the method for determining the folding angle of a semi-trailer provided in this application;

[0104] Figure 3a A flowchart illustrating Embodiment 2 of the method for determining the folding angle of a semi-trailer provided in this application;

[0105] Figure 3b A schematic diagram of the projection points provided in this application;

[0106] Figure 3c This is a schematic diagram of the detection area provided in this application;

[0107] Figure 3d A schematic diagram of the initial folding angle provided in this application;

[0108] Figure 4a A flowchart illustrating Embodiment 3 of the method for determining the folding angle of a semi-trailer provided in this application;

[0109] Figure 4b A schematic diagram illustrating the process of generating the target folding angle provided in this application;

[0110] Figure 4c A schematic diagram of the architecture of the vehicle body edge key point detection model provided in this application;

[0111] Figure 5a A flowchart illustrating Embodiment 4 of the method for determining the folding angle of a semi-trailer provided in this application;

[0112] Figure 5bA schematic diagram of the preset detection area provided in this application;

[0113] Figure 5c A schematic diagram of the steering vector and straight-line vector provided in this application;

[0114] Figure 5d A schematic diagram of the second preset area provided in this application;

[0115] Figure 6a A flowchart illustrating Embodiment 5 of the method for determining the folding angle of a semi-trailer provided in this application;

[0116] Figure 6b This is a diagram showing the segmentation provided by the entity itself.

[0117] Figure 7 A schematic diagram of the structure of an embodiment of the folding angle determination device for semi-trailers provided in this application;

[0118] Figure 8 This is a schematic diagram of the structure of an electronic device provided in this application.

[0119] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0120] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0121] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0122] A semi-trailer consists of a tractor and a trailer, which are hinged together by a pin. The tractor pulls the trailer. The angle formed by the tractor and the trailer is called the folding angle or articulation angle. This parameter is needed in technologies such as trajectory prediction and obstacle avoidance, so it is necessary to determine the folding angle of the semi-trailer.

[0123] For example, Figure 1 A schematic diagram of the folding angle provided in this application is shown below. Figure 1 As shown in the image, the black dot represents the connection point between the tractor and the trailer. This is the fold angle.

[0124] In existing technologies, the determination of the folding angle typically involves modifying the connection between the tractor and trailer to install an optical encoder or a magnet and Hall sensor. The data detected by the optical encoder or Hall sensor is then converted to obtain the folding angle. This modification to the semi-trailer results in high costs.

[0125] To address the problems existing in the prior art, the inventors, during their research on methods for determining the folding angle of semi-trailer trains, discovered that to reduce costs, instead of modifying the connection between the tractor and trailer, a camera could be installed, and the folding angle determined using images captured by the camera. At least one camera is mounted on the first part of the semi-trailer train, facing the second part, where the first part is the tractor and the second part is the trailer, or vice versa. When the vehicle is detected to be turning, the current frame image captured by each camera is acquired. Based on each current frame image, key point detection processing is performed to obtain the key point positions. Then, based on each key point position and the acquired straight-ahead reference point position, the target folding angle is calculated. Based on the above inventive concept, the folding angle determination scheme for semi-trailer trains in this application was designed.

[0126] The execution subject of the method for determining the folding angle of a semi-trailer in this application can be a controller in the semi-trailer, or it can be an on-board terminal, server, etc. This application does not limit it. The following explanation uses a controller as an example.

[0127] The following examples illustrate the application scenarios of the method for determining the folding angle of a semi-trailer provided in this application.

[0128] For example, in this application scenario, the semi-trailer can achieve trajectory prediction function to improve driving safety. At least one camera is installed on the first part of the semi-trailer and faces the second part, where the first part is the tractor and the second part is the trailer, or the first part is the trailer and the second part is the tractor.

[0129] When the controller in the semi-trailer detects that the vehicle is turning, it acquires the current frame image captured by each camera, indicating whether the turning state is a left turn or a right turn.

[0130] It should be noted that the vehicles mentioned in this application refer to semi-trailer trains.

[0131] Then, keypoint detection is performed on each current frame image to obtain the keypoint positions. Finally, based on each keypoint position and the obtained straight reference point position, the target folding angle is calculated.

[0132] After obtaining the target folding angle, the controller can predict the trajectory based on the target folding angle, and then display the predicted trajectory through the vehicle terminal for the driver to view.

[0133] It should be noted that the above scenario is only an example of an application scenario provided by the embodiments of this application. The embodiments of this application do not limit the actual form of the various devices included in the scenario, nor do they limit the interaction method between devices. In the specific application of the solution, it can be set according to actual needs.

[0134] The technical solution of this application will now be described in detail through specific embodiments. It should be noted that the following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0135] Figure 2 This is a flowchart illustrating an embodiment of the method for determining the folding angle of a semi-trailer provided in this application. This embodiment describes how the controller determines the folding angle based on images captured by a camera. The method in this embodiment can be implemented through software, hardware, or a combination of both. Figure 2 As shown, the method for determining the folding angle of this semi-trailer includes the following steps:

[0136] S201: When the vehicle is detected to be turning, acquire the current frame image captured by each camera.

[0137] To determine the folding angle using images, at least one camera needs to be installed on the semi-trailer. The camera is mounted on the first part of the semi-trailer and faces the second part, which can be either the tractor unit (first part) or the trailer unit (second part).

[0138] When the number of cameras is greater than one, all cameras are mounted on the sides of the first section, with at least one camera mounted on both the left and right sides of the first section. One or more cameras in the semi-trailer are mounted on the left side of the first section, and the remaining cameras are mounted on the right side of the first section.

[0139] For example, when the number of cameras is 2, one camera is mounted on the left side of the first part and the other camera is mounted on the right side of the first part.

[0140] When there is one camera, the first part is the trailer, and the camera is installed at the front of the trailer.

[0141] In this step, when the semi-trailer is traveling straight, the folding angle is 0, so there is no need to determine the folding angle. Therefore, the controller will determine the folding angle only when it detects that the vehicle is turning. First, it will acquire the current frame image captured by each camera, and the turning state will be either a left turn or a right turn.

[0142] S202: Perform key point detection processing on each current frame image to obtain the key point positions.

[0143] In this step, after the controller obtains the current frame image captured by each camera, in order to determine the folding angle, it needs to perform key point detection processing based on each current frame image to obtain the key point position.

[0144] When the number of cameras is greater than 1, only the left-hand camera can capture the second part when the semi-trailer turns left, and only the right-hand camera can capture the second part when the semi-trailer turns right. Only images of the second part can be used to determine the folding angle. Therefore, it is necessary to select the target image from all current frame images according to the turning state.

[0145] With two cameras, when turning left, the current frame image captured by the camera on the left side of the first section is used as the target image; when turning right, the current frame image captured by the camera on the right side of the first section is used as the target image.

[0146] When there are more than 2 cameras, when the turning state is left turn, any one of the current frame images captured by the camera on the left side of the first part will be used as the target image; when the turning state is right turn, any one of the current frame images captured by the camera on the right side of the first part will be used as the target image.

[0147] Alternatively, when the turning state is left turn, the current frame image captured by the camera on the left side of the first part is fused to obtain the target image; when the turning state is right turn, the current frame image captured by the camera on the right side of the first part is fused to obtain the target image.

[0148] Then, based on the target image, key point detection processing is performed to obtain the key point locations.

[0149] When there is only one camera, since there is only one current frame image, key point detection processing is performed based on the current frame image to obtain the key point positions.

[0150] S203: Calculate the target folding angle based on the location of each key point and the obtained straight reference point.

[0151] In this step, after the controller obtains the key point positions, it calculates the target folding angle by combining the obtained straight reference point positions.

[0152] Since the straight-ahead reference point is the location of key points in the image taken by the camera when the semi-trailer is traveling straight, the target folding angle can be calculated based on the location of the key points and the straight-ahead reference point.

[0153] When the number of key point locations is 1, the number of straight reference point locations is also 1. The steering vector and straight vector can be calculated based on the key point locations and the straight reference point locations, and then the target folding angle can be calculated.

[0154] When the number of keypoints is greater than one, straight lines can be fitted based on the keypoint positions and the straight reference point positions, and the target folding angle can be determined based on the angle between the two lines. Alternatively, the homography matrix can be calculated based on the keypoint positions and the straight reference point positions to determine the target folding angle.

[0155] It should be noted that when there are more than two cameras, in a left-turn situation, the current frame image captured by the camera on the left side of the first section can also be used as the target image; in a right-turn situation, the current frame image captured by the camera on the right side of the first section can be used as the target image. Each target image is then processed to obtain the key point positions, and the fold angle corresponding to each target image is calculated. The average value of the fold angles corresponding to each target image is taken as the target fold angle.

[0156] It should be noted that after the controller obtains the target folding angle, to improve accuracy, a Kalman filter can be applied to the target folding angle to obtain an updated target folding angle. The Kalman filter algorithm can be used for this process, and the parameters in the Kalman filter algorithm are updated each time the filtering is performed.

[0157] The method for determining the folding angle of a semi-trailer provided in this embodiment involves installing at least one camera on the first part of the semi-trailer, facing the second part, where the first part is the tractor and the second part is the trailer, or vice versa. When the vehicle is detected to be turning, the current frame image captured by each camera is acquired. Based on each current frame image, key point detection processing is performed to obtain the key point positions. Then, based on each key point position and the acquired straight-ahead reference point position, the target folding angle is calculated. This solution determines the folding angle using images captured by cameras, eliminating the need for modifications to the connection between the tractor and trailer, effectively reducing costs.

[0158] Figure 3a This is a flowchart illustrating a second embodiment of the method for determining the folding angle of a semi-trailer provided in this application. Based on the above embodiments, this application describes the situation where, when the number of cameras is two, the controller detects the key point positions of the vehicle body contour based on the target image, and then determines the target folding angle. Figure 3a As shown, the method for determining the folding angle of this semi-trailer includes the following steps:

[0159] S301: Perform distortion correction and viewpoint transformation on the target image to obtain the first bird's-eye view image.

[0160] When the first part is the tractor unit, the projection point of the connection between the tractor unit and the trailer on the right edge of the trailer is the first projection point, and the projection point of the connection on the left edge of the trailer is the second projection point. After distortion correction and bird's-eye view transformation processing of the image taken by the right-side camera of the tractor unit, the first projection point is on the right half of the processed image. After distortion correction and bird's-eye view transformation processing of the image taken by the left-side camera of the tractor unit, the second projection point is on the left half of the processed image. Furthermore, when the semi-trailer is traveling straight, after distortion correction and bird's-eye view transformation processing of the images taken by the left and right cameras of the tractor unit, the bottom edge of the trailer is parallel to the bottom edge of the image.

[0161] When the first part is a trailer, the projection point of the connection between the tractor and trailer on the right edge of the tractor is the first projection point, and the projection point of the connection on the left edge of the tractor is the second projection point. After distortion correction and bird's-eye view transformation processing of the image taken by the camera on the right side of the trailer, the first projection point is on the left half of the processed image. After distortion correction and bird's-eye view transformation processing of the image taken by the camera on the left side of the trailer, the second projection point is on the right half of the processed image. Furthermore, when the semi-trailer is traveling straight, after distortion correction and bird's-eye view transformation processing of the images taken by the cameras on the left and right sides of the trailer, the bottom edge of the tractor is parallel to the bottom edge of the image.

[0162] For example, Figure 3b The projection point diagram provided in this application is as follows: Figure 3b As shown, point A is the first projection point, and point B is the second projection point.

[0163] In this step, after the controller obtains the target image, it combines the camera's intrinsic and extrinsic parameters to perform distortion correction and viewpoint transformation on the target image to obtain the first bird's-eye view image.

[0164] S302: Determine the detection area in the first bird's-eye view image based on whether the folding angle, first part, turning state and first preset error angle have been determined in the previous frame image of the target image.

[0165] In this step, after the controller obtains the first bird's-eye view image, in order to improve the detection accuracy, it needs to determine the detection area in the first bird's-eye view image based on whether the folding angle, the first part, the turning state, and the first preset error angle have been determined in the previous frame of the target image.

[0166] Specifically, based on the first part and the turning state, the target side is determined, which is either the left or right side.

[0167] If the first part is a trailer and the turning state is left turn, then the target side is determined to be the right side.

[0168] If the first part is a trailer and the turning state is a right turn, then the target side is determined to be the left side.

[0169] If the first part is a tractor and the turning state is a left turn, then the target side is determined to be the left side.

[0170] If the first part is a tractor unit and the turning state is a right turn, then the target side is determined to be the right side.

[0171] After the controller obtains the target side, it determines whether the folding angle of the previous frame of the target image has been determined.

[0172] If the folding angle is not determined in the previous frame of the target image, then the target side half region in the first bird's-eye view image is used as the detection region.

[0173] It should be noted that the case where the folding angle of the previous frame of the target image is not determined includes the case where the target image does not have a previous frame.

[0174] If the folding angle has been determined in the previous frame of the target image, then a first angle and a second angle are generated based on the folding angle and a first preset error angle.

[0175] According to the formula Calculate the first angle, where, Showing the first angle, This indicates the folding angle that has been determined in the previous frame of the target image. This indicates the first preset error angle.

[0176] According to the formula Calculate the second angle, where, Indicates the second angle. This indicates the folding angle that has been determined in the previous frame of the target image. This indicates the first preset error angle.

[0177] It should be noted that the first error angle can be 15 degrees, 20 degrees, 25 degrees, etc. The embodiments of this application do not limit the first error angle, and it can be determined according to the actual situation.

[0178] Then, a detection region is generated based on the first angle, the second angle, and the midpoint of the bottom edge of the first bird's-eye view image.

[0179] A first ray is determined based on a first angle, and a second ray is determined based on a second angle. The vertices of the first and second rays are the midpoints of the bottom edge of the first bird's-eye view image. The angle between the first ray and the bottom edge of the first bird's-eye view image is the first angle, and the angle between the second ray and the bottom edge of the first bird's-eye view image is the second angle. When the target side is on the left, the first and second rays point to the left; when the target side is on the right, the first and second rays point to the right.

[0180] The region formed by the boundaries of the first ray, the second ray, and the first bird's-eye view image is used as the detection area.

[0181] For example, Figure 3c This is a schematic diagram of the detection area provided in this application, such as... Figure 3c As shown in the figure, the two curves are the first ray and the second ray, respectively, and the gray area is the detection area.

[0182] Because the camera captures images at a high frequency, the vehicle's position changes little between two frames. The detection area can be generated based on the folding angle determined in the previous frame of the target image, which can improve the accuracy of the detection area.

[0183] S303: The detection area is processed using a contour extraction algorithm to obtain an initial set of contour points.

[0184] In this step, after the controller obtains the detection area, in order to obtain the orientation of the second part, a contour extraction algorithm is used to process the detection area to obtain an initial set of contour points.

[0185] It should be noted that the contour extraction algorithm can be the Otsu algorithm, Snake algorithm, Suzuki85 algorithm, etc. This application does not limit the contour extraction algorithm, and it can be determined according to the actual situation.

[0186] S304: Filter all initial contour point sets according to the preset filtering strategy to obtain the target contour point set.

[0187] In this step, after the controller obtains the initial set of contour points, some of these points do not belong to the vehicle body contour and therefore need to be filtered. The initial set of contour points is filtered according to a preset filtering strategy to obtain the target set of contour points.

[0188] The default filtering strategy is to use the initial set of contour points that meet all filtering conditions as the target set of contour points.

[0189] For example, the filtering criteria could be: the perimeter of the smallest bounding rectangle of the initial contour point set falls within a preset perimeter range.

[0190] The filtering criteria can also be: the area of ​​the smallest bounding rectangle of the initial contour point set, which falls within the preset area range.

[0191] The filtering criteria can also be: the coordinates of the center point of the smallest bounding rectangle of the initial contour point set are within the preset coordinate range.

[0192] It should be noted that the preset perimeter range can be 500-800, 600-1000, 700-1200, etc., in pixels. The preset area range can be 1000-3000, 1200-3500, 1500-4000, etc., in squared pixels. The preset coordinate range is divided into a horizontal coordinate range and a vertical coordinate range, with the origin being the lower left corner of the first bird's-eye view image. When the target side is on the left, the horizontal coordinate range can be 100-400, 150-350, 200-300, etc., and the vertical coordinate range can be 100-500, 150-450, 180-400, etc. When the target side is on the right, the horizontal coordinate range can be 600-900, 650-850, 700-800, etc., and the vertical coordinate range can be 100-500, 150-450, 180-400, etc.

[0193] This application does not limit the screening conditions, preset perimeter range, preset area range, or preset coordinate range; these can be determined according to the actual situation.

[0194] It should be noted that if none of the initial contour sets are the target contour set, it is determined whether the fold angle of the previous frame of the target image has been determined. If the fold angle of the previous frame of the target image has not been determined, the determination of the target fold angle ends, and the target image has not yet determined its target fold angle. If the fold angle of the previous frame of the target image has been determined, then that fold angle is determined as the target fold angle.

[0195] If the target contour set exists in any of the initial contour sets, the consecutive cumulative count is updated to 0; if none of the initial contour sets are target contour sets, the consecutive cumulative count is incremented by one. The updated consecutive cumulative count is then checked against the preset alarm count. If the updated consecutive cumulative count equals the preset alarm count, an anomaly has occurred, and an alarm is triggered; if the updated consecutive cumulative count is less than the preset alarm count, no alarm is triggered.

[0196] The preset alarm count can be 10, 20, 50, etc. This application embodiment does not limit the preset alarm count, which can be determined according to the actual situation.

[0197] S305: Take the position of each point in the set of target contour points in the first bird's-eye view as the key point position.

[0198] In this step, after the controller determines the target contour point set, the position of each point in each target contour point set in the first bird's-eye view image is used as the key point position.

[0199] S306: Calculate the target folding angle based on the location of each key point and the obtained straight reference point.

[0200] In this step, after the controller determines the location of the key points, it calculates the target folding angle based on the location of each key point and the obtained straight reference point location.

[0201] Specifically, based on the location of each key point, a turning line corresponding to each target contour point set is generated. That is, for the position of each point in the target contour point set, a straight line is fitted to obtain the turning line.

[0202] In one implementation, the straight line containing the longest side of the smallest bounding rectangle of the target contour point set can be used as the turning line.

[0203] Based on the position of the straight reference point, a straight line is fitted and generated.

[0204] It should be noted that the vertical coordinates of the straight reference points are all 0, meaning that the straight line coincides with the bottom edge of the first bird's-eye view image.

[0205] Multiple initial folding angles are generated based on each turning line and straight line. That is, the angle between each turning line and straight line is used as the initial folding angle.

[0206] For example, Figure 3d A schematic diagram of the initial folding angle provided in this application is shown below. Figure 3d As shown in the figure, the points are from the target contour point set, and the turning line is obtained by fitting. The upper dashed line is the turning line, and the lower dashed line is the straight line. The angle between the two dashed lines is the initial folding angle.

[0207] Then, based on whether the folding angle has been determined in the previous frame of the target image, and each initial folding angle, the target folding angle is generated.

[0208] If the folding angle of the previous frame of the target image is not determined, then the average of all initial folding angles is taken as the target folding angle.

[0209] It should be noted that the case where the folding angle of the previous frame of the target image is not determined includes the case where the target image does not have a previous frame.

[0210] If the folding angle of the previous frame of the target image has been determined, an error angle range is generated based on the folding angle and the second preset error angle. The minimum value of the error angle range is the difference between the folding angle and the second preset error angle, and the maximum value of the error angle range is the sum of the folding angle and the second preset error angle.

[0211] It should be noted that the second preset error angle can be 3 degrees, 4 degrees, 5 degrees, etc. This application embodiment does not limit the second preset error angle, and it can be determined according to the actual situation.

[0212] The target folding angle is generated based on the error angle range and each initial folding angle.

[0213] In other words, the folding angle determined in the previous frame of the target image is used as the reference folding angle. The folding angles that fall within the error angle range in the initial folding angles are used as the first undetermined folding angles.

[0214] If the number of first undetermined folding angles is less than or equal to a preset number, then the weight of the first undetermined folding angle is set according to the absolute value of the difference between the first undetermined folding angle and the reference folding angle. The smaller the absolute value of the difference, the greater the weight. The sum of the weights of all the first undetermined folding angles is 1. Then, the first undetermined folding angles are weighted and summed to obtain the target folding angle.

[0215] If the number of first undetermined folding angles is greater than a preset number, select the folding angle with the smallest absolute value of the difference between the preset number of first undetermined folding angles and the reference folding angle as the second undetermined folding angle. Based on the absolute value of the difference between the second undetermined folding angle and the reference folding angle, assign a weight to the second undetermined folding angle; the smaller the absolute value of the difference, the greater the weight. The sum of the weights of all second undetermined folding angles is 1. Then, perform a weighted summation on the second undetermined folding angles to obtain the target folding angle.

[0216] It should be noted that the preset quantity can be 3, 4, 5, etc. This application embodiment does not limit the preset quantity, and it can be determined according to the actual situation.

[0217] It should be noted that if all initial fold angles are outside the error angle range, it is determined whether the fold angle of the previous frame of the target image has been determined. If the fold angle of the previous frame of the target image has not been determined, the determination of the target fold angle ends, and the target image has not yet determined its target fold angle. If the fold angle of the previous frame of the target image has been determined, then that fold angle is determined as the target fold angle.

[0218] Because the camera captures images at a high frequency, the vehicle's position changes little between two frames. When the folding angle has been determined in the previous frame of the target image, the target folding angle can be determined based on the error angle range generated by that folding angle, which can improve the accuracy of the target folding angle.

[0219] It should be noted that the controller detects whether the vehicle is turning in the following way: if the left turn signal is detected, it determines that the vehicle is turning left; if the right turn signal is detected, it determines that the vehicle is turning right.

[0220] The method for determining the folding angle of a semi-trailer provided in this embodiment converts the target image into a first bird's-eye view image, determines the detection area, extracts the set of contour points, and obtains the key point positions. Then, based on the key point positions and the straight-line reference point positions, straight lines are generated to determine the initial folding angle. Finally, based on whether the folding angle has been determined in the previous frame of the target image, the target folding angle is calculated, thus improving the accuracy of the target folding angle.

[0221] Figure 4a This is a flowchart illustrating a third embodiment of the method for determining the folding angle of a semi-trailer provided in this application. Based on the above embodiments, this application describes the situation where, when the number of cameras is two, the controller detects the key point positions on the edge of the vehicle body based on the target image, and then determines the target folding angle. Figure 4a As shown, the method for determining the folding angle of this semi-trailer includes the following steps:

[0222] S401: The vehicle body edge key point detection model is used to detect the target image and obtain the initial edge points.

[0223] When the first part is the tractor unit, and the semi-trailer is traveling straight, after distortion removal and bird's-eye view conversion processing of the images taken by the left and right cameras of the tractor unit, the bottom edge of the trailer is parallel to the bottom edge of the image.

[0224] When the first part is a trailer, and the semi-trailer is traveling straight, after distortion removal and bird's-eye view conversion processing of the images taken by the cameras on the left and right sides of the trailer, the bottom edge of the tractor is parallel to the bottom edge of the image.

[0225] In this step, after the controller obtains the target image, it uses the vehicle body edge key point detection model to detect the target image and obtain the initial edge points.

[0226] Among them, the vehicle body edge key point detection model is a pre-trained deep learning model used to obtain edges from images. The vehicle body edge key point detection model includes a backbone network, a neck network, a curve feature extraction layer, a feature fusion layer, and a key point regression head.

[0227] S402: Perform distortion correction and viewpoint transformation on the target image to obtain the second bird's-eye view image.

[0228] In this step, after the controller determines that the target image has been obtained, in order to calculate the target folding angle, the target image also needs to be distorted and the viewpoint transformed to obtain the second bird's-eye view image. There is a correspondence between the points in the target image and the second bird's-eye view image.

[0229] S403: Use the position of the point corresponding to each initial edge point in the second bird's-eye view as the key point position.

[0230] In this step, after the controller obtains the second bird's-eye view and the initial edge points, it uses the position of the point corresponding to each initial edge point in the second bird's-eye view as the key point position.

[0231] S404: Calculate the target folding angle based on the location of each key point and the obtained straight reference point.

[0232] In this step, after the controller determines the location of the key points, it calculates the target folding angle based on the location of each key point and the obtained straight reference point location.

[0233] Specifically, a steering line is generated by fitting based on the locations of all key points.

[0234] Generate a straight line based on the position of the straight reference point.

[0235] It should be noted that the vertical coordinates of the straight reference points are all 0, meaning that the straight line coincides with the bottom edge of the first bird's-eye view image.

[0236] The target folding angle is determined based on the turning line and the straight line, which is the angle between the turning line and the straight line.

[0237] It should be noted that if no initial edge point is detected, the process checks whether the fold angle of the previous frame of the target image has been determined. If the fold angle of the previous frame of the target image has not been determined, the determination of the target fold angle ends, and the target image has not yet determined its target fold angle. If the fold angle of the previous frame of the target image has been determined, then that fold angle is determined as the target fold angle.

[0238] It should be noted that when an initial edge point is detected, the cumulative count is updated to 0; when no initial edge point is detected, the cumulative count is incremented by one. The updated cumulative count is then checked against the preset alarm count. If the updated cumulative count equals the preset alarm count, an anomaly has occurred, an alarm is triggered, and Kalman filtering is no longer performed. If the updated cumulative count is less than the preset alarm count, no alarm is triggered.

[0239] It should be noted that the controller detects whether the vehicle is turning in the following way: if the left turn signal is detected, it determines that the vehicle is turning left; if the right turn signal is detected, it determines that the vehicle is turning right.

[0240] For example, Figure 4b A schematic diagram illustrating the generation process of the target fold angle provided in this application is shown below. Figure 4b As shown, the target image is detected to obtain initial edge points, which are then used to determine the position in the second bird's-eye view. A turning line is obtained by fitting the data. The angle between the turning line and the straight line is the target folding angle.

[0241] The following section introduces the key point detection model for vehicle body edges.

[0242] For example, Figure 4c A schematic diagram of the architecture of the vehicle body edge key point detection model provided in this application is shown below. Figure 4c As shown, the vehicle body edge key point detection model includes a backbone network, a neck network, a curve feature extraction layer, a feature fusion layer, and a key point regression head.

[0243] The target image is input into the backbone network for feature extraction, outputting four first feature maps of different sizes, which are 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the target image size, respectively. These four first feature maps are then input into the neck network, outputting four second feature maps of different sizes.

[0244] The vehicle body edge keypoint detection model has pre-trained convolutional kernels and weights in eight directions. The curve feature extraction layer uses a second feature map, representing one-quarter of the target image size, as the target feature map. For each direction, the convolutional kernel in that direction is convolved with the target feature map, and then multiplied by the weights for that direction to obtain a sub-direction feature map. The sub-direction feature maps from the eight directions are then summed to obtain the directional feature map.

[0245] The feature fusion layer convolves the orientation feature map using a 1x1 convolution kernel, and then generates an attention weight map through softmax. The attention weight map is then multiplied element-wise with a second feature map that is 1 / 4 the size of the target image to obtain the curve enhancement feature map.

[0246] The keypoint regression head processes the curve enhancement feature map to obtain the initial edge points.

[0247] By adding a curve feature extraction layer and a feature fusion layer, the model's edge points can be made to have the characteristics of being on the same curve, thus improving the extraction accuracy.

[0248] It should be noted that the vehicle edge key point detection model can also be enhanced through training. By randomly increasing the brightness, adding shadows, and changing the colors of the training images, the model can be enhanced through training with the processed images, thereby improving its generalization ability.

[0249] The method for determining the folding angle of a semi-trailer provided in this embodiment first detects the target image to obtain initial edge points, then converts it to a bird's-eye view image and fits a turning line. After fitting the straight line based on the position of the straight reference point, the angle between the turning line and the straight line is the target folding angle, thus improving the accuracy of the target folding angle.

[0250] Figure 5a This is a flowchart illustrating Embodiment 4 of the method for determining the folding angle of a semi-trailer provided in this application. Based on the above embodiments, this application embodiment describes the situation where, when the number of cameras is 2, the controller detects the key point positions of the rearview mirror based on the target image, and then determines the target folding angle. Figure 5a As shown, the method for determining the folding angle of this semi-trailer includes the following steps:

[0251] S501: Within the preset detection area of ​​the target image, perform matching processing based on the preset rearview mirror image to obtain the rearview mirror area.

[0252] In this embodiment, the first part is a trailer. When the semi-trailer is traveling straight, the images captured by the cameras on the left and right sides of the trailer include the rearview mirror.

[0253] In this step, after the controller obtains the target image, it performs matching processing on the preset rearview mirror image within the preset detection area of ​​the target image to obtain the rearview mirror area.

[0254] It should be noted that the preset detection area is the region in the target image where the rearview mirror exists. For example, Figure 5b A schematic diagram of the preset detection area provided in this application, such as Figure 5b As shown in the figure, the dashed box represents the preset detection area, and the trapezoid represents the rearview mirrors at various turning angles. Regardless of the angle at which the semi-trailer turns, the rearview mirrors are always within the predicted detection area.

[0255] Both the preset rearview mirror image and the preset detection area are rectangles. The preset rearview mirror image is smaller. The preset rearview mirror image is moved and traversed within the preset detection area. Each time it is moved, a matching is performed. If the matching is successful, the area where the preset rearview mirror image is currently located is taken as the rearview mirror area.

[0256] The matching process is as follows: Calculate the first gradient of each pixel in the preset rearview mirror image, and calculate the second gradient of each pixel within the current region of the preset rearview mirror image. For each identical position in the preset rearview mirror image and its current region, take the absolute value of the difference between the first gradient and the second gradient of the pixel at that position as the gradient difference for that position. Determine whether the sum of the gradient differences at all positions is less than a preset threshold. If the sum of the gradient differences at all positions is less than the preset threshold, the match is considered successful; if the sum of the gradient differences at all positions is greater than or equal to the preset threshold, the match is considered unsuccessful.

[0257] It should be noted that the preset threshold can be 4000, 10000, 15000, etc. This application embodiment does not limit the preset threshold, and it can be determined according to the actual situation.

[0258] It should be noted that when there are multiple preset rearview mirror images, if multiple rearview mirror areas are determined, the intersection of these rearview mirror areas will be used as the new rearview mirror area.

[0259] S502: Use the position of the center point of the rearview mirror area as the key point position.

[0260] In this step, after the controller determines the rearview mirror area, it uses the position of the center point of the rearview mirror area as the key point position.

[0261] S503: Calculate the target folding angle based on the location of each key point and the obtained straight reference point.

[0262] In this step, after the controller determines the location of the key points, it calculates the target folding angle based on the location of each key point and the obtained straight reference point location.

[0263] The straight-ahead reference points include the left-side straight-ahead point and the right-side straight-ahead point. The left-side straight-ahead point is the center of the rearview mirror in an image taken from the left side of the trailer when the semi-trailer is traveling straight. The right-side straight-ahead point is the center of the rearview mirror in an image taken from the right side of the trailer when the semi-trailer is traveling straight.

[0264] Specifically, depending on the turning state, the target straight-ahead point is determined from the left-hand straight-ahead point position and the right-hand straight-ahead point position. If the turning state is left-turn, the target straight-ahead point position is the left-hand straight-ahead point position; if the turning state is right-turn, the target straight-ahead point position is the right-hand straight-ahead point position.

[0265] Calculate the turning vector based on the key point locations and preset camera intrinsic parameters.

[0266] Calculate the straight-line vector based on the target straight-line point position and preset camera intrinsic parameters.

[0267] Calculate the target folding angle based on the steering vector and the straight-line vector.

[0268] It should be noted that both the turning vector and the straight vector are unit vectors in a two-dimensional coordinate system. The origin of the two-dimensional coordinate system is the optical center of the camera, the horizontal axis is the direction of the optical axis, and the vertical axis is the direction of vertical upward.

[0269] For example, Figure 5c A schematic diagram of the steering vector and straight-line vector provided in this application is shown below. Figure 5c As shown in the figure, the upper dashed line is the turning vector, and the upper dashed line is the straight vector.

[0270] According to the formula Calculate the target folding angle, where, Indicates the target fold angle. Represents the steering vector. This represents a linear vector.

[0271] The following explains how to determine the location of the straight reference point.

[0272] Acquire the left-side image and the right-side image of the semi-trailer traveling straight. The left-side image is taken by the camera on the left side of the trailer when the semi-trailer is traveling straight; the right-side image is taken by the camera on the right side of the trailer when the semi-trailer is traveling straight.

[0273] Then, the left and right images of the vertical line are taken as the target vertical images, and the following processing is performed:

[0274] The target region is determined from the first preset region and the second preset region based on whether the target straight image is a straight left image.

[0275] The first preset region is the area where the rearview mirror is located in the left-hand image of a straight line, and the second preset region is the area where the rearview mirror is located in the right-hand image of a straight line. If the target straight image is the left-hand image of a straight line, the target region is the first preset region. If the target straight image is the right-hand image of a straight line, the target region is the second preset region.

[0276] For example, Figure 5d A schematic diagram of the second preset area provided in this application, such as Figure 5d As shown in the figure, the dashed box represents the second preset area, and the trapezoid represents the rearview mirror, which is located in the second preset area.

[0277] Contour extraction is performed on the straight-line image of the target to obtain the vehicle body contour points.

[0278] Based on the target area and vehicle body contour points, the first rearview mirror contour points are determined. This involves identifying the vehicle body contour points within the target area and calculating the gradient of each point. Since the rearview mirror protrudes and its color differs from surrounding pixels, the gradients of the mirror's contour points are relatively large. Therefore, points with gradients greater than a preset gradient are used as the rearview mirror contour generation points. Based on these contour generation points, a minimum bounding rectangle is determined, and the points within this minimum bounding rectangle are then used as the first rearview mirror contour points.

[0279] It should be noted that the preset gradient can be 100, 200, 300, etc. This application embodiment does not limit the preset gradient, and it can be determined according to the actual situation.

[0280] Within the target area of ​​the straight-ahead image, a matching process is performed based on the preset rearview mirror image to determine the second rearview mirror contour point. That is, the rearview mirror area is obtained using the method in step S501, and then the boundary of the rearview mirror area is used as the second rearview mirror contour point.

[0281] Based on the contour points of the first and second rearview mirrors, Kalman filtering is performed to obtain the target rearview mirror contour points. This improves the accuracy of the subsequently obtained straight-ahead reference point position.

[0282] If the target straight-ahead image is the left side image, then the center point of the target rearview mirror outline is taken as the left straight-ahead point position. If the target straight-ahead image is the right side image, then the center point of the target rearview mirror outline is taken as the right straight-ahead point position.

[0283] It should be noted that the controller detects whether the vehicle is turning in the following way: if the left turn signal is detected, it determines that the vehicle is turning left; if the right turn signal is detected, it determines that the vehicle is turning right.

[0284] The method for determining the folding angle of a semi-trailer provided in this embodiment improves the accuracy of the target folding angle by using a rearview mirror to determine the target folding angle.

[0285] Figure 6a This is a flowchart illustrating Embodiment 5 of the method for determining the folding angle of a semi-trailer provided in this application. Based on the above embodiments, this embodiment describes a scenario where, with one camera, the first part being the trailer, and the camera installed in front of the trailer, the controller detects feature points of the tractor unit based on the current frame image, uses the location of these feature points as key point locations, and then determines the target folding angle. Figure 6a As shown, the method for determining the folding angle of this semi-trailer includes the following steps:

[0286] S601: Perform foreground segmentation processing on the current frame image to obtain the first tractor image.

[0287] In this step, since the camera is installed in front of the trailer, it can capture images of the back of the tractor. After the controller obtains the current frame image, it first performs foreground segmentation processing on the current frame image in order to calculate the folding angle and obtain the first tractor image.

[0288] A preset segmentation model can be used to segment the current frame image to obtain the first tractor image. The preset segmentation model can be the U-net segmentation model, the FUTURIST model, the UniPixel model, etc. This application embodiment does not limit the preset segmentation model, and it can be determined according to the actual situation.

[0289] For example, Figure 6b The segmentation diagram provided by itself, such as Figure 6b As shown in the figure, the gray area represents the tractor unit. After foreground segmentation, the image of the tractor unit is obtained.

[0290] S602: Perform feature point extraction processing on the image of the first tractor to obtain multiple feature points and the feature vector of each feature point.

[0291] In this step, after the controller obtains the image of the first tractor, it performs feature point extraction processing on the image of the first tractor to obtain multiple feature points and feature vectors for each feature point. The feature vectors include neighborhood gradients.

[0292] Feature points can be extracted using a preset feature extraction algorithm. The preset feature extraction algorithm can be ORB (Oriented FAST and Rotated BRIEF), SIFT (Scale-Invariant Feature Transform), SURF (Speeded-Up Robust Features), etc. This application embodiment does not limit the preset feature extraction algorithm, and it can be determined according to the actual situation.

[0293] S603: For each feature point, take the position of the feature point in the first tractor image as the key point position, and take the feature vector of the feature point as the feature vector of the key point position.

[0294] In this step, after the controller obtains the feature points and their feature vectors, for each feature point, the position of the feature point in the first tractor image is taken as the key point position, and the feature vector of the feature point is taken as the feature vector of the key point position.

[0295] S604: Calculate the target folding angle based on the location of each key point and the obtained straight reference point.

[0296] In this step, after the controller determines the location of the key points, it calculates the target folding angle based on the location of each key point and the obtained straight reference point location.

[0297] Specifically, based on the position of each straight reference point, the feature vector of each straight reference point position is obtained.

[0298] The position of the straight reference point and the feature vector are obtained by performing foreground segmentation and feature extraction on images taken by the camera when the semi-trailer is traveling straight, resulting in the position and feature vector of the feature points.

[0299] Based on the feature vectors of each key point and the feature vectors of each straight reference point, FLANN matching is performed to obtain initial position pairs.

[0300] Based on the key point position and straight reference point position in each initial position pair, as well as the neighborhood gradient in the feature vectors of the key point position and straight reference point position, a filtering process is performed to obtain the target position pair.

[0301] The selection process is as follows: For each initial position pair, calculate the absolute value of the difference between the L1 norms of the neighborhood gradients of the keypoint position and the straight-line reference point position in the initial position pair, obtaining the gradient difference of the initial position pair. The smaller the gradient difference, the closer the pixels corresponding to the keypoint position and the straight-line reference point position are, and the higher the matching degree between the keypoint position and the straight-line reference point position. Then, sort the initial position pairs according to the gradient difference from smallest to largest, and select the first preset number of initial position pairs to obtain the first position pair. Then, use the RANSAC algorithm to select the first position pairs, removing mismatched position pairs to obtain the second position pairs. Establish a polar coordinate system in the first tractor image. For each second position pair, determine the absolute value of the angle difference between the keypoint position and the straight-line reference point position in the second position pair, obtaining the angle difference of the second position pair. For each second position pair, if the angle difference of the second position pair is less than a preset angle threshold, then the second position pair is taken as the target position pair.

[0302] It should be noted that the preset number of filters can be 10, 20, 50, etc., and the preset angle threshold can be 10 degrees, 15 degrees, 20 degrees, etc. This application embodiment does not limit the preset number of filters and the preset angle threshold, which can be determined according to the actual situation.

[0303] Generate a homography matrix based on all target position pairs.

[0304] The target folding angle is determined based on the homography matrix. The homography matrix is ​​decomposed into Euler angles, and the angle of rotation about the vertically upward axis in the Euler angles is the target folding angle.

[0305] It should be noted that after the controller obtains the target folding angle, if the folding angle of the previous frame of the current frame has been determined, it calculates the absolute value of the difference between that folding angle and the target folding angle to obtain the update parameter. If the update parameter is greater than the preset update threshold, the folding angle determined in the previous frame of the current frame is taken as the new target folding angle.

[0306] The following explains the detection process for whether a vehicle is turning.

[0307] Acquire the initial image captured by the camera in real time.

[0308] The initial image is segmented to obtain the second tractor image.

[0309] The image of the second tractor is processed by contour extraction to obtain the initial contour.

[0310] The contour with the largest area among all initial contours is taken as the target contour; the area of ​​the contour is the area of ​​the smallest bounding rectangle of the contour, and the contour with the largest area represents the tractor.

[0311] The position of the center point of the outer rectangle of the target contour in the initial image is used as the turning state detection position.

[0312] Based on the turning detection location and the preset straight-ahead area, it is determined whether the vehicle is in a turning state. If the turning detection location is within the preset straight-ahead area, the vehicle is determined to be in a straight-ahead state. If the turning detection location is not within the preset straight-ahead area, the vehicle is determined to be in a turning state; furthermore, if the turning detection location is within the preset left-turn area, the vehicle is determined to be in a left-turn state; if the turning detection location is within the preset right-turn area, the vehicle is determined to be in a right-turn state.

[0313] In the image, the preset left turn area is located to the left of the preset straight area, and the preset right turn area is located to the right of the preset straight area.

[0314] In one implementation, if N frames of images exist before the initial image, and it is determined that the vehicle is in a straight-ahead state based on both the initial image and all N frames of images, then the vehicle is determined to be in a straight-ahead state. If it is determined that the vehicle is in a turning state based on the initial image, or if it is determined that the vehicle is in a turning state based on at least one of the N frames of images, then the vehicle is determined to be in a turning state.

[0315] It should be noted that N can be 2, 3, 5, etc. The embodiments of this application do not limit the value of N, and it can be determined according to the actual situation.

[0316] It should be noted that when it is determined that the vehicle is in a straight-ahead state, the initial image can be processed by foreground segmentation and feature point extraction to obtain the new straight-ahead reference point position and feature vector.

[0317] The method for determining the folding angle of a semi-trailer provided in this embodiment improves the accuracy of the target folding angle by matching feature points in the image of the semi-trailer turning with feature points in the image of it traveling straight, thereby generating a homography matrix.

[0318] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0319] Figure 7 This is a structural schematic diagram of an embodiment of the semi-trailer folding angle determining device provided in this application. Figure 7 As shown, the folding angle determining device 70 of the semi-trailer includes:

[0320] The acquisition module 71 is used to acquire the current frame image captured by each camera when the vehicle is detected to be turning, and the turning state is either a left turn or a right turn.

[0321] The processing module 72 is used to perform key point detection processing based on each current frame image to obtain the key point positions;

[0322] Angle determination module 73 is used to calculate the target folding angle based on the position of each key point and the obtained straight reference point position.

[0323] Furthermore, if the number of cameras is greater than one, at least one camera is mounted on the side of the first part, and the processing module 72 is specifically used for:

[0324] Select the target image from all current frame images based on the turning status;

[0325] Based on the target image, key point detection processing is performed to obtain the key point locations.

[0326] Furthermore, at least one or more cameras are mounted on the left side of the first part, and the remaining cameras are mounted on the side of the first part.

[0327] Furthermore, the processing module 72 is specifically used for:

[0328] The target image is subjected to distortion correction and viewpoint transformation processing to obtain the first bird's-eye view image;

[0329] Based on whether the folding angle, first part, turning state and first preset error angle have been determined in the previous frame of the target image, the detection area in the first bird's-eye view image is determined;

[0330] A contour extraction algorithm is used to process the detection area to obtain an initial set of contour points;

[0331] The target contour point set is obtained by filtering all initial contour point sets according to the preset filtering strategy.

[0332] The position of each point in each target contour point set in the first bird's-eye view image is used as the key point position.

[0333] Furthermore, the processing module 72 is specifically used for:

[0334] Based on the first part and the turning status, determine the target side, which is either the left or right side;

[0335] If the folding angle is not determined in the previous frame of the target image, then the target side half region in the first bird's-eye view image is used as the detection region.

[0336] If the folding angle of the previous frame of the target image has been determined, then the first angle and the second angle are generated based on the folding angle and the first preset error angle.

[0337] The detection region is generated based on the first angle, the second angle, and the midpoint of the bottom edge of the first bird's-eye view image.

[0338] Furthermore, the processing module 72 is specifically used for:

[0339] If the first part is a trailer and the turning state is left turn, then the target side is determined to be the right side;

[0340] If the first part is a trailer and the turning state is a right turn, then the target side is determined to be the left side;

[0341] If the first part is a tractor and the turning state is a left turn, then the target side is determined to be the left side;

[0342] If the first part is a tractor unit and the turning state is a right turn, then the target side is determined to be the right side.

[0343] Furthermore, the angle determination module 73 is specifically used for:

[0344] Based on the location of each key point, generate the turning line corresponding to each target contour point set;

[0345] Generate a straight line based on the position of the straight reference point;

[0346] Multiple initial folding angles are generated based on each turning line and straight line;

[0347] The target fold angle is generated based on whether the fold angle has been determined in the previous frame of the target image, and each initial fold angle.

[0348] Furthermore, the angle determination module 73 is specifically used for:

[0349] If the folding angle of the previous frame of the target image is not determined, then the average of all initial folding angles is taken as the target folding angle;

[0350] If the folding angle of the previous frame of the target image has been determined, then an error angle range is generated based on the folding angle and the second preset error angle.

[0351] The target folding angle is generated based on the error angle range and each initial folding angle.

[0352] Furthermore, the processing module 72 is specifically used for:

[0353] A vehicle body edge key point detection model is used to detect the target image and obtain initial edge points. The vehicle body edge key point detection model is a pre-trained deep learning model used to obtain edges from the image. The vehicle body edge key point detection model includes a backbone network, a neck network, a curve feature extraction layer, a feature fusion layer, and a key point regression head.

[0354] The target image is subjected to distortion correction and viewpoint transformation to obtain a second bird's-eye view image. There is a correspondence between the points in the target image and the second bird's-eye view image.

[0355] The position of each point corresponding to the initial edge point in the second bird's-eye view is taken as the key point position.

[0356] Furthermore, the angle determination module 73 is specifically used for:

[0357] Generate a turning line based on the locations of all key points;

[0358] Generate a straight line based on the position of the straight reference point;

[0359] Determine the target folding angle based on the turning line and the straight line.

[0360] Furthermore, the first part is the trailer, and the processing module 72 is specifically used for:

[0361] Within a preset detection area of ​​the target image, a matching process is performed based on a preset rearview mirror image to obtain the rearview mirror area;

[0362] The center point of the rearview mirror area is used as the key point.

[0363] Furthermore, the straight-ahead reference point positions include the left straight-ahead point position and the right straight-ahead point position. The angle determination module 73 is specifically used for:

[0364] Based on the turning status, determine the target straight-ahead position from the left straight-ahead position and the right straight-ahead position;

[0365] Calculate the turning vector based on the key point locations and preset camera intrinsic parameters;

[0366] Calculate the straight-line vector based on the target straight-line point position and preset camera intrinsic parameters;

[0367] Calculate the target folding angle based on the steering vector and the straight-line vector.

[0368] Furthermore, when the vehicle is detected to be turning, before acquiring the current frame image captured by each camera, the acquisition module 71 is also used to acquire the left-hand image and the right-hand image of the vehicle going straight.

[0369] Processing module 72 is further configured to take the left and right images of the vertical line as target vertical images respectively, and perform the following processing:

[0370] Based on whether the target straight image is a straight left image, the target region is determined from the first preset region and the second preset region;

[0371] Contour extraction is performed on the straight-line image of the target to obtain the vehicle body contour points;

[0372] Determine the first rearview mirror outline point based on the target area and the vehicle body outline points;

[0373] Within the target area of ​​the target straight image, the second rearview mirror contour point is determined by matching the preset rearview mirror image.

[0374] Based on the contour points of the first and second rearview mirrors, Kalman filtering is performed to obtain the target rearview mirror contour points.

[0375] If the target straight image is the left straight image, then the position of the center point of the target rearview mirror outline point is taken as the position of the left straight point;

[0376] If the target straight image is the image on the right side of the straight line, then the position of the center point of the target rearview mirror outline point is taken as the position of the right straight point.

[0377] Furthermore, the processing module 72 is also used for:

[0378] If the left turn signal is detected to be on, it is determined that the vehicle is in a left turn state.

[0379] If the right turn signal is detected to be on, it indicates that the vehicle is in a right turn state.

[0380] Furthermore, if the number of cameras is 1, the first part is a trailer, the camera is installed at the front of the trailer, and the processing module 72 is specifically used for:

[0381] Perform foreground segmentation on the current frame image to obtain the first tractor image;

[0382] Feature point extraction processing is performed on the image of the first tractor to obtain multiple feature points and feature vectors of each feature point. The feature vectors include neighborhood gradients.

[0383] For each feature point, the position of the feature point in the first tractor image is taken as the key point position, and the feature vector of the feature point is taken as the feature vector of the key point position.

[0384] Furthermore, the angle determination module 73 is specifically used for:

[0385] Based on the position of each straight reference point, obtain the feature vector of each straight reference point position;

[0386] Based on the feature vectors of each key point and the feature vectors of each straight reference point, a matching process is performed to obtain initial position pairs;

[0387] Based on the key point position and straight reference point position in each initial position pair, as well as the neighborhood gradient in the feature vector of the key point position and straight reference point position, a filtering process is performed to obtain the target position pair;

[0388] Generate a homography matrix based on all target location pairs;

[0389] The target folding angle is determined based on the homography matrix.

[0390] Furthermore, the acquisition module 71 is also used to acquire the initial image captured by the camera in real time;

[0391] Processing module 72 is also used for:

[0392] The initial image is segmented to obtain the second tractor image;

[0393] The image of the second tractor is processed by contour extraction to obtain the initial contour;

[0394] The contour with the largest area among all initial contours is taken as the target contour;

[0395] The position of the center point of the bounding rectangle of the target contour in the initial image is used as the turning state detection position.

[0396] Based on the turning detection location and the preset straight-ahead area, determine whether the vehicle is in a turning state.

[0397] Furthermore, the processing module 72 is also used for:

[0398] The target fold angle is processed by Kalman filtering to obtain the updated target fold angle.

[0399] The folding angle determination device for semi-trailers provided in this embodiment is used to execute the technical solution in any of the aforementioned method embodiments. Its implementation principle and technical effect are similar, and will not be described again here.

[0400] Figure 8 This is a schematic diagram of the structure of an electronic device provided in this application. Figure 8 As shown, the electronic device 80 includes:

[0401] Processor 81, memory 82, and communication interface 83;

[0402] Memory 82 is used to store executable instructions of processor 81;

[0403] The processor 81 is configured to execute the technical solutions in any of the foregoing method embodiments by executing executable instructions.

[0404] Optionally, the memory 82 can be either standalone or integrated with the processor 81.

[0405] Optionally, when the memory 82 is a device independent of the processor 81, the electronic device 80 may further include:

[0406] Bus 84, memory 82 and communication interface 83 are connected to processor 81 through bus 84 and complete communication with each other. Communication interface 83 is used to communicate with other devices.

[0407] Optionally, the communication interface 83 can be implemented using a transceiver. The communication interface is used to enable communication between the database access device and other devices (e.g., clients, read-write databases, and read-only databases). The memory may include random access memory (RAM) and may also include non-volatile memory, such as at least one disk drive.

[0408] Bus 84 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, only one thick line is used in the diagram, but this does not indicate that there is only one bus or one type of bus.

[0409] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0410] The electronic device is used to execute the technical solutions in any of the foregoing method embodiments. Its implementation principle and technical effect are similar, and will not be described again here.

[0411] This application also provides a semi-trailer train, which includes at least one camera and a controller.

[0412] The at least one camera is mounted on the first part of the semi-trailer and faces the second part, the first part being the tractor and the second part being the trailer, or the first part being the trailer and the second part being the tractor.

[0413] The controller is used to execute the technical solutions in any of the foregoing method embodiments. Its implementation principle and technical effect are similar, and will not be repeated here.

[0414] This application also provides a readable storage medium storing a computer program thereon, which, when executed by a processor, implements the technical solutions provided in any of the foregoing method embodiments.

[0415] This application also provides a computer program product, including a computer program, which, when executed by a processor, is used to implement the technical solutions provided in any of the foregoing method embodiments.

[0416] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0417] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A method for determining the folding angle of a semi-trailer train, characterized in that, At least one camera is mounted on a first part of a semi-trailer and faces a second part, wherein the first part is a tractor and the second part is a trailer, or the first part is a trailer and the second part is a tractor, the method comprising: When the vehicle is detected to be turning, the current frame image captured by each camera is acquired, wherein the turning state is either a left turn or a right turn. Based on each current frame image, key point detection processing is performed to obtain the key point locations; Calculate the target folding angle based on the location of each key point and the obtained straight reference point.

2. The method according to claim 1, characterized in that, If the number of cameras is greater than one, the at least one camera is mounted on the side of the first part, and the key point detection processing based on each current frame image to obtain the key point position includes: Based on the turning state, select the target image from all current frame images; Based on the target image, key point detection processing is performed to obtain the key point locations.

3. The method according to claim 2, characterized in that, One or more of the at least one camera are mounted on the left side of the first part, and the remaining camera is mounted on the right side of the first part.

4. The method according to claim 3, characterized in that, The step of performing key point detection processing based on the target image to obtain the key point locations includes: The target image is subjected to distortion correction and viewpoint transformation processing to obtain a first bird's-eye view image; Based on whether the folding angle, the first part, the turning state, and the first preset error angle have been determined in the previous frame of the target image, the detection area in the first bird's-eye view image is determined; The detection area is processed using a contour extraction algorithm to obtain an initial set of contour points; The target contour point set is obtained by filtering all initial contour point sets according to the preset filtering strategy. The position of each point in each target contour point set in the first bird's-eye view image is used as the key point position.

5. The method according to claim 4, characterized in that, Based on whether the folding angle, the first portion, the turning state, and the first preset error angle have been determined in the previous frame of the target image, the detection area in the first bird's-eye view image is determined, including: Based on the first part and the turning state, the target side is determined, which is either the left or the right side; If the folding angle of the previous frame of the target image is not determined, then the target side half region in the first bird's-eye view image is taken as the detection region; If the folding angle of the previous frame of the target image has been determined, then a first angle and a second angle are generated based on the folding angle and the first preset error angle. The detection area is generated based on the first angle, the second angle, and the midpoint of the bottom edge of the first bird's-eye view image.

6. The method according to claim 5, characterized in that, Determining the target side based on the first part and the turning state includes: If the first part is a trailer and the turning state is a left turn, then the target side is determined to be the right side; If the first part is a trailer and the turning state is a right turn, then the target side is determined to be the left side; If the first part is a tractor and the turning state is a left turn, then the target side is determined to be the left side; If the first part is a tractor and the turning state is a right turn, then the target side is determined to be the right side.

7. The method according to claim 4, characterized in that, The step of calculating the target folding angle based on the location of each key point and the obtained straight reference point includes: Based on the location of each key point, generate the turning line corresponding to each target contour point set; Generate a straight line based on the position of the straight reference point; Based on each turning line and the straight line, generate multiple initial folding angles; The target fold angle is generated based on whether the fold angle has been determined in the previous frame of the target image, and each initial fold angle.

8. The method according to claim 7, characterized in that, The target fold angle is generated based on whether the fold angle has been determined in the previous frame of the target image, and each initial fold angle, including: If the folding angle of the previous frame of the target image is not determined, then the average value of all initial folding angles is taken as the target folding angle; If the folding angle of the previous frame of the target image has been determined, then an error angle range is generated based on the folding angle and the second preset error angle. The target folding angle is generated based on the error angle range and each initial folding angle.

9. The method according to claim 3, characterized in that, The step of performing key point detection processing based on the target image to obtain the key point locations includes: The target image is detected using a vehicle body edge key point detection model to obtain initial edge points. The vehicle body edge key point detection model is a pre-trained deep learning model for obtaining edges from images. The vehicle body edge key point detection model includes a backbone network, a neck network, a curve feature extraction layer, a feature fusion layer, and a key point regression head. The target image is subjected to distortion correction and viewpoint transformation processing to obtain a second bird's-eye view image, and there is a correspondence between the points in the target image and the second bird's-eye view image; The position of each point corresponding to the initial edge point in the second bird's-eye view is taken as the key point position.

10. The method according to claim 9, characterized in that, The step of calculating the target folding angle based on the location of each key point and the obtained straight reference point includes: Generate a turning line based on the locations of all key points; Generate a straight line based on the position of the straight reference point; The target folding angle is determined based on the turning line and the straight line.

11. The method according to claim 3, characterized in that, The first part is a trailer. The step of performing key point detection processing based on the target image to obtain the key point locations includes: Within a preset detection area of ​​the target image, a matching process is performed based on a preset rearview mirror image to obtain the rearview mirror area; The location of the center point of the rearview mirror area is taken as the location of the key point.

12. The method according to claim 11, characterized in that, The straight-ahead reference point positions include the left straight-ahead point position and the right straight-ahead point position. The calculation of the target folding angle based on each key point position and the obtained straight-ahead reference point positions includes: Based on the turning state, the target straight-ahead point position is determined from the left straight-ahead point position and the right straight-ahead point position; Calculate the turning vector based on the key point locations and preset camera intrinsic parameters; Calculate the straight-line vector based on the target straight-line point position and the preset camera intrinsic parameters; The target folding angle is calculated based on the steering vector and the straight-line vector.

13. The method according to claim 12, characterized in that, Before acquiring the current frame image captured by each camera when the vehicle is detected to be turning, the method further includes: Obtain the left and right images of the vertical line; The left and right images of the straight line are respectively taken as target straight line images, and the following processing is performed: Based on whether the target straight image is a straight left image, the target region is determined from the first preset region and the second preset region; Contour extraction is performed on the straight-line image of the target to obtain the vehicle body contour points; The first rearview mirror contour point is determined based on the target area and the vehicle body contour points; Within the target area of ​​the target straight image, a matching process is performed based on a preset rearview mirror image to determine the contour points of the second rearview mirror. Based on the first rearview mirror contour points and the second rearview mirror contour points, Kalman filtering is performed to obtain the target rearview mirror contour points. If the target straight image is a straight left image, then the position of the center point of the target rearview mirror contour point is taken as the position of the left straight point; If the target straight image is a straight right image, then the position of the center point of the target rearview mirror contour point is taken as the position of the right straight point.

14. The method according to claim 2, characterized in that, The method further includes: If the left turn signal is detected to be on, it is determined that the vehicle is in a left turn state. If the right turn signal is detected to be on, it indicates that the vehicle is in a right turn state.

15. The method according to claim 1, characterized in that, If the number of cameras is 1, the first part is a trailer, and the camera is installed at the front of the trailer, the key point detection processing based on each current frame image to obtain the key point position includes: Perform foreground segmentation processing on the current frame image to obtain the first tractor image; The first tractor image is processed by feature point extraction to obtain multiple feature points and a feature vector for each feature point, wherein the feature vector includes the neighborhood gradient. For each feature point, the position of the feature point in the first tractor image is taken as the key point position, and the feature vector of the feature point is taken as the feature vector of the key point position.

16. The method according to claim 15, characterized in that, The step of calculating the target folding angle based on the location of each key point and the obtained straight reference point includes: Based on the position of each straight reference point, obtain the feature vector of each straight reference point position; Based on the feature vectors of each key point and the feature vectors of each straight reference point, a matching process is performed to obtain initial position pairs; Based on the key point position and straight reference point position in each initial position pair, as well as the neighborhood gradient in the feature vector of the key point position and straight reference point position, a filtering process is performed to obtain the target position pair; Generate a homography matrix based on all target location pairs; The target folding angle is determined based on the homography matrix.

17. The method according to claim 16, characterized in that, The method further includes: Acquire the initial image captured by the camera in real time; The initial image is segmented to obtain the second tractor image; The image of the second tractor is subjected to contour extraction processing to obtain an initial contour; The contour with the largest area among all initial contours is taken as the target contour; The position of the center point of the bounding rectangle of the target contour in the initial image is used as the turning state detection position. Based on the turning state detection location and the preset straight-ahead area, it is determined whether the vehicle is in a turning state.

18. The method according to any one of claims 1 to 17, characterized in that, The method further includes: The target fold angle is processed by Kalman filtering to obtain the updated target fold angle.

19. A device for determining the folding angle of a semi-trailer train, characterized in that, include: The acquisition module is used to acquire the current frame image captured by each camera when the vehicle is detected to be turning, wherein the turning state is either a left turn or a right turn. The processing module is used to perform key point detection processing on each current frame image to obtain the key point positions; The angle determination module is used to calculate the target folding angle based on the position of each key point and the position of the obtained straight reference point.

20. An electronic device, characterized in that, include: Processor, memory, communication interface; The memory is used to store the executable instructions of the processor; The processor is configured to execute the method for determining the folding angle of a semi-trailer train according to any one of claims 1 to 18 by executing the executable instructions.

21. A semi-trailer train, characterized in that, Includes at least one camera and controller; The at least one camera is mounted on the first part of the semi-trailer and faces the second part, wherein the first part is the tractor and the second part is the trailer, or the first part is the trailer and the second part is the tractor; The controller is used to execute the method for determining the folding angle of a semi-trailer as described in any one of claims 1 to 18.

22. A readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the method for determining the folding angle of a semi-trailer train as described in any one of claims 1 to 18.

23. A computer program product, characterized in that, Includes a computer program, which, when executed by a processor, is used to implement the method for determining the folding angle of a semi-trailer train as described in any one of claims 1 to 18.

Images