Tractor docking auxiliary method, device, equipment and storage medium

Abstract

The present application discloses a method, device, equipment and storage medium for assisting tractor docking, the method comprising: if it is determined that the towing pin appears in the camera's field of view, based on the relative motion trajectory of the camera following the movement of the tractor relative to the towing pin, controlling the rotating mechanism to drive the camera to rotate so that the towing pin is always in the field of view; determining the azimuth projection of the saddle in the field of view; determining the relative position of the towing pin and the azimuth projection in the field of view, so that the user can dock the towing pin and the saddle based on the relative position. In the present application, during the docking process of the tractor and the trailer, the camera is adjusted according to the motion trajectory of the towing pin in the field of view, so that the towing pin is always in the field of view, and the user can control the docking of the tractor and the trailer through the field of view, without the need for reminders from observers, thereby improving the efficiency of docking.

Description

Technical Field

[0001] The present application relates to the technical field of tractor drop-and-hook docking, and in particular to a tractor docking assistance method, device, equipment and storage medium. Background Art

[0002] During the transportation process, the trailer can be separated from the tractor after the goods are transported to the destination, so that the tractor can directly hook up other trailers or return directly, saving the time waiting for loading and unloading of goods and increasing the rate of obtaining transportation.

[0003] Since the drop-and-hook transport can separate the tractor and trailer to save transportation time, the tractor and trailer need to be docked before each drop-and-hook transport. At present, the method of docking the tractor and trailer is to install a camera and a distance sensor on the saddle of the tractor, obtain the field of view of the towing pin through the camera, and determine the height of the towing pin to the saddle through the distance sensor, so as to adjust the height of the saddle. However, this adjustment method requires another observer to guide the tractor to the point where the camera can recognize the towing pin. In the end, two people are still needed to complete the connection between the tractor and the trailer, resulting in low efficiency of the hanger. Summary of the invention

[0004] The main purpose of the present application is to provide a tractor docking auxiliary method, device, equipment and storage medium, aiming to solve the technical problem in the prior art that two people are required to complete the coupling of the tractor and the trailer, resulting in low efficiency of the hook.

[0005] To achieve the above-mentioned purpose, the present application provides a tractor docking assistance method, which is applied to an auxiliary controller in a tractor docking assistance system. The tractor docking assistance system also includes a tractor and a trailer. The trailer is provided with a traction pin, the tractor is provided with a saddle and a rotating mechanism, a camera is provided on the rotating mechanism, the saddle and the camera follow the movement of the tractor, and the auxiliary controller controls the rotating mechanism to drive the camera to rotate so that the traction pin is within the field of view of the camera, so as to complete the docking of the saddle and the traction pin. The tractor docking assistance method includes:

[0006] If it is determined that the traction pin appears in the camera's field of view, based on the relative motion trajectory of the camera following the movement of the tractor relative to the traction pin, the rotating mechanism is controlled to drive the camera to rotate so that the traction pin is always in the field of view;

[0007] Determining the azimuth projection of the saddle in the field of view;

[0008] The relative position of the traction pin and the azimuth projection in the field of view is determined so that the user can dock the traction pin and the saddle based on the relative position.

[0009] Optionally, if it is determined that the towing pin appears in the camera field of view, based on the relative motion trajectory of the camera following the movement of the towing vehicle relative to the towing pin, the step of controlling the rotating mechanism to drive the camera to rotate so that the towing pin is always in the field of view includes:

[0010] If it is determined that the traction pin appears in the camera field of view, determining the field of view position of the traction pin in the field of view;

[0011] Determine the relative motion trajectory of the camera following the tractor relative to the traction pin using a preset target recognition algorithm;

[0012] Based on the visual field position and the relative motion trajectory, determining an adjustment angle so that the traction pin is always in the visual field;

[0013] The rotating mechanism is controlled to drive the camera to rotate the adjustment angle so that the traction pin is always in the field of view.

[0014] Optionally, the step of determining, based on the field of view position and the relative motion trajectory, an adjustment angle for making the traction pin always be in the field of view includes:

[0015] Based on the field of view position and the relative motion trajectory, an adjustment angle is determined so that the traction pin is always located at a preset center position in the field of view.

[0016] Optionally, the step of determining the relative motion trajectory of the camera following the movement of the tractor relative to the traction pin by using a preset target recognition algorithm comprises:

[0017] Determine the historical movement trajectory of the traction pin in the field of view;

[0018] By using a preset target recognition algorithm and based on the historical motion trajectory, the movement trend of the traction pin in the field of view is analyzed to obtain the relative motion trajectory of the camera following the movement of the tractor relative to the traction pin.

[0019] Optionally, the step of determining the relative position of the traction pin and the azimuth projection in the field of view so that the user can dock the traction pin and the saddle based on the relative position includes:

[0020] Determining the relative position of the traction pin and the azimuth projection in the visual field;

[0021] Based on the relative position, a docking guide line is outputted, so that the user can dock the traction pin and the saddle based on the docking guide line.

[0022] Optionally, the step of determining the relative position of the traction pin and the azimuth projection in the field of view includes:

[0023] Identifying a reference feature from the field of view, and determining a relative position of the traction pin with respect to the azimuth projection based on the reference feature;

[0024] The step of outputting a docking guide line based on the relative position comprises:

[0025] Based on the relative position, a virtual line is formed between the traction pin and the azimuth projection to obtain the docking guide line.

[0026] Optionally, the step of determining the azimuth projection of the saddle in the field of view includes:

[0027] When the saddle does not exist in the visual field, projecting the saddle along the plane of rotation of the camera onto the edge of the visual field to obtain an azimuthal projection;

[0028] When the saddle exists in the visual field, the saddle is projected into the visual field to obtain an azimuthal projection.

[0029] The present application also provides a tractor docking auxiliary device, which is applied to an auxiliary controller in a tractor docking auxiliary system. The tractor docking auxiliary system also includes a tractor and a trailer. The trailer is provided with a traction pin, the tractor is provided with a saddle and a rotating mechanism, a camera is provided on the rotating mechanism, the auxiliary controller controls the rotating mechanism to drive the camera to rotate so that the traction pin is within the field of view of the camera. The tractor docking auxiliary device includes:

[0030] An adjustment module, for controlling the rotating mechanism to drive the camera to rotate, if it is determined that the traction pin appears in the camera field of view, based on the relative motion trajectory of the camera following the movement of the tractor relative to the traction pin, so that the traction pin is always in the field of view;

[0031] A projection determination module, used to determine the azimuth projection of the saddle in the field of view;

[0032] The position determination module is used to determine the relative position of the traction pin and the azimuth projection in the field of view, so that the user can dock the traction pin and the saddle based on the relative position.

[0033] The present application also provides a tractor docking assistance device, which is a physical node device. The tractor docking assistance device includes: a memory, a processor, and a program of the tractor docking assistance method stored in the memory and executable on the processor. When the program of the tractor docking assistance method is executed by the processor, the steps of the tractor docking assistance method as described above can be implemented.

[0034] The present application also provides a storage medium, on which is stored a program for implementing the above-mentioned tractor docking assistance method. When the program of the tractor docking assistance method is executed by a processor, the steps of the above-mentioned tractor docking assistance method are implemented.

[0035] The present application provides a tractor docking assistance method, device, equipment and storage medium. Compared with the prior art in which two persons are required to complete the coupling of the tractor and the trailer, resulting in low efficiency of the coupling, in the present application, if it is determined that the towing pin appears in the camera's field of view, based on the relative motion trajectory of the camera following the movement of the tractor relative to the towing pin, the rotating mechanism is controlled to drive the camera to rotate so that the towing pin is always in the field of view; the azimuth projection of the saddle in the field of view is determined; and the relative position of the towing pin and the azimuth projection in the field of view is determined, so that the user can dock the towing pin and the saddle based on the relative position. In the present application, when it is determined that the towing pin appears in the camera's field of view, the rotation mechanism is controlled to drive the camera to rotate according to the movement trajectory of the towing pin in the field of view, so that the towing pin is always in the field of view, and the azimuthal projection of the saddle in the field of view and the relative position of the towing pin in the field of view are determined, so that the user can dock the towing pin and the saddle based on the relative position. That is, in the present application, during the docking process of the tractor and the trailer, the camera is adjusted according to the movement trajectory of the towing pin in the field of view, so that the towing pin is always in the field of view, and the user can control the docking of the tractor and the trailer through the field of view, without the need for reminders from observers, thereby improving the efficiency of docking. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0037] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0038] Figure 1 This is a flow chart of the first embodiment of the tractor docking assistance method of the present application;

[0039] Figure 2 This is a schematic diagram of the application principle of the tractor docking auxiliary system for this application;

[0040] Figure 3 A schematic diagram of the device structure of the hardware operating environment involved in the embodiment of the present application;

[0041] Figure 4 for Figure 2 Structural diagram of the rotating mechanism;

[0042] Figure 5 This is a schematic diagram of the application process of the tractor docking assistance method of this application.

[0043] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0044] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0045] The present application embodiment provides a tractor docking assistance method. In the first embodiment of the tractor docking assistance method of the present application, refer to Figure 1 , the tractor docking auxiliary method comprises:

[0046] Step S10, if it is determined that the traction pin appears in the camera field of view, based on the relative motion trajectory of the camera following the movement of the tractor relative to the traction pin, the rotation mechanism is controlled to drive the camera to rotate so that the traction pin is always in the field of view;

[0047] Step S20, determining the azimuth projection of the saddle in the visual field;

[0048] Step S30, determining the relative position of the traction pin and the azimuth projection in the field of view, so that the user can dock the traction pin and the saddle based on the relative position.

[0049] The purpose of this embodiment is to reduce the number of observers during the coupling process and improve the efficiency of the coupling between the tractor and the trailer.

[0050] In this embodiment, it should be noted that the tractor docking assistance method can be applied to a tractor docking assistance device, which belongs to a tractor docking assistance equipment, and the tractor docking assistance equipment belongs to a tractor docking assistance system.

[0051] Specifically, the tractor docking assistance method is applied to an auxiliary controller in a tractor docking assistance system, the tractor docking assistance system also includes a tractor and a trailer, the trailer is provided with a traction pin, the tractor is provided with a saddle and a rotating mechanism, a camera is provided on the rotating mechanism, the saddle and the camera follow the movement of the tractor, and the auxiliary controller controls the rotating mechanism to drive the camera to rotate so that the traction pin is within the field of view of the camera, so as to complete the docking of the saddle and the traction pin.

[0052] In this embodiment, an auxiliary controller is provided in the vehicle control system. It should be noted that the auxiliary controller can perform feature data analysis on the collected field of view images, determine the angle of the rotating mechanism that needs to be adjusted, and can also transmit the field of view images to the tractor.

[0053] In this embodiment, refer to Figure 2 , Figure 2 This is a schematic diagram of the application principle of the tractor docking auxiliary system. The auxiliary controller can be connected to an external camera and a rotating mechanism. The camera is used to shoot the traction pin so that the traction pin appears in the camera's field of view, and the camera is adjusted through the rotating mechanism.

[0054] In this embodiment, reference Figure 4 , Figure 4 This is a structural diagram of the rotating mechanism. It should be noted that the camera is installed in the installation slot 12 of the rotating mechanism. By adjusting the rack 11, the rotating rod 10 is stably adjusted to drive the camera to rotate. By changing the shooting angle of the camera, the position of the traction pin in the field of view is adjusted.

[0055] In this embodiment, when the tractor and trailer begin to dock, the camera shoots in a direction parallel to the tractor. When the user drives the tractor in reverse to dock the trailer, the user observes the position of the trailer through the rearview mirror and adjusts the position of the tractor until the towing pin can be observed from the field of view captured by the camera. The user constantly observes the position of the towing pin through the display screen in the cockpit and adjusts the tractor according to the position of the towing pin so that the towing pin and the saddle on the tractor are aligned. During the backward movement of the tractor, the auxiliary controller determines the adjustment data of the rotating mechanism through the field of view captured by the camera, and adjusts the rotating mechanism according to the adjustment data so that the camera tracks and locates the towing pin so that the towing pin is always located in the center of the field of view, thereby preventing the tractor from losing the field of view of the towing pin during the reverse movement and facilitating the user to quickly find the towing pin from the field of view. In this embodiment, rapid and accurate positioning of the tractor and trailer is achieved.

[0056] In this embodiment, the specific application scenario may be:

[0057] When transporting checked baggage from the terminal to the aircraft cargo hold at the airport, a tractor is required to hook up the bucket full of baggage. During the hooking process, the user is required to drive the tractor while turning his head to look at the bucket behind the tractor. When the towing pin of the bucket and the saddle of the tractor are almost aligned, he needs to get off the vehicle and drag the bucket to align the towing pin and saddle and connect them, which is time-consuming and labor-intensive.

[0058] Alternatively, when using a tractor to plow the land, a plowshare needs to be installed on the back of the tractor, and the plowshare and the tractor are hung together. When the plowshare is installed on the tractor, the tractor needs to be adjusted to a rough position, and then another user pulls the saddle and the traction pin to connect during the tractor adjustment process, resulting in a safety hazard to the user during the connection process.

[0059] In this embodiment, the field of view of the towing pin captured by the camera is used as feedback, and the shooting angle of the camera is adjusted so that the towing pin is always located in the center range of the field of view, and the field of view is transmitted to the display screen of the tractor through the auxiliary controller, so as to facilitate the driving user to adjust the tractor.

[0060] In this embodiment, a trailer and a tractor are taken as examples for illustration.

[0061] The specific steps are as follows:

[0062] Step S10, if it is determined that the traction pin appears in the camera field of view, based on the relative motion trajectory of the camera following the movement of the tractor relative to the traction pin, the rotation mechanism is controlled to drive the camera to rotate so that the traction pin is always in the field of view;

[0063] In this embodiment, the towing pin of the trailer is photographed by a camera. After the features of the towing pin are found in the field of view captured by the camera, the towing pin is tracked and located. During the entire process of coupling the trailer and the tractor, the camera follows the position of the towing pin and adjusts it in real time.

[0064] Reference Figure 5 , for example, in Figure 5 In the M, when the tractor and the trailer just start to dock, the camera's shooting direction is parallel to the tractor. When the user aligns the tractor and the trailer, the horizontally placed camera captures the field of view directly behind the tractor and uploads the field of view to the auxiliary controller. The auxiliary controller identifies the trailer's towing pin, and after identifying the towing pin, tracks and locates the towing pin. When the tractor continues to reverse, the relative position of the towing pin changes in the horizontal direction. When the relative position of the towing pin changes, the auxiliary controller controls the rotating structure to drive the camera to rotate. Figure 5In N, when the tractor backs up to the position where the relative position of the towing pin is directly above the camera, the auxiliary controller also rotates and adjusts relative to the control rotation mechanism to adjust the camera shooting direction to be perpendicular to the horizontal plane so that the camera can shoot the towing pin. Figure 5 In the Q, when the tractor backs up until the towing pin and the saddle of the tractor are successfully docked, the auxiliary controller controls the rotating mechanism to drive the camera to rotate so that the camera can capture the docked towing pin. The rotating mechanism drives the camera to rotate in the process of following the relative movement of the towing pin, not from Figure 5 The position of M can be quickly adjusted to Figure 5 The position of N.

[0065] In this embodiment, it should be noted that by setting the initial shooting direction of the camera to the horizontal direction, the field of view of the tractor when reversing can be increased, and the towing pin of the trailer can be identified when the tractor and the trailer are at a safe distance, without the need for another user to act as an observer to direct the driving user.

[0066] Specifically, if it is determined that the traction pin appears in the camera field of view, based on the relative motion trajectory of the camera following the movement of the tractor relative to the traction pin, the step of controlling the rotating mechanism to drive the camera to rotate so that the traction pin is always in the field of view includes:

[0067] Step S11, if it is determined that the traction pin appears in the camera field of view, determining the field of view position of the traction pin in the field of view;

[0068] Step S12, using a preset target recognition algorithm to determine the relative motion trajectory of the camera following the movement of the tractor relative to the traction pin;

[0069] Step S13, based on the visual field position and the relative motion trajectory, determining an adjustment angle for making the traction pin always be in the visual field;

[0070] Step S14, controlling the rotating mechanism to drive the camera to rotate the adjustment angle so that the traction pin is always in the field of view.

[0071] The auxiliary controller is provided with a database obtained through iterative training of data of preset traction pin labels.

[0072] Specifically, the type of the traction pin is not limited to one, and may include but is not limited to a mushroom-shaped traction pin, a cross-shaped traction pin, or a straight-shaped traction pin, etc.

[0073] It should be noted that since the auxiliary controller is equipped with a database obtained through iterative training of data of preset traction pin labels, different types of traction pins can be identified according to the database, making it convenient to extract the characteristic data of the traction pin from the field of view picture according to the database, and then determine the position of the traction pin in the field of view picture.

[0074] The characteristic data may include but are not limited to the shape, size, installation direction, installation position, etc. of the traction pin.

[0075] In this embodiment, the position of the traction pin in the field of view is identified based on its shape, size, installation position, and installation direction, and then the trajectory of the traction pin is analyzed through a target recognition algorithm to determine the trajectory along which the traction pin has moved. The future position of the traction pin is predicted based on the trajectory that has already moved to obtain the predicted trajectory of the traction pin. The field of view position of the traction pin in the field of view can be predicted in advance based on the predicted trajectory, and the adjustment angle of the rotating mechanism can be determined based on the field of view position of the traction pin in the field of view to avoid the traction pin being lost in the field of view.

[0076] In this embodiment, the auxiliary controller controls the rotating mechanism to drive the camera to rotate according to the adjustment angle, and adjusts the position of the traction pin in the field of view so that the traction pin is always in the field of view, thereby avoiding the traction pin from being lost in the field of view and preventing the user from seeing the position of the traction pin in the field of view.

[0077] Specifically, the step of determining the adjustment angle for making the traction pin always be in the field of view based on the field of view position and the relative motion trajectory includes:

[0078] Step S131, based on the field of view position and the relative motion trajectory, determining an adjustment angle for making the traction pin always be at a preset center position in the field of view.

[0079] In this embodiment, a center range can be preset in the field of view. Since the position of the towing pin in the field of view can be predicted, when the towing pin has not exceeded the center range, that is, when the towing pin has not exceeded the danger frame, the adjustment length of the towing pin that needs to be adjusted is determined, and the adjustment length is converted into the adjustment angle required by the camera. According to the adjustment angle, the shooting angle of the camera is adjusted in advance to adjust the towing pin to the center range.

[0080] The adjustment length may include but is not limited to the distance from the traction pin to the center of the field of view and the adjustment angle converted based on the distance.

[0081] It should be noted that it is necessary to refer to the time for the towing pin to move, and to adjust the camera in combination with the time required for camera adjustment to avoid excessive adjustment. For example, it is predicted that the towing pin will exceed the center range after 2.5 seconds, and the camera needs to be rotated 10 degrees to adjust the towing pin to the center position of the center range, and the time required to adjust the camera 10 degrees is 0.5 seconds. If the camera is adjusted within 1 second, the position of the towing pin will exceed the center range above the display screen, and the camera needs to be adjusted within 1 second to 2 seconds.

[0082] Among them, the central range can be brighter than other fields of view, and other fields of view will not affect normal viewing.

[0083] In this embodiment, based on the predicted position of the towing pin, it is judged and analyzed whether the camera can capture the towing pin at the predicted position, and whether the captured towing pin is within the center range. If the towing pin is not within the center range, the adjustment angle of the camera is determined, and the rotating mechanism is adjusted based on the adjustment angle, thereby adjusting the shooting angle of the camera to position the towing pin within the center range and obtain a field of view image.

[0084] In this embodiment, the auxiliary controller tracks and calculates the relative position of the traction pin according to the field of view, and predicts the relative position of the traction pin in the next frame. When it is predicted that the relative position of the traction pin in the next frame will exceed the center range, the adjustment angle that the camera needs to adjust is determined. The auxiliary controller adjusts the angle of the rotating mechanism according to the adjustment angle so that the image of the traction pin captured by the camera is located in the center range, and determines the field of view that the user can observe in the display, thereby avoiding the loss of the field of view of the traction pin in the display.

[0085] Among them, the field of view picture is an image of the field of view video captured and played on the display screen. Since the display screens have different sizes, the center range in the field of view picture is determined based on the display screen, that is, the center range of the display screen.

[0086] In this embodiment, the field of view is used as feedback, and the auxiliary controller is used to perform feature recognition on the field of view. The traction pin is identified from the field of view, and the camera adjustment angle is determined by the position of the traction pin in the field of view. The position of the traction pin in the field of view is always kept within a preset center range to avoid loss of field of view of the traction pin.

[0087] For example, the top of the field of view is the direction in which the traction pin approaches the saddle, and the bottom of the field of view is the direction in which the traction pin moves away from the saddle. If the center range is preset to spread outward from the center position of the field of view to half of the entire field of view area, the auxiliary controller detects that the position of the traction pin in the field of view is outside the center range and is above the field of view, and the distance from the center position is three centimeters. If the auxiliary controller calculates that the adjustment angle corresponding to three centimeters is five degrees, the auxiliary controller adjusts the rotating mechanism to rotate five degrees toward the saddle, so that the position of the traction pin in the field of view is within the center range, and the field of view viewed by the user is obtained through the display screen.

[0088] In this embodiment, the auxiliary controller adjusts the acquisition angle through the image data fed back by the field of view picture to obtain a field of view picture with the towing pin in the middle range, and transmits the field of view picture to the display screen in the tractor's cockpit, so that the user can control the tractor through the position of the towing pin in the field of view picture and adjust the saddle.

[0089] Specifically, the step of using a preset target recognition algorithm to determine the relative motion trajectory of the camera following the movement of the tractor relative to the traction pin includes:

[0090] Step S121, determining the historical movement trajectory of the traction pin in the field of view;

[0091] Step S122, using a preset target recognition algorithm, based on the historical motion trajectory, the movement trend of the traction pin in the field of view is analyzed to obtain the relative motion trajectory of the camera following the movement of the tractor relative to the traction pin.

[0092] In this embodiment, during the docking process between the tractor and the trailer, since the visual field captured by the camera is a continuous moving video of the towing pin relative to the tractor, the most likely position of the towing pin in the next frame can be predicted through the continuous relative moving video, and the predicted positions can be combined into a relative motion trajectory. The time when the towing pin is lost in the visual field can be predicted in advance through the relative predicted trajectory, and the camera can be adjusted in time.

[0093] For example, by analyzing the continuous relative movement of the traction pin in the video, it is predicted that the most likely position of the traction pin in the next frame is one centimeter to the right of the current position, and the field of view position is determined to be one centimeter to the right of the current position.

[0094] It should be noted that the prediction of the traction pin position is an iterative prediction. When the traction pin is just detected, the position of the traction pin is far away from the saddle, and the error of the predicted position of the traction pin is the largest. Since the initial error has little effect on the final docking, it can be used as a training object for iterative prediction. As the distance between the traction pin and the saddle gets closer, the error of the predicted position of the traction pin becomes smaller and smaller, so that the accuracy of the docking between the traction pin and the saddle becomes higher and higher.

[0095] Step S20, determining the azimuth projection of the saddle in the field of view.

[0096] In this embodiment, the saddle position is projected into the visual field, and an azimuth projection is obtained at the edge of the visual field. The azimuth projection is regarded as the saddle in the visual field, and the position between the traction pin and the saddle is determined through the azimuth projection.

[0097] Specifically, the step of determining the azimuth projection of the saddle in the field of view includes:

[0098] Step S21, when the saddle does not exist in the visual field, projecting the saddle along the plane of rotation of the camera onto the edge of the visual field to obtain an azimuthal projection;

[0099] Step S22: When the saddle exists in the visual field, project the saddle into the visual field to obtain an azimuth projection.

[0100] In this embodiment, since the camera is rotating for shooting, when there is no saddle in the visual field, the saddle is projected onto the edge of the visual field along the plane of camera rotation to obtain the azimuth projection of the saddle.

[0101] Step S30, determining the relative position of the traction pin and the azimuth projection in the field of view, so that the user can dock the traction pin and the saddle based on the relative position.

[0102] In this embodiment, since the traction pin and the saddle need to be connected by inserting the traction pin into the saddle for connection, it is necessary to determine whether the socket of the saddle corresponds to the traction pin after the traction pin and the saddle are positioned and are about to be connected.

[0103] In this embodiment, when the traction pin and the saddle are docked, the position of the traction pin and the saddle basically coincide. At this time, the camera can capture the saddle, which is convenient for the user to observe. If the socket of the saddle and the traction pin do not correspond, the connection position of the saddle and the traction pin is analyzed through the field of view, the current socket position of the saddle is determined, the angle at which the saddle needs to be adjusted is determined, and the adjustment angle is obtained. The adjustment angle is fed back to the display screen to prompt the user of the exact angle at which the saddle needs to be adjusted so that the saddle and the traction pin can be accurately docked.

[0104] It should be noted that if the socket of the saddle and the towing pin correspond, the user can be prompted to directly dock through the display screen, reducing the risk of rollover that may occur during docking. For example, if the towing pin and the vehicle mouth of the saddle do not correspond, then when the towing pin and the saddle are docked, the towing pin cannot be inserted into the socket of the saddle. As the tractor reverses, the saddle will press against the towing pin and the trailer to retreat together. Since the saddle and the towing pin are not docked but only abutted, when the user finds that the trailer retreats with the tractor, he will think that the trailer and the tractor have been successfully connected, and may control the tractor to stop. At this time, the trailer will continue to reverse uncontrollably due to inertia, which may cause rollover or other risks.

[0105] In this embodiment, since the saddle may have deviations, the traction pin and the saddle may not be properly docked, causing the saddle or the traction pin to wear out faster or be damaged, thereby reducing the service life of the saddle and the traction pin. After determining the adjustment angle of the saddle's socket, the adjustment angle between the saddle and the traction pin is determined in combination with the docking guide line to improve the accuracy of the docking between the traction pin and the saddle.

[0106] In this embodiment, after the camera detects the towing pin, the field of view of the towing pin is obtained, and the towing pin is tracked so that the towing pin is always within the center range of the field of view on the display screen to avoid loss of the field of view of the towing pin. When the towing pin and the saddle are docked, the camera can capture the saddle. After capturing the saddle, the position of the saddle socket is determined, and based on the position of the saddle socket, the adjustment angle is determined so that the saddle and the towing pin can be accurately docked.

[0107] The present application provides a tractor docking assistance method, device, equipment and storage medium. Compared with the prior art in which two persons are required to complete the coupling of the tractor and the trailer, resulting in low efficiency of the coupling, in the present application, if it is determined that the towing pin appears in the camera's field of view, based on the relative motion trajectory of the camera following the movement of the tractor relative to the towing pin, the rotating mechanism is controlled to drive the camera to rotate so that the towing pin is always in the field of view; the azimuth projection of the saddle in the field of view is determined; and the relative position of the towing pin and the azimuth projection in the field of view is determined, so that the user can dock the towing pin and the saddle based on the relative position. In the present application, when it is determined that the towing pin appears in the camera's field of view, the rotation mechanism is controlled to drive the camera to rotate according to the movement trajectory of the towing pin in the field of view, so that the towing pin is always in the field of view, and the azimuthal projection of the saddle in the field of view and the relative position of the towing pin in the field of view are determined, so that the user can dock the towing pin and the saddle based on the relative position. That is, in the present application, during the docking process of the tractor and the trailer, the camera is adjusted according to the movement trajectory of the towing pin in the field of view, so that the towing pin is always in the field of view, and the user can control the docking of the tractor and the trailer through the field of view, without the need for reminders from observers, thereby improving the efficiency of docking.

[0108] Further, based on the above embodiment of the present application, another embodiment of the present application is provided. In this embodiment, the step of determining the relative position of the traction pin and the azimuth projection in the field of view so that the user can dock the traction pin and the saddle based on the relative position includes:

[0109] Step A10, determining the relative position of the traction pin and the azimuth projection in the visual field;

[0110] Step A20: output a docking guide line based on the relative position, so that the user can dock the traction pin and the saddle based on the docking guide line.

[0111] It should be noted that the position between the saddle and the camera is known and can be used directly as a parameter, avoiding the situation where the position between the saddle and the camera is ignored, resulting in a large error in the relative position between the saddle and the traction pin, causing the traction pin and the saddle to be unable to dock accurately.

[0112] In this embodiment, a docking guide line is set according to the relative positions of the towing pin and the azimuth projection, that is, the towing pin and the azimuth projection can be virtually connected to obtain the docking guide line, and the docking guide line is displayed in the field of view, so that the user can more intuitively observe the position of the towing pin relative to the saddle, and the guide line can guide the user to control the docking of the tractor and trailer.

[0113] Specifically, the step of determining the relative position of the traction pin and the azimuth projection in the field of view includes:

[0114] Step A11, identifying a reference feature from the visual field image, and determining the relative position of the traction pin with respect to the azimuth projection based on the reference feature;

[0115] Specifically, the step of outputting a docking guide line based on the relative position includes:

[0116] Step A12: based on the relative position, virtually connect the traction pin and the azimuth projection to obtain the docking guide line. Step S31: identify a reference feature from the field of view, and determine the relative position of the traction pin with respect to the azimuth projection based on the reference feature.

[0117] In this embodiment, a feature that always exists in the field of view is identified from the field of view, and the feature is used as a reference to determine the relative positions of the towing pin and the saddle. For example, there are electric wires above the tractor and the trailer, and the camera can just capture the electric wires within the field of view. During the process of connecting the saddle of the tractor and the towing pin of the trailer, the electric wires can always be captured by the camera, and the electric wires can be used as a reference feature.

[0118] In this embodiment, the user is prompted with reference features to avoid the user having to look for reference objects as reference features when adjusting the tractor to determine the relative positions of the tractor and the trailer; the reference features in the field of view can be located by the auxiliary controller to analyze the specific positions between the reference features and the tractor, and the specific positions between the reference features and the trailer, so that the relative positions of the tractor and the trailer obtained by the user are more accurate.

[0119] Further, based on the above embodiment of the present application, another embodiment of the present application is provided. In this embodiment, if the field of view position exceeds the central range, the distance from the traction pin to the center of the field of view is calculated to obtain the step of adjusting the length, and the method includes:

[0120] Step B10, creating a virtual coordinate system for the visual field picture, determining the visual field coordinates of the traction pin in the virtual coordinate system, and determining the boundary coordinates of the visual field boundary;

[0121] Step B20, calculating the distance between the field of view coordinates and the boundary coordinates to obtain the escape distance between the traction pin and the field of view boundary;

[0122] Step B30: If the escape distance is greater than the preset loss distance, determine that the position of the traction pin in the field of view exceeds the center range.

[0123] In this embodiment, the origin of the virtual coordinate system created for the visual field picture may be set at the center of the visual field picture, or may be set at a border corner of the visual field picture, etc., without specific limitation.

[0124] It should be noted that the coordinates can be used to more accurately determine the field of view coordinates of the traction pin to reduce errors in docking.

[0125] In this embodiment, the boundary coordinates of the visual field boundary are determined by the coordinate system, that is, the range size of the visual field screen is determined, and the visual field coordinates of the traction pin in the virtual coordinates are determined. Since the visual field coordinates of the traction pin are a certain distance away from the boundary coordinates, it is possible that the user cannot quickly find the traction pin in the display screen and is prone to lose the visual field of the traction pin. Therefore, the distance between the current visual field coordinates and the boundary is calculated to obtain the escape distance of the traction pin from the visual field boundary, and then the escape distance is compared with the preset loss distance. If the escape distance is greater than the preset loss distance, it is determined that the position of the traction pin in the visual field screen exceeds the center range. At this time, it is necessary to determine the acquisition angle that needs to be adjusted and adjust it. In this embodiment, the user can always observe the traction pin when docking the trailer, and adjust the tractor according to the observed position of the traction pin, so that the tractor and the trailer are accurately docked.

[0126] Reference Figure 3 , Figure 3 It is a schematic diagram of the device structure of the hardware operating environment involved in the embodiment of the present application.

[0127] like Figure 3 As shown, the tractor docking auxiliary device may include: a processor 1001, such as a CPU, a memory 1005, and a communication bus 1002. The communication bus 1002 is used to realize the connection and communication between the processor 1001 and the memory 1005. The memory 1005 may be a high-speed RAM memory or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0128] Optionally, the tractor docking auxiliary device may also include a rectangular user interface, a network interface, a camera, an RF (Radio Frequency) circuit, a sensor, an audio circuit, a WiFi module, etc. The rectangular user interface may include a display screen (Display), an input submodule such as a keyboard (Keyboard), and the optional rectangular user interface may also include a standard wired interface and a wireless interface. The network interface may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface).

[0129] Those skilled in the art will understand that Figure 3 The structure of the tractor docking auxiliary device shown in the figure does not constitute a limitation on the tractor docking auxiliary device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0130] like Figure 3 As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, and a tractor docking assistance program. The operating system is a program that manages and controls the hardware and software resources of the tractor docking assistance device, and supports the operation of the tractor docking assistance program and other software and / or programs. The network communication module is used to realize the communication between the components inside the memory 1005, and the communication with other hardware and software in the tractor docking assistance system.

[0131] exist Figure 3 In the tractor docking assistance device shown, the processor 1001 is used to execute the tractor docking assistance program stored in the memory 1005 to implement the steps of any of the above-mentioned tractor docking assistance methods.

[0132] The specific implementation of the tractor docking assistance device of the present application is basically the same as the various embodiments of the tractor docking assistance method described above, and will not be repeated here.

[0133] The present application also provides a tractor docking auxiliary device, which is applied to an auxiliary controller in a tractor docking auxiliary system. The tractor docking auxiliary system also includes a tractor and a trailer. The trailer is provided with a traction pin, the tractor is provided with a saddle and a rotating mechanism, a camera is provided on the rotating mechanism, the auxiliary controller controls the rotating mechanism to drive the camera to rotate so that the traction pin is within the field of view of the camera. The tractor docking auxiliary device includes:

[0134] An adjustment module, for controlling the rotating mechanism to drive the camera to rotate, if it is determined that the traction pin appears in the camera field of view, based on the relative motion trajectory of the camera following the movement of the tractor relative to the traction pin, so that the traction pin is always in the field of view;

[0135] A projection determination module, used to determine the azimuth projection of the saddle in the field of view;

[0136] The position determination module is used to determine the relative position of the traction pin and the azimuth projection in the field of view, so that the user can dock the traction pin and the saddle based on the relative position.

[0137] Optionally, the adjustment module includes:

[0138] A determination module, for determining a visual position of the traction pin in the visual image if it is determined that the traction pin appears in the visual image of the camera;

[0139] A recognition module, used to determine the relative motion trajectory of the camera following the movement of the tractor relative to the traction pin by using a preset target recognition algorithm;

[0140] A calculation module, for determining, based on the visual field position and the relative motion trajectory, an adjustment angle for making the traction pin always be in the visual field;

[0141] The control module controls the rotating mechanism to drive the camera to rotate the adjustment angle so that the traction pin is always in the field of view.

[0142] Optionally, the calculation module includes:

[0143] The calculation submodule is used to determine, based on the field of view position and the relative motion trajectory, an adjustment angle for making the traction pin always be at a preset center position in the field of view.

[0144] Optionally, the identification module includes:

[0145] A determination submodule, used to determine the historical movement trajectory of the traction pin in the field of view;

[0146] The analysis module is used to analyze the movement trend of the traction pin in the field of view based on the historical motion trajectory using a preset target recognition algorithm, and obtain the relative motion trajectory of the camera following the movement of the tractor relative to the traction pin.

[0147] Optionally, the location determination module includes:

[0148] A determination unit, used to determine the relative position of the traction pin and the azimuth projection in the field of view;

[0149] The output module is used to output a docking guide line based on the relative position, so that the user can dock the traction pin and the saddle based on the docking guide line.

[0150] Optionally, the determining unit includes:

[0151] An identification submodule, configured to identify a reference feature from the field of view, and determine a relative position of the traction pin relative to the azimuth projection based on the reference feature;

[0152] The step of outputting a docking guide line based on the relative position comprises:

[0153] A connection module is used to virtually connect the traction pin and the azimuth projection based on the relative position to obtain the docking guide line.

[0154] Optionally, the projection determination module includes:

[0155] A first projection module, used for projecting the saddle along the plane of rotation of the camera onto the edge of the visual field to obtain an azimuthal projection when the saddle does not exist in the visual field;

[0156] The second projection module is used to project the saddle into the visual field picture to obtain an azimuth projection when the saddle exists in the visual field picture.

[0157] The specific implementation of the tractor docking assistance device of the present application is basically the same as the various embodiments of the tractor docking assistance method described above, and will not be repeated here.

[0158] An embodiment of the present application provides a storage medium, and the storage medium stores one or more programs, and the one or more programs can also be executed by one or more processors to implement the steps of any of the above-mentioned tractor docking assistance methods.

[0159] The specific implementation method of the storage medium of the present application is basically the same as the embodiments of the above-mentioned tractor docking auxiliary method, and will not be repeated here.

[0160] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0161] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0162] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0163] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A tractor docking assistance method, characterized in that: An auxiliary controller applied to a tractor docking auxiliary system, the tractor docking auxiliary system also includes a tractor and a trailer, the trailer is provided with a traction pin, the tractor is provided with a saddle and a rotating mechanism, a camera is provided on the rotating mechanism, the saddle and the camera follow the movement of the tractor, the auxiliary controller controls the rotating mechanism to drive the camera to rotate, so that the traction pin is within the field of view of the camera, so as to complete the docking of the saddle and the traction pin, the tractor docking auxiliary method includes: If it is determined that the traction pin appears in the camera's field of view, based on the relative motion trajectory of the camera following the movement of the tractor relative to the traction pin, the rotating mechanism is controlled to drive the camera to rotate so that the traction pin is always in the field of view; Determine an azimuth projection of the saddle in the visual field, wherein the azimuth projection is a projection of the saddle position onto an edge of the visual field; Determine the relative position of the traction pin and the azimuth projection in the visual field, so that the user can dock the traction pin and the saddle based on the relative position; The docking of the traction pin and the saddle includes: When a saddle is present in the visual field, the position of the saddle socket is determined, and the adjustment angle is determined according to the position of the saddle socket so that the saddle and the traction pin can be accurately docked.

2. The tractor docking assisting method according to claim 1, characterized in that: If it is determined that the traction pin appears in the camera field of view, based on the relative motion trajectory of the camera following the movement of the tractor relative to the traction pin, the step of controlling the rotating mechanism to drive the camera to rotate so that the traction pin is always in the field of view includes: If it is determined that the traction pin appears in the camera field of view, determining the field of view position of the traction pin in the field of view; Determine the relative motion trajectory of the camera following the tractor relative to the traction pin using a preset target recognition algorithm; Based on the visual field position and the relative motion trajectory, determining an adjustment angle so that the traction pin is always in the visual field; The rotating mechanism is controlled to drive the camera to rotate the adjustment angle so that the traction pin is always in the field of view.

3. The tractor docking assistance method according to claim 2, characterized in that: The step of determining the adjustment angle to keep the traction pin in the field of view all the time based on the field of view position and the relative motion trajectory comprises: Based on the field of view position and the relative motion trajectory, an adjustment angle is determined so that the traction pin is always located at a preset center position in the field of view.

4. The tractor docking assistance method according to claim 2, characterized in that: The step of using a preset target recognition algorithm to determine the relative motion trajectory of the camera following the movement of the tractor relative to the traction pin comprises: Determine the historical movement trajectory of the traction pin in the field of view; By using a preset target recognition algorithm and based on the historical motion trajectory, the movement trend of the traction pin in the field of view is analyzed to obtain the relative motion trajectory of the camera following the movement of the tractor relative to the traction pin.

5. The tractor docking assistance method according to claim 1, characterized in that: The step of determining the relative position of the traction pin and the azimuth projection in the visual field so that the user can dock the traction pin and the saddle based on the relative position includes: Determining the relative position of the traction pin and the azimuth projection in the visual field; Based on the relative position, a docking guide line is outputted, so that the user can dock the traction pin and the saddle based on the docking guide line.

6. The tractor docking assistance method according to claim 5, characterized in that: The step of determining the relative position of the traction pin and the azimuth projection in the field of view includes: Identifying a reference feature from the field of view, and determining a relative position of the traction pin with respect to the azimuth projection based on the reference feature; The step of outputting a docking guide line based on the relative position comprises: Based on the relative position, a virtual line is formed between the traction pin and the azimuth projection to obtain the docking guide line.

7. The tractor docking assistance method according to claim 1, characterized in that: The step of determining the azimuth projection of the saddle in the field of view includes: When the saddle does not exist in the visual field, projecting the saddle along the plane of rotation of the camera onto the edge of the visual field to obtain an azimuthal projection; When the saddle exists in the visual field, the saddle is projected into the visual field to obtain an azimuthal projection.

8. A tractor docking auxiliary device, characterized in that: An auxiliary controller applied to a tractor docking auxiliary system, the tractor docking auxiliary system also includes a tractor and a trailer, the trailer is provided with a traction pin, the tractor is provided with a saddle and a rotating mechanism, a camera is provided on the rotating mechanism, the auxiliary controller controls the rotating mechanism to drive the camera to rotate so that the traction pin is within the field of view of the camera, and the tractor docking auxiliary device includes: An adjustment module, for controlling the rotating mechanism to drive the camera to rotate, if it is determined that the traction pin appears in the camera field of view, based on the relative motion trajectory of the camera following the movement of the tractor relative to the traction pin, so that the traction pin is always in the field of view; A projection determination module, used to determine the azimuth projection of the saddle in the visual field, wherein the azimuth projection is a projection of the saddle position onto the edge of the visual field; A position determination module, used to determine the relative position of the traction pin and the azimuth projection in the visual field, so that the user can dock the traction pin and the saddle based on the relative position; The position determination module is also used to determine the position of the saddle socket when there is a saddle in the field of view, and determine the adjustment angle according to the position of the saddle socket so that the saddle and the traction pin can be accurately connected.

9. A tractor docking auxiliary device, characterized in that: The tractor docking assistance device comprises: a memory, a processor, and a program stored in the memory for implementing the tractor docking assistance method. The memory is used to store a program for implementing a tractor docking assistance method; The processor is used to execute a program for implementing the tractor docking assistance method to implement the steps of the tractor docking assistance method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium stores a program for implementing the tractor docking assistance method, and the program for implementing the tractor docking assistance method is executed by the processor to implement the steps of the tractor docking assistance method as claimed in any one of claims 1 to 7.

Images