System and method for positioning and marking trailer couplings

Abstract

A system and method for positioning a trailer coupling on a trailer to assist in coupling the trailer coupling with a vehicle hitch on a vehicle. The system includes a human machine interface, a camera, a global positioning system, a remote computing system, a vehicle-to-infrastructure communication network, a controller, a memory, a sensor, and a trailer coupler marking application. The trailer coupler marking application includes: sensing at least one of a hitched state and an unhitched state of the trailer coupler; when the vehicle hook is in a hooked state and a non-hooked state, capturing optical data by using a camera; when the vehicle hook is in the hooking state, the distance of the trailer connector relative to the camera is determined; a position of the trailer coupling is identified on an image defined by the captured data and a marker is created on the image of the trailer coupling.

Description

Technical Field

[0001] The present disclosure relates to advanced driver assistance systems (ADAS), and more particularly to driver assistance systems that identify trailer couplers used to couple a trailer to a vehicle. Vehicles are typically used to tow trailers to transport equipment, cargo, items, animals, other vehicles including boats and off-road vehicles from one place to another along a road. The vehicle is equipped with a hook that is detachably coupled to the trailer coupler of the trailer. The ADAS system needs to correctly identify the position or location of the trailer coupler in order to help couple the trailer to the vehicle. However, due to the differences in color, type, and shape of trailer couplers and the lighting conditions of the trailer's surrounding environment, it may be difficult for the computer vision used by the ADAS system to detect all different types of trailer couplers. Background Art

[0002] Therefore, trailer coupler identification systems have been developed to help locate trailer couplers on trailers. For example, some ADAS systems place an electronic tag or ID on the trailer coupler that can be easily identified by the ADAS's vehicle computer vision system. Although current systems and methods for trailer coupler identification achieve their intended purposes, a new and improved trailer coupler identification system and method is still needed to improve the accuracy of locating a trailer coupler on a trailer from one location to another, while utilizing existing hardware, reducing computational burden, reducing component complexity, and reducing trailer coupling difficulty, thereby improving customer confidence and satisfaction. Summary of the invention

[0003] According to several aspects of the present invention, a system is provided for locating a trailer coupler on a trailer to assist in coupling the trailer coupler to a vehicle hook on a vehicle. The system includes a human-machine interface (HMI) disposed in a vehicle, a camera disposed on the vehicle and detecting the vehicle's surroundings, a global positioning system (GPS) including one or more GPS satellites for obtaining vehicle location information, a remote computing system, a vehicle-to-infrastructure (V2I) communication network, and one or more controllers. Each of the one or more controllers has a processor, a memory, and one or more input / output (I / O) ports, and the I / O ports communicate with one or more sensors, one or more HMIs, GPS, and a remote computing system via the V2I communication network. The memory stores program control logic. The processor executes the program control logic. The program control logic includes a trailer coupler labeling application (TCLA). The TCLA includes program control logic, including: a first control logic for sensing at least one of an attached state and an unattached state of a trailer coupler with a vehicle hitch; a second control logic for capturing optical data using a camera when the vehicle hitch is in an attached state and an unattached state; a third control logic for determining a distance between the vehicle hitch and the trailer coupler relative to the camera when the vehicle hitch is in an attached state; a fourth control logic for sensing movement of the vehicle relative to the trailer when the vehicle hitch is in an unattached state; a fifth control logic for determining a position of the trailer coupler when the vehicle hitch is in an unattached state based at least in part on the distance between the vehicle hitch and the trailer coupler relative to the camera determined when the vehicle hitch is in an attached state and the sensed movement of the vehicle relative to the trailer; and a sixth control logic for identifying a position of the trailer coupler in the captured optical data and creating a mark on the trailer coupler in an image defined by the captured optical data.

[0004] In another aspect of the present disclosure, the first control logic further includes: control logic to request an operator to provide input to the HMI; and control logic to receive the operator input and determine that the vehicle hook is in at least one of a hooked state and an unhooked state.

[0005] In yet another aspect of the present disclosure, the second control logic further includes control logic for storing the optical data as a two-dimensional image of the vehicle hook in an attached state and an undocked state.

[0006] In yet another aspect of the present disclosure, the third control logic further includes control logic that performs feature detection and size calculation on an image defined by the captured optical data to determine a position of the vehicle hook in the image when the vehicle hook is in an engaged state.

[0007] In yet another aspect of the present disclosure, the third control logic further includes control logic to perform feature detection and size calculation on the optical data to determine a position of the vehicle hook when the vehicle hook is in an unhooked state.

[0008] In yet another aspect of the present disclosure, the sixth control logic further includes control logic to create a mark on the trailer coupler in the image defined by the captured optical data at different points in time when the vehicle hitch is in the unhooked state.

[0009] In yet another aspect of the present disclosure, a seventh control logic is further included for training a local machine learning algorithm to identify a trailer coupler based on optical data captured by the camera at different times.

[0010] In another aspect of the present invention, an eighth control logic is included for sending a set of model parameters of the local machine learning algorithm to the remote computer system.

[0011] In yet another aspect of the present disclosure, a ninth control logic is also included for creating one of a plurality of global machine learning algorithms on a remote computer system based on the model parameters received by the local machine learning algorithm.

[0012] In yet another aspect of the present disclosure, a tenth control logic is further included for grouping each of the plurality of global machine learning algorithms into categories based on at least one of vehicle operator provided information, vehicle information, and context information.

[0013] In yet another aspect of the present invention, an eleventh control logic is further included for storing each of the plurality of global machine learning algorithms in a global machine learning database.

[0014] In another aspect of the present disclosure, a twelfth control logic is also included for deploying one of a plurality of global machine learning algorithms stored on the global machine learning database to a vehicle based on at least one of information provided by a vehicle operator, vehicle information, and contextual information.

[0015] In another aspect of the present disclosure, a method is provided for detecting a trailer coupler on a trailer to assist in coupling the trailer coupler to a vehicle hook on a vehicle. The method includes providing a human machine interface (HMI) disposed in a vehicle, providing a camera disposed on the vehicle and detecting the surrounding environment of the vehicle, providing a global positioning system (GPS) including one or more GPS satellites for obtaining vehicle location information, providing a remote computing system, providing a vehicle-to-infrastructure (V2I) communication network, and providing one or more controllers, each of the one or more controllers having a processor, a memory, and one or more input / output (I / O) ports, the I / O ports communicating with one or more sensors, one or more HMIs, GPS, and the remote computing system via the V2I communication network. The memory stores program control logic, which is used to use the program control logic to sense at least one of a hooked state and an unhooked state of a trailer coupler with a vehicle hook; use the program control logic to capture optical data using a camera when the vehicle hook is in the hooked state and the unhooked state; use the program control logic to determine the distance of the vehicle hook and the trailer coupler relative to the camera when the vehicle hook is in the hooked state; use the program control logic to sense the movement of the vehicle relative to the vehicle when the vehicle hook is in the unhooked state; determine the position of the trailer coupler when the vehicle hook is in the unhooked state based at least in part on the distance of the vehicle hook and the trailer coupler relative to the camera determined when the vehicle hook is in the hooked state and the movement of the vehicle relative to the trailer sensed using the program control logic; and use the program control logic to identify the position of the trailer coupler in the captured optical data and create a mark on the trailer coupler in an image defined by the captured optical data.

[0016] In yet another aspect of the present disclosure, the method further includes requesting an operator to provide input to the HMI, and receiving the operator input and determining using program control logic that the vehicle hitch is in at least one of an engaged state and an unengaged state.

[0017] In yet another aspect of the present disclosure, the method further includes storing, using program control logic, the optical data as two-dimensional images of the vehicle hitch in an engaged state and an unengaged state.

[0018] In another aspect of the present disclosure, the method also includes using program control logic to perform feature detection and size calculations on the optical data to determine the position of the vehicle hitch when the vehicle hitch is in a hooked state; and using program control logic to perform feature detection and size calculations on the optical data to determine the position of the trailer coupler when the vehicle hitch is in an unhooked state.

[0019] In yet another aspect of the present disclosure, the method further includes creating, using program control logic, a mark on the trailer coupler in an image defined by the captured optical data at different points in time when the vehicle hitch is in an unhooked state.

[0020] In yet another aspect of the present disclosure, the method further includes training a local machine learning algorithm to identify a trailer coupler based on optical data captured by a camera at different times using program control logic; and sending a set of model parameters of the local machine learning algorithm to a remote computer system using the program control logic.

[0021] In yet another aspect of the present disclosure, the method further includes creating, using program control logic, one of a plurality of global machine learning algorithms on a remote computer system based on model parameters received by the local machine learning algorithm.

[0022] In another aspect of the present disclosure, the method also includes grouping each of a plurality of global machine learning algorithms into categories based on at least one of information provided by a vehicle operator, vehicle information, and contextual information, storing each of the plurality of global machine learning algorithms in a global machine learning database, and deploying one of the plurality of global machine learning algorithms stored on the global machine learning database to a vehicle based on at least one of the information provided by a vehicle operator, vehicle information, and contextual information.

[0023] Further areas of applicability will become apparent from the description provided herein.It should be understood that the description and specific examples are intended for purposes of illustration only and are not intended to limit the scope of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The drawings described herein are for illustration purposes only and are not intended to limit the scope of the present disclosure in any way.

[0025] Figure 1 is a schematic diagram of a system and method for providing trailer coupler identification and marking according to an exemplary embodiment;

[0026] Figure 2 is a perspective view of an interior of a vehicle according to an exemplary embodiment, including a human machine interface (HMI) that displays a Figure 1 A view of a vehicle's surroundings providing systems and methods for trailer coupler tagging;

[0027] Figure 3 is a flow chart of a method for implementing a trailer coupler tag application according to an exemplary embodiment;

[0028] Figure 4 is a schematic diagram showing camera poses as a vehicle and a camera move over time according to an exemplary embodiment; and

[0029] Figure 5 is a block diagram of a process for enhancing a local machine learning model used by a trailer coupler marking application in a vehicle, according to an exemplary embodiment. DETAILED DESCRIPTION

[0030] The following description is merely exemplary in nature and is not intended to limit the present disclosure, application, or uses.

[0031] Reference Figure 1 , a schematic diagram of a system 10 for locating and marking or labeling trailer couplers is shown. The system 10 includes a vehicle 12 and a trailer 14 connected to the vehicle 12 via a hitch 16. Although the vehicle 12 is shown as a passenger car or a sport utility vehicle (SUV), it should be understood that the vehicle 12 can be any type of vehicle without departing from the scope or intent of the present disclosure. In some examples, the vehicle 12 can be a car, a truck, an SUV, a bus, a semi-tractor, a tractor used for agriculture or construction, etc., a pushback tractor or tugboat, a luggage tractor or cargo tractor, etc. Likewise, although Figure 1 The trailer 14 shown in the figure is a multi-axle trailer, but the trailer 14 can be any of a variety of trailer 14 types without departing from the scope or intent of the present disclosure. In some examples, the trailer 14 can be a closed or open-air trailer 14, including a flatbed trailer, a dry cargo transporter, a refrigerated trailer, a dolly trailer, a tractor, a gooseneck trailer, a luggage or cargo trailer, etc. In other examples, the trailer 14 can be another vehicle 12 as described above, or a vehicle 12 such as an airplane, a helicopter, or other such aircraft. That is, a vehicle 12 can tow another vehicle 12 as a trailer 14, and multiple trailers 14 can be towed together in a single line or in parallel with each other.

[0032] The trailer 14 is detachably fixed or mounted to the vehicle 12 via a hook 16 on the vehicle 12 and a trailer coupler 17 on the trailer 14. The hook 16 and the trailer coupler 17 allow the trailer 14 to rotate relative to the vehicle 12, thereby providing a means for the trailer 14 and the vehicle 12 to pass through a turn during travel. Figure 1 The hitch 16 shown is a ball hitch, however, it should be understood that other forms of hitch 16 may be used without departing from the scope or intent of the present invention. For example, the hitch 16 may be a ball hitch, a receiver hitch, a fifth wheel hitch, a gooseneck hitch, a pivot hitch, a cushioned hitch, a weight distribution hitch, etc. In several aspects, the hitch 16 acts as a pivot. When attempting to hitch a trailer to the vehicle, it is desirable to manage the vehicle 12 so that when the vehicle is reversed and backed toward the trailer 14, the vehicle 12 does not come into physical contact with the trailer or obstacles in the vehicle 12 and the trailer's surroundings 18.

[0033] The system 10 includes one or more control modules 20. The control module 20 is a non-universal electronic control device having a pre-programmed digital computer or processor 22, a non-transitory computer-readable medium or memory 24 for storing data such as control logic, software applications, instructions, computer code, data, lookup tables, etc., and one or more input / output (I / O) ports 26. Computer-readable media include any type of media that can be accessed by a computer, such as read-only memory (ROM), random access memory (RAM), hard drive, compact disk (CD), digital video disk (DVD), or any other type of memory. Non-transitory computer-readable media or memory 24 include media that can store data permanently and media that can store data and rewrite it later, such as rewritable optical disks or erasable storage devices. Computer code includes any type of program code, including source code, object code, and executable code. The processor 22 is configured to execute code or instructions. In some examples, the control module 20 can be a dedicated Wi-Fi controller or an engine control module, a transmission control module, a body control module, an infotainment control module, etc. The I / O port 26 is configured to communicate via a wired or wireless connection using Wi-Fi, Bluetooth communication protocols, radio frequency (RF) protocols, etc. under IEEE802.11x. In some examples, the control module 20 also includes one or more applications 28. Application 28 is a software program configured to perform a specific function or a set of functions. Application 28 may include one or more computer programs, software components, instruction sets, programs, functions, objects, classes, instances, related data, or parts thereof suitable for implementation in suitable computer-readable program codes. Application 28 may be stored in memory 24 or in an additional or separate memory. Examples of application 28 include audio or video streaming services, games, browsers, social media, suspension and engine control programs, body control programs, advanced driver assistance systems (ADAS) programs, etc. In the specific application 28 of the present disclosure, the system 10 includes a trailer coupling labeling application (TCLA) 30.

[0034] The system 10 also includes one or more sensors 32. The sensors 32 generate real-time positioning information about the position and / or movement of the vehicle 12. The sensors 32 may include any of a variety of different types of sensors 32, including, but not limited to: a camera 34, a light detection and ranging (LiDAR) sensor, a radio detection and ranging (RADAR) sensor, a sound navigation and ranging (SONAR) sensor, an ultrasonic sensor, or a combination thereof. In addition, the sensors 32 may also communicate with a global positioning system (GPS), particularly with image data collected by satellites 33 in orbit around the earth, so as to more accurately and precisely report the position of the vehicle 12. In a further example, the sensors 32 may include wheel speed sensors disposed on one or more of the vehicle 12 and the trailer 14, a hitch angle estimation sensor 36, or a hitch angle estimation algorithm, as taught in U.S. Patents 10,346,705 and 11,560,026.

[0035] In other examples, the sensor 32 may include an inertial measurement unit (IMU) 38. The IMU 38 uses a combination of some or all of an accelerometer, a gyroscope, and a magnetometer to measure and report the attitude or position, linear velocity, acceleration, and angular rate relative to a global reference frame. In some examples, the IMU 38 may also utilize global positioning system (GPS) data to indirectly measure attitude or position, velocity, acceleration, and angular rate. When used in the system 10 of the present disclosure, the IMU 38 measures and reports the attitude or position, linear velocity, acceleration, and angular rate of the vehicle 12 and the camera 34. The control module 20 and the sensor 32 may be connected to and communicate on one or more communication networks 40, including one or more of a vehicle-to-vehicle (V2V) communication network, a vehicle-to-infrastructure (V2I) communication network 44, and the like. For simplicity, the term "infrastructure" is used to collectively refer to network entities capable of self-organizing wireless communications in direct links or side links, as well as cellular communication networks owned by service providers, so vehicle-to-network (V2N) communications are also included in the scope of V2I.

[0036] Now refer to Figure 2 And continue to refer to Figure 1, the system 10 also includes an onboard human machine interface (HMI) 46 for interacting with an operator of the vehicle 12. In several aspects, the HMI 46 includes one or more devices capable of interacting with the operator of the vehicle 12, such as a screen disposed within the vehicle 12, such as an instrument panel 48, an infotainment screen 50, a head-up display (HUD) 52, an interior rear view screen, such as a rear view mirror enhanced by a screen 53, etc. However, it should be understood that other HMIs 46 are also contemplated herein. For example, the HMI 46 can be a mobile device, such as a tablet computer, a mobile phone, etc., and the HMI 46 can be provided by an operator of the vehicle 12 and temporarily installed on or disposed on an interior passenger compartment component of the vehicle 12. In several aspects, the HMI 46 communicates with the control module 20 via the I / O port 26, and sends information to and receives information from the I / O port 26 of the control module 20, including during operation of the TCLA 30.

[0037] For example, the TCLA 30, in conjunction with other vehicle control applications, can assist the operator of the vehicle 12 in maneuvering the vehicle 12 by turning the steering wheel 62 to reverse toward the trailer 14 while connecting, hooking, or coupling the trailer 14 to the vehicle 12. More specifically, the TCLA 30 includes a plurality of subroutines or control logic portions that are stored in the memory 24 of the control module 20 and executed by the processor 22 while receiving data from sensors 32 such as cameras 34, hitch angle estimation sensors 36, IMUs 38, and / or satellites 33 reporting GPS data through the I / O port 26. The TCLA 30 also includes a plurality of subroutines or control logic portions that send data from the control module 20 to the HMI 46.

[0038] Now refer to Figure 3 , a flow chart of a method 100 of a TCLA 30 is shown according to the present disclosure. In several aspects, the TCLA 30 is initialized at box 102 when one or more specific conditions occur within the first control logic. These conditions include manual initialization by an operator of the vehicle 12 through the HMI 46 or operation of a physical button disposed inside the vehicle 12. In a further example, these conditions may be automatically satisfied when the sensor 32 detects that the trailer 14 is connected to the vehicle 12.

[0039] When confirmation is received at block 102 that the vehicle 12 operator desires to engage the TCLA 30, the control module 20 executes control logic to present a prompt to the vehicle 12 operator using the HMI 46 to confirm that the vehicle 12 operator desires to engage the TCLA 30. The prompt may be displayed on a screen of the HMI 46, sounded through a stereo system of the vehicle 12, etc.

[0040] Continue to refer Figure 3 Now refer to Figure 4 , TCLA 30 uses a visual simultaneous localization and mapping (vSLAM) algorithm to estimate the motion of the sensor (i.e., camera 34) and reconstruct structures in environment 18. More specifically, in block 104, the vSLAM algorithm uses visual or optical data captured by camera 34 within the second control logic. For example, Figure 4 As shown, as the vehicle 12 and camera 34 move over time t1, t2, t3, the camera's poses 72, 74, 76 are captured. The vSLAM algorithm provides estimates of the position and orientation of the camera on the vehicle 12 relative to the environment 18, while mapping the environment. The camera 34 captures images of the environment 18 so that the vSLAM algorithm can extract features such as the trailer coupler 17 and the trailer hook 16. For example, the features of the trailer coupler 17 are matched on consecutive frames to estimate the movement of the camera 34 and the 3D position of the trailer coupler 17 and / or other features in the environment 18. The camera poses 72, 74, 76 and 3D features captured by the camera 34 form a map of the environment 18, which is updated when the system 10 captures and processes new images. The vSLAM algorithm also performs loop closure detection to identify that the camera 34 is returning to a previously captured environment 18 and corrects drift and camera poses in the map.

[0041] At box 106, the vSLAM algorithm performs camera pose calculations within the third control logic using the optical data received at box 104. At time t1, the vehicle 12 is hooked to the trailer 14 (hooked state). At box 108, in the hooked state, when both the distance "a" and the distance "b" are known, the position of the hook 16 relative to the camera 34, that is, the distance between the camera and "p1", can be calculated. Distance "a" is the vertical height of the camera 34 in the vehicle, and distance "b" is the horizontal distance from the camera to the hook 16. Because the vehicle 12 and the trailer 14 are in the hooked state, distance "b" is also the position of the trailer coupler 17 relative to the camera 34 (i.e., p1).

[0042] At box 110, the vehicle 12 is positioned in the X, Y, Z coordinate system with the sensor 32 by measuring the vehicle dynamics within the fourth control logic as the vehicle 12 moves over time (α=pitch, β=roll, γ=yaw and its angular rotation). In the fifth control logic, the vSLAM algorithm can be used to locate the trailer coupler 17 in each camera pose 72, 74, 76 at each time t1, t2 and t3, thereby projecting the coupler's 3D position in each image of each camera pose 72, 74, 76 to the 2D image, as shown in FIG. Figure 4As shown. The environmental scan data received from the sensor 32 is then processed in one or more image or video processing algorithms included in the fifth control logic. In several aspects, the image or video processing algorithms may include one or more image capture algorithms that capture images of the trailer 14 and trailer coupler 17 and perform image distortion correction as needed to correct for the lens shape of the camera 34, etc. Feature detection within the images of the trailer 14 and trailer coupler 17 may then be performed using a feature detection algorithm such as a Canny edge algorithm.

[0043] At block 112 and within the sixth control logic, the TCLA 30 creates a bounding box or marker 80 around the trailer coupler 17 at the re-projected position or location in the 2D images 72, 74, and 76 defined by the captured data (received at block 104), such as Figure 4 The marker 80 locates the trailer coupler 17 in the 2D image under different kinds of conditions including, for example, low lighting conditions.

[0044] In some aspects, at block 114, a remote cloud-based computing and / or storage system (e.g., a cloud computing server 60 ( Figure 1 )) to perform machine learning (ML) training to improve the object detection accuracy of the trailer coupler 17 to the TCLA 30. At box 116, the retrained ML algorithm is deployed to various vehicles to enhance the identification and labeling of the trailer coupler 17, as will be described in further detail below.

[0045] Now refer to Figure 5 , which is a block diagram of a process 200 for enhancing a local machine learning model used by a TCLA 30 in a vehicle 12 within a seventh control logic according to the present disclosure. The process 200 includes a vehicle-side sub-process 202 and a cloud-side sub-process 204. The vehicle-side sub-process 202 includes a local federated machine learning model 206, which is a machine learning model (i.e., a deep neural network (DNN)) resident in the memory 24 of the system 10 and used by the TCLA 30. The local federated machine learning model 206 uses raw data collected locally on the vehicle 12 (e.g., images of the trailer coupler 17 and the label 80 of the coupler 17) to perform machine learning training to enable the TCLA 30 to recognize trailer couplers of different types and under different conditions.

[0046] In a non-limiting example, the local federated machine learning model 206 includes a plurality of layers, including an input layer and an output layer, and one or more hidden layers. The input layer receives an optical scan of the trailer coupler 17 as input. The optical scan is received from the vehicle 12 at different times and positions of the vehicle 12 relative to the trailer 14. In addition, the optical scan may include images of different types of trailer hooks 16 and trailer couplers 17. These inputs are then passed to the hidden layers. Each hidden layer applies a transformation (e.g., a nonlinear transformation) to the data and passes the result to the next hidden layer until the last hidden layer. The output layer produces the final predicted position of the trailer coupler 17 within the image. To train the ML algorithm, an input data set and its corresponding position of the trailer coupler 17 are used. The algorithm is trained by adjusting the internal weights between the nodes in each hidden layer to minimize the prediction error. During the training process, an optimization technique (e.g., gradient descent) is used to adjust the internal weights to reduce the prediction error. The training process is repeated for the entire data set until the prediction error is minimized, and then the resulting training model is used to classify new input data. After the ML algorithm is sufficiently trained, the federated learning model 206 is able to accurately and precisely determine the trailer coupler 17 within an image based on images captured of the vehicle 12 at different times relative to the trailer 14 in an attached and unattached state. By adjusting the weights between nodes in each hidden layer during the training process, the algorithm "learns" to recognize patterns in the data that indicate a trailer coupler 17 in images in different scenarios. Advantageously, once training is complete, the vehicle-side sub-process 202 simply uploads the model parameters 220 of the locally trained federated machine learning model 206 to the cloud computing server 60 within the eighth control logic.

[0047] In addition, the vehicle-side sub-process 202 includes user-provided information 208, vehicle-specific information 210, and contextual feature extraction 212. The user-provided information 208 may include, but is not limited to, the type of hitch 16 or ball that the user or vehicle operator is using (i.e., a prong hitch, ball size 2 inches, 2.5 / 8 inches, etc.), the type of trailer 14, the type of trailer coupler 17, and the modifications to the user's vehicle 12. The vehicle-specific information 210 may include, but is not limited to, the year, make, and model of the vehicle 12, trim packages on the vehicle 12, other vehicle optional equipment, etc. The contextual feature extraction 212 utilizes the sensors 32 to capture and extract information about the environment 18 from the camera poses 72, 74, and 76, including, but not limited to, lighting conditions.

[0048] The cloud sub-process 204 includes a federated aggregation model 214, a global learning model database 216, and a context-aware clustering algorithm 218. The federated aggregation model 214 receives the local model parameters 220 of the local federated learning model 206. The federated aggregation model 214 uses the model parameters 220 received from the local federated machine learning model 206 to build a plurality of machine learning models, and stores the plurality of machine learning models in the global model database 216 through the ninth control logic. The global model database 216 contains global machine learning models that are separated, stored, and classified based on the input received by the context-aware clustering algorithm 218. The context-aware clustering algorithm 218 receives the information 208 provided by the user, the vehicle-specific information 210, and the context features provided by the context feature extraction 212 from the vehicle-side sub-process 202 to group or cluster the machine learning models stored in the global model database 216. Therefore, a plurality of global machine learning models are created, stored, and grouped according to the context features through the tenth control logic.

[0049] In some aspects of the present disclosure, the model parameters 220 of the local joint machine learning model 206 are transmitted to the cloud sub-process 204 for aggregation. The joint aggregation process 214 uses the context feature information to search the global database 216 to perform global aggregation of the machine learning model so that the machine learning model is grouped with other machine learning models developed based on the same or similar context features. If there is no machine learning model based on the same or similar context features, a new machine learning model is created by the cloud sub-process 204 and then stored in the global model database 216 using the eleventh control logic. In several aspects of the present disclosure, the machine learning model stored in the global model database 216 is searched according to the context features, and it is only deployed to vehicles with the same or similar context features. Therefore, the process 200 deploys different machine learning models developed in the cloud computing server 60 to vehicles with matching context features to prevent the locally stored machine learning model residing in the vehicle 12 from being damaged by the twelfth control logic.

[0050] The disclosed system 10 and method 200 for providing locating and marking a trailer coupler 17 has several advantages over prior art systems. These advantages include: improving the accuracy of identifying the trailer coupler 17 between different locations while utilizing pre-existing hardware, reducing computational burden, and reducing component complexity while reducing trailer hitching difficulties, thereby increasing customer confidence and satisfaction.

[0051] The description of the present disclosure is merely exemplary in nature, and variations that do not depart from the gist of the present disclosure are intended to be within the scope of the present disclosure. Such variations should not be regarded as departing from the scope and scope of the present disclosure.

Claims

1. A system for positioning a trailer coupler on a trailer to assist in coupling the trailer coupler to a vehicle hitch on a vehicle, the system comprising: a human-machine interface (HMI), disposed within the vehicle; A camera, disposed on the vehicle and detecting the surrounding environment of the vehicle; a global positioning system (GPS), including one or more GPS satellites for obtaining vehicle location information; Remote computing systems; Vehicle-to-Infrastructure (V2I) communication networks; as well as one or more controllers, each of the one or more controllers having a processor, a memory, and one or more input / output (I / O) ports, the I / O ports communicating with one or more sensors, the one or more HMIs, the GPS, and the remote computing system via the V2I communication network; the memory storing program control logic; the processor executing the program control logic; The program control logic includes a trailer coupling labeling application (TCLA), the TCLA includes program control logic, the program control logic includes: a first control logic for sensing at least one of a hooked state and an unhooked state of the trailer coupler hooked to the vehicle; a second control logic for capturing optical data using the camera when the vehicle hook is in the hooked state and the unhooked state; a third control logic for determining a distance between the vehicle hook and the trailer coupler relative to the camera when the vehicle hook is in the hooked state; fourth control logic for sensing movement of the vehicle relative to the trailer when the vehicle hitch is in the unhitched state; fifth control logic for determining a position of the trailer coupler when the vehicle hitch is in the unhitched state based at least in part on distances of the vehicle hitch and the trailer coupler relative to the camera and sensed movement of the vehicle relative to the trailer when the vehicle hitch is in the hooked state; and A sixth control logic identifies a location of the trailer coupler in the captured optical data and creates a mark on the trailer coupler in an image defined by the captured optical data.

2. The system according to claim 1, wherein: The first control logic further includes: control logic to request an operator to provide input to the HMI; and Control logic is configured to receive the operator input and determine that the vehicle hitch is in at least one of an engaged state and an unengaged state.

3. The system according to claim 1, wherein: The second control logic further includes: Control logic for storing the optical data as a two-dimensional image of the vehicle hitch in the attached state and the unattached state.

4. The system according to claim 1, wherein: The third control logic further includes: Control logic performs feature detection and size calculation on an image defined by the captured optical data to determine a position of the vehicle hitch in the image when the vehicle hitch is in an engaged state.

5. The system according to claim 1, wherein: The third control logic further includes: Control logic performs feature detection and size calculations on an image defined by the captured optical data to determine a location of the trailer coupler in the image when the vehicle hitch is in an unattached state.

6. The system according to claim 5, wherein: The sixth control logic further includes: Control logic creates the marking on the trailer coupler in an image defined by the captured optical data at different points in time when the vehicle hitch is in the unattached state.

7. The system of claim 1 further comprising seventh control logic for training a local machine learning algorithm to identify the trailer coupler based on the optical data captured by the camera at different times.

8. The system of claim 7, further comprising an eighth control logic for sending a set of model parameters of the local machine learning algorithm to a remote computer system.

9. The system of claim 8, further comprising ninth control logic for creating one of a plurality of global machine learning algorithms on the remote computer system based on the model parameters received by the local machine learning algorithm.

10. The system of claim 9, further comprising tenth control logic for grouping each of the plurality of global machine learning algorithms into categories based on at least one of vehicle operator provided information, vehicle information, and contextual information.

Images