Smart device for estimating trailer characteristics

Abstract

The present disclosure provides a smart device for estimating trailer characteristics. A trailer measurement system includes a processor that receives a first set of image data obtained from a first imager included in a portable electronic device external to a vehicle, identifies a vehicle and a trailer coupled to the vehicle in the first set of image data, and identifying a reference feature having at least one known size in the first image data set. Then, the processor derives a scaling factor for the first image data set by comparing a size of the reference feature with respect to the first image data set with the at least one known size, and deriving at least one trailer feature size from the trailer identified in the first image data set using the scaling factor.

Description

TECHNICAL FIELD

[0001] The present disclosure relates generally to a system for measuring various trailer dimensions. More specifically, a trailer measurement system uses a smart device to provide various data for determining desired trailer measurement values. BACKGROUND

[0002] There are many so-called advanced driver assistance systems (“ADAS”) that are intended to make it easier for customers to tow a trailer. These systems often require various measurements to be taken of the trailer to best perform and / or utilize all available features. For example, in a trailer backup assist (“TBA”) system, it is necessary to know the location of the rear axle of the trailer relative to the hitch point. For systems that include trailer towing functionality, such as a blind spot information system (“BLIS”), the length and width of the trailer become important characteristics. For lane offset, the width of the trailer is important. In the past, there have been two main methods to obtain these measurements. The first method is to ask the driver to measure the trailer with a tape measure. This is not always easy to do accurately, given the height and unusual dimensions of some trailers. Manually measuring the dimensions of a trailer can be time consuming and cumbersome, and writing down the measurements and then transferring them to the vehicle is a tedious process. Another method for measuring the dimensions of a trailer is to measure key distances from on-board sensors that will automate the process. This can be a convenient way to automate the information gathering for the customer. SUMMARY

[0003] According to one aspect of the present disclosure, a trailer measurement system includes a processor that receives a first image data set obtained from a first imager included in a portable electronic device that is external to a vehicle, identifies a vehicle and a trailer coupled with the vehicle in the first image data set, and identifies a reference feature having at least one known dimension in the first image data set. The processor then derives a scaling factor for the first image data set by comparing a size of the reference feature relative to the first image data set to the at least one known dimension, and derives at least one trailer feature dimension from the trailer identified in the first image data set using the scaling factor.

[0004] Embodiments of the first aspect of the application can include any one or combination of the following features:

[0005] - the reference feature can be one of a bumper height, a wheel size, or a side panel length.

[0006] - the at least one trailer feature dimension can be one of a distance between a hitch point of the trailer and an axle of the trailer, a length of the trailer, or a width of the trailer.

[0007] - the portable electronic device can be a mobile phone, the first imager can comprise a camera included in the mobile phone, and the image data set can comprise visual image data received from the camera.

[0008] - the image data set can comprise the visual image data in the form of a plurality of images, the plurality of images comprising the vehicle and trailer from a corresponding plurality of positions around the vehicle and trailer, and prior to identifying the reference feature, the processor can use a photogrammetry process to construct the first image data set into a three-dimensional model of the vehicle and trailer.

[0009] - the plurality of images can be photographs received collectively as the first data set.

[0010] - the plurality of images can be selected images from still frame images of a video received as the first data set.

[0011] - the trailer measurement system can further comprise a lidar sensor, the processor can receive three-dimensional point location data from the lidar sensor and use the three-dimensional point location data in conjunction with the first image data set identifying the reference feature, derive the scaling factor and derive the at least one trailer feature dimension.

[0012] - the trailer measurement system can further comprise a mobile processor running a program, the program transmitting the first image data set from a memory of the portable electronic device to the first processor.

[0013] - the program can direct a user through a series of steps to capture the plurality of images using the camera of the portable electronic device.

[0014] - the program can further cause the portable electronic device to obtain the three-dimensional point location data related to the first image data set from the lidar sensor included in the portable electronic device, and transmit the first data set and the three-dimensional point location data from a memory of the portable electronic device to the first processor.

[0015] According to another aspect of this disclosure, a trailer measurement system includes a first processor that receives a plurality of images as a first visual image dataset, the plurality of images including the vehicle and trailer from corresponding plurality of locations around the vehicle and trailer, the plurality of images being acquired from a camera in a portable electronic device included outside the vehicle. Using a photogrammetric process, the processor constructs a three-dimensional model of the vehicle and trailer from the first image dataset. The processor identifies the vehicle and the trailer attached to the vehicle in the three-dimensional model of the vehicle and trailer, identifies a reference feature having at least one known size in the three-dimensional model of the vehicle and trailer, and derives at least one trailer feature size of the trailer identified in the first image dataset, including scaling the first image dataset based on the known size of the reference feature using the three-dimensional model of the vehicle and trailer, the at least one trailer feature size being measured in at least the scaled first image dataset.

[0016] According to another aspect of this disclosure, a trailer measurement system includes a processor that receives a first image dataset from a first imager included in a portable electronic device external to the vehicle, and receives three-dimensional point position data from a lidar sensor included in the portable electronic device. The processor also identifies the vehicle and a trailer attached to the vehicle in the first image dataset, identifies a reference feature having at least one known size in the first image dataset, and derives at least one trailer feature size from the trailer identified in the first image dataset, including scaling the first image dataset based on the known size of the reference feature using the three-dimensional point position data, wherein the at least one trailer feature size is measured in at least the scaled first image dataset.

[0017] Those skilled in the art will understand and appreciate these and other aspects, objectives, and features of this disclosure upon studying the following specification, claims, and drawings. Attached Figure Description

[0018] In the attached diagram:

[0019] Figure 1 It is a perspective view of an exemplary trailer having various dimensions for measurement using a system according to this disclosure;

[0020] Figure 2 It is shown in conjunction with the vehicle. Figure 1 A side view of an exemplary trailer, the vehicle including reference features for use by this system;

[0021] Figure 3 This is a perspective view showing a smartphone used in conjunction with this system for measuring various trailer dimensions;

[0022] Figure 4 It is a schematic diagram of the system according to this disclosure; and

[0023] Figure 5 This is a flowchart illustrating the steps in a method for measuring trailer dimensions using the disclosed system. Detailed Implementation

[0024] For the purposes of this description, the terms “up,” “down,” “right,” “left,” “back,” “front,” “vertical,” “horizontal,” “inner,” “outer,” and their derivatives should be used as follows: Figure 1 The device is associated with the orientation specified therein. However, it should be understood that the device may take various alternative orientations unless explicitly stated otherwise. It should also be understood that the specific devices and processes shown in the drawings and described in the following description are merely exemplary embodiments of the inventive concept defined in the appended claims. Therefore, unless otherwise expressly stated in the claims, the specific dimensions and other physical characteristics relating to the embodiments disclosed herein should not be considered limiting. Furthermore, unless otherwise specified, it should be understood that the discussion of specific features of a component extending in or along a given direction does not imply that the feature or component follows a straight line or axis in this direction, or extends only in this direction or in this plane without other directional components or deviations, unless otherwise specified.

[0025] Ordinal modifiers (i.e., "first," "second," etc.) can be used to distinguish the various structures of the disclosed transport frame in various contexts; however, such ordinal numbers are not necessarily intended to apply to elements outside the specific context in which such elements are used, and in various respects, different elements within the same class can be identified by the same context-specific ordinal number. In such cases, other specific names of the elements are used to clarify the overall relationship between such elements. Ordinal numbers are not used to specify the position of an element, nor do they exclude additional or intermediate unordered elements, or indicate the importance or ranking of an element within a particular category.

[0026] The terms “comprising,” “including,” or any other variation thereof are intended to cover non-exclusive inclusion, such that a process, method, article of manufacture, or apparatus that comprises a list of elements includes not only those elements but may also include other elements not expressly listed or inherent to such process, method, article of manufacture, or apparatus. Unless otherwise specified, an element preceded by “comprising…” does not exclude the presence of additional identical elements in the process, method, article of manufacture, or apparatus that includes said element.

[0027] For the purposes of this disclosure, the term "connection" (in all its forms: link, linked, connected, etc.) generally means that two components (electrical or mechanical) are directly or indirectly connected to each other. Such a connection may be fixed in nature or movable in nature. Such a connection may be achieved using two components (electrical or mechanical), and any additional intermediate component may form a single unit with or between the two components. Unless otherwise stated, such a connection may be permanent in nature, or may be removable or detachable in nature.

[0028] For the purposes of this disclosure, the terms “about,” “approximately,” or “substantially” are intended to mean that the value of a parameter is close to the stated value or location. However, small differences can prevent the value or location from being exactly the same as stated. Therefore, unless otherwise stated, for a given value, a difference of up to ten percent (10%) is a reasonable difference from an ideal target exactly as described. In many cases, a difference greater than ten percent (10%) may be a significant difference unless otherwise understood by one of ordinary skill in the art based on the context in which the term is used.

[0029] refer to Figure 1 Reference numeral 10 generally denotes a trailer measurement system. The trailer measurement system 10 includes a processor 12 that receives a first image dataset 14 obtained from a first imager 16 included in a portable electronic device 18 external to a vehicle 20, identifies the vehicle 20 and the trailer 22 attached to the vehicle 20 in the first image dataset 14, and identifies trailers with at least one known dimension D in the first image dataset 14. k Reference feature 24. Then, the processor 12 modifies the size of the reference feature 24 relative to the first image dataset 14 with the at least one known size D. k The scaling factor of the first image dataset 14 is derived by comparison, and at least one trailer feature size D is derived from the trailer 22 identified in the first image dataset 14 using the scaling factor. t .

[0030] As mentioned above, there are many so-called Advanced Driver Assistance Systems (“ADAS”) designed to make trailer towing easier for customers. These systems typically require various measurements of the trailer to optimally perform and / or utilize all available features. For example, in a Trailer Back-Up Assist (“TBA”) system, it is necessary to know the position of the trailer’s rear axle relative to the attachment point. For systems that include trailer towing capabilities, such as Blind Spot Information Systems (“BLIS”), the length of the trailer becomes an important characteristic. For lane offset, the width of the trailer is important. Historically, there have been two main methods for obtaining these measurements. This system utilizes increasingly sophisticated imagers and sensors in smartphones to accurately collect the required dimensional information, thereby reducing the user’s workload or chance of error. It is noteworthy that users rarely lack or have access to smartphones or other smart devices with advanced mobile processors that are used to run applications of this type and are configured to perform the functions discussed herein, as well as various sensors available for collecting information for vehicle and trailer measurements with acceptable accuracy. In fact, many car owners already utilize smartphone apps that can communicate with their vehicles via Bluetooth, mobile internet, or direct Wi-Fi connections to control various vehicle functions (door locks, remote start, etc.) and assist with various vehicle ownership and management tasks. An example of such an app is available from Ford Motor Company in Dearborn, Michigan. Application. In one example, the system 10 may utilize a modified or enhanced version of such an application. In this regard, the processor 12 may be configured to communicate with the application 26 on the portable electronic device 18, and may include programming to interoperate with the application 26 on the portable electronic device 18, as discussed further below.

[0031] refer to Figure 2The system 10 can use the known dimensions of a feature of the vehicle 20 (i.e., reference feature 24) to determine the proportions of various trailer 22 features of interest to the system 10 in the measurement image dataset 14. In various implementations, the reference feature 24 may be one of the height 28 of the bumper 30, the diameter 32 of one of the wheel 34 sizes of the vehicle 20, or the length 36 of a vehicle panel (such as a side panel 38). Other easily identifiable features with known lengths may be used. Additionally, multiple features of such a feature may be used, where the resulting scaling factors are averaged or checked. Furthermore, different reference features 24 may be used with different vehicles, depending on the geometry of such features in their specific implementation on the vehicle 20, or a particular feature 24 that is best identified in the first image dataset 14 (e.g., given lighting or weather conditions, viewing angle, etc.) may be selected in real time by the processor 12. Alternatively, a measuring instrument in the form of an elongated article (e.g., a plastic block, rod, etc.) that can be marked or printed in a manner recognizable by the system 10 may be placed near the trailer 22 to serve as a reference feature 24.

[0032] Once reference feature 24 has been identified (optionally included after selection), processor 12 determines the size of reference feature 24 in the first image dataset 14, which can be measured, for example, in the number of pixels from one specified point or edge on reference feature 24 to another specified point. The size of reference feature 24 in the image is then compared to the known size of reference feature 24 relative to the actual vehicle 20 to determine a scaling factor of the first image dataset 14 relative to the vehicle and / or real-world scene depicted by the first image data set 14. In one example, the scaling factor can be expressed as the number of image pixels per real-world inch (or centimeter, millimeter, etc.). This scaling factor can then be used to determine the relevant dimensions of trailer 22 by determining the pixel-based dimensions of such features and then using the scaling factor to convert them to real-world dimensions. In another aspect, a trailer database may be available to system 10 (e.g., in the cloud and accessible via an internet connection or stored in memory associated with the smartphone 18 of vehicle 20), which may include multiple trailer images and associated measurements. System 10 can use image recognition to match the identified trailer 22 in image data 14 with trailer images in a database and associate relevant measurements with trailer 22. In this regard, for example, a scaling factor can be used as additional information to perform or confirm database identification. Image recognition can be based on trailer size or by reading an indication of trailer type from the sidewall of trailer 22. Various types of image processing techniques can be used to identify reference feature 24, as well as the measurement features discussed below, and both the vehicle and trailer. Such image processing techniques can be included in the programming of processor 12, in a specific program or application performing the current measurement process, and stored in memory 40 associated with or otherwise accessible to processor 12, and can include edge detection, corner detection, texture analysis, feature extraction, or various combinations thereof. Such analysis can also be performed or enhanced by various machine learning techniques, including those utilizing various neural networks and / or computer vision processes, which can be included in programming or accessible by processor 12, for example, via the Internet.

[0033] Using the techniques described herein, system 10 can obtain the following measurements:

[0034] - The distance 42 between the rear axle 44 (or the center of multiple rear axles) of trailer 22 and the attachment point 46 between trailer 22 and vehicle 20;

[0035] -The height of the tongue bar 52 of trailer 22 is 48;

[0036] -The total width of trailer 22 is 54 and the length is 55;

[0037] -The total height of trailer 22 is 56, including any load extending above the top of trailer 22;

[0038] - The length of the pull rod 60 is 58;

[0039] - The total height of vehicle 20 is 62, including any additional height from aftermarket fixtures (such as lights, luggage racks, etc.); and

[0040] - The size (width and / or height) of the taillights 63 or the rear window opening 65 of vehicle 20; and

[0041] - The distance 67 from the top of the wheel well 69 to the top 71 of the vehicle 20 or the side of the truck bed 73.

[0042] It should be understood that the above list is provided by way of example and additional or alternative measurements may be made depending on the use of such measurements and the overall capabilities of vehicle 20. In one aspect, the actual dimensions of the aforementioned features may be associated with the vehicle identification number (“VIN”) of vehicle 20, which can be read in image data 14 when smartphone 18 is properly positioned.

[0043] like Figure 3As shown and as described above, the portable electronic device 18 can be a mobile phone. In this implementation, the imager 16 mentioned above can be a camera (also referred to by reference numeral 16) included in the mobile phone (also referred to by reference numeral 18). In conjunction with the use of the camera 16, the first image dataset 14 can include visual image data received from the camera 16, including visual image data received by a specific sensor used in conjunction with it. Additionally, most smartphones (including those conceived to be compatible with this system 10) can record and associate additional data about the images. This additional information can be included with the first image dataset 14 as “metadata” and can include information about the type of lens and / or sensor used by the camera 16 when recording the image dataset 14, the aperture (if adjustable), and the focal length, as well as location information (provided by a positioning device and / or related software or programming within the smartphone 18). This metadata can be transmitted from the smartphone 18 (including by the wireless communication module 64) to the processor 12 as part of the first image dataset 14. In this way, the characteristics of the camera 16 can be taken into account when performing the above-described image-based measurements for accuracy. In one aspect, this can be accomplished by taking into account the characteristics of camera 16 as an additional step or function of the algorithm using the scaling factor described above. In another aspect, vehicle control module 72 can be configured to allow smartphone 18 to operate via application 26 as a key to unlock and / or start vehicle 20. In some implementations of such functionality, the vehicle may include multiple ultra-wideband (“UWB”) anchors that communicate with smartphone 18 to determine the location of smartphone 18 within and around the vehicle for a general purpose, namely, to establish a minimum distance from vehicle 20 for unlocking vehicle 20 or for automatically unlocking or locking the vehicle when the driver approaches or leaves vehicle 20. These features and the resulting functionality can be used to correlate specific locations of any images captured around vehicle 20 to improve the accuracy of measurements obtained using the image data set 14. The distance from smartphone 18 to vehicle 20 can be used to determine the distance to trailer 22 for use with image processing and other methods to determine trailer characteristics, as discussed herein.

[0044] In a particular implementation of system 10 described herein, the first image dataset 14 may include visual image data from camera 16 in the form of multiple images, including images of vehicle 20 and trailer 22 from corresponding multiple locations around vehicle 20 and trailer 22. Before identifying the reference feature 24, processor 12 may use a photogrammetric process to construct a three-dimensional model of vehicle 20 and trailer 22 from the first image dataset 14. The three-dimensional model can then be used for the measurement of reference feature 24 and subsequent measurements of various trailer 22 characteristics. The use of the three-dimensional model helps to account for perspective shortening and / or distortion that may occur in a single image, and to consider accuracy by adding information (i.e., additional images).

[0045] Photogrammetry is a technique for measuring objects using photographic images. The process involves capturing a series of images of an object from different points or angles, where these images share common features such that they partially “overlap.” In various examples, the images can include still photographs or still frames from video. These images are then processed using specialized software or larger software applications or programming within programs. The program uses the overlapping images to triangulate points and create surfaces. This allows for the creation of fairly accurate 2D or 3D models. In some respects, the accuracy of the collected data can be related to the quality of the images. Therefore, ensuring high resolution and proper overlap of the images can be beneficial. In one aspect, the software or programming used by the processor 12 in conjunction with the system 10 can guide the user through a photographic process designed to overlap various images. Additionally, such software or programming can confirm the appropriate image resolution before moving to subsequent image instructions. Points used for triangulation in photogrammetry are identified through a process called point matching. This process operates by first identifying common points in the overlapping images. These common points are called tie points, and they represent the same locations in adjacent images. Rays corresponding to these points are then defined. Specifically, each connection point defines a ray in 3D space that originates from camera 16 and extends to the real object. Triangulation of these points in space can be performed by defining multiple (e.g., three or more) rays associated with each connection point. As discussed above, the first image dataset 14 may include metadata associated with each individual image, where the software utilized takes into account characteristics of camera 16, such as focal length, pixel size, and lens distortion, to calibrate the geometric intersections of the rays, as conveyed in the metadata. A technique called bundle adjustment is used for triangulation, and the images can be adjusted simultaneously to create the intersections of all rays at each transit point and ground control point. This, in turn, solves for the unknowns consisting of X, Y, and Z object space coordinates. These coordinates are then used to construct the desired 3D model.

[0046] In an alternative implementation, an application 26 for collecting image data 14 on a smartphone 18 can perform a photogrammetric process, including by utilizing software (if present) within the operating system of the smartphone 18 with such capability. The first image dataset 14 can then be transmitted to the processor 12 in the form of a described 3D model. As mentioned, the plurality of images can be photographs received together as the first dataset 14, taken at various intermediate locations around the entire vehicle 20 and trailer 22. Alternatively, the plurality of images can be selected images from still frame images of a video received as the first dataset 14. It is understood that video is a series of frames or still images captured and played back at a specified frame rate. In this way, the mobile application 26 or the processor 12 can be programmed to use still frame images from video taken while the user walks around the vehicle 20 and trailer 22 (including under the guidance of the mobile application), or the processor 12 can extract still images at predetermined intervals (e.g., per second or approximately 60 frames), or can select images corresponding to desired viewpoints with desired overlap.

[0047] As described above, once the 3D model has been generated, the processor 12 derives at least one trailer feature size (including one or more of the specific features listed above) of the trailer 22 identified in the first image dataset 14. In one aspect, this involves scaling the first image dataset 14 based on the known size of the reference feature 24 using the 3D models of the vehicle 20 and the trailer 22. Subsequently, the desired trailer feature size is measured in at least the first image dataset 14 according to the derived scale.

[0048] Telephones are also increasingly likely to be equipped with light-detecting and ranging "LiDAR" sensors, which can be used to enhance the camera-based results discussed above and provide greater accuracy. Typically, LiDAR uses laser pulses (infrared light) to measure distances and create 3D models of objects and the environment. Unlike radar, which uses radio waves, LiDAR operates on a smaller scale and can provide accurate measurements over short distances. When a LiDAR sensor emits a laser, it reflects back from objects in the environment. By measuring the time it takes for these pulses to return, the sensor calculates the distance between the emitter and the object from which the light pulses reflect. Notably, smartphones incorporating augmented reality ("AR") experiences can use LiDAR data to enhance interactions with virtual objects. Specifically, smartphones can use LiDAR data to better understand their environment, with the aim of making AR interactions smoother and more accurate. In this way, smartphone 18 can create a field of points mapping distances and sizes in the environment. This field, or "point cloud," composed of 3D point location data 70, can be overlaid on a first image dataset 14 and can help identify key points in the first image dataset 14 and / or help measure desired features of reference feature 24 and trailer 22.

[0049] In one aspect, LiDAR data can be used in conjunction with the photogrammetric process discussed above to construct a more accurate 3D model of vehicle 20 and trailer 22. Smartphone application 26 can initially determine whether smartphone 18 includes LiDAR sensor 68 and can collect and transmit LiDAR data (if available). In one implementation, trailer measurement system 10 can include a LiDAR sensor that wirelessly communicates with smartphone 18 as described above, and processor 12 can receive 3D point location data 70 from LiDAR sensor 68 and use the 3D point location data 70 in conjunction with the first image dataset 14 identifying the reference feature 24 to derive the scaling factor and the at least one trailer feature size. Processor 12 can derive a desired trailer feature size from the trailer 22 identified in the first image dataset 14, including scaling the first image dataset 14 based on the known size of the reference feature 24 by incorporating the 3D point location data 70, the desired trailer feature size being measured in at least the scaled first image dataset 14.

[0050] Figure 4An example of the system 10, including various optional components, is shown. In one aspect, system 10 includes the processor 12 mentioned above, which may be included in a vehicle control module or controller 72. When included in such a controller 72, processor 12 may be a microprocessor that processes logic and programs stored in memory 74. In this respect, processor 12 may receive information from various sensors and vehicle systems, including a hitch angle detection system 76, a power steering control module 78, a vehicle braking control module 80, a powertrain control module 82, and other vehicle sensors and devices. In the example shown, vehicle 20 also includes a trailer assist system 84, which may be utilized by processor 12 to generate vehicle steering information and commands based on all or part of the information received from the sensors. Subsequently, vehicle steering information and commands can be provided to the power steering module 78 to influence the steering of the vehicle 20 to achieve a combined travel path for the vehicle 20 and trailer 22, as further discussed in 10,023,229, 9,714,051, and 9,840,278, the entire disclosure of which is incorporated herein by reference. It should be understood that the controller 72 may be a standalone dedicated controller or a shared controller integrated with other control functions, such as vehicle sensor systems (e.g., steering angle detection device 79), the power steering module 72, and other conceivable onboard or offboard vehicle control systems.

[0051] System 10 may also include a mobile processor 86 within smartphone 18, with processor 12 connected to the mobile processor via wireless communication module 64. As discussed above, mobile processor 86 may run a program or application 26 (which may be stored in memory 40 associated with smartphone 18) that transfers a first image dataset 14 from smartphone 18's memory 40 to processor 12. Once in vehicle 20, the first image dataset 14 can be used to obtain desired trailer measurements, which may be stored in vehicle memory 40 as a record 50 for a specific trailer 22, and then used for various ADAS features, including... Figure 4 The trailer reversing assist feature and other features shown in the system are illustrated above.

[0052] Turn Figure 5An example of a process for obtaining the trailer measurements discussed above is shown. Specifically, the process includes using a smartphone 18 running the aforementioned application 26, which can guide the user through certain steps of the process. Initially, process 110 can be initiated using smartphone 18 or an onboard human-machine interface (“HMI”). Then, in step 112, the user is guided via smartphone 18 or HMI to complete basic trailer information (trailer name, brake type, and optional brake gain). Once this information is entered, measurement process 114 is initiated. If the process is initiated by HMI (step 116), the user is prompted to launch mobile application 26 via a compatible connected smartphone 18 (step 118). Once application 26 is opened, measurement mode can be entered automatically or by the user upon processor instruction during communication establishment (step 120). Application 26 can then guide the user (step 122) to capture one or more images from a specified perspective, allowing system 10 to obtain the necessary information for the desired measurements, as discussed above. The guidance can be at least partially guided by a neural network to evaluate the perspective for feature sharpness and the angle, perspective, or alignment of vehicle 20 and trailer 22. In another variation, application 26 can list features for measurement, whereby a user taps features in an image via touchscreen 88 of smartphone 18, where the touch point is recorded and associated with the resulting image data 14 and / or LiDAR data 70 for use by processor 12 to identify or confirm the identification of a specific feature to be measured. As discussed above, in variations that use multiple images or videos to construct a 3D model, smartphone 18 can similarly guide the process using overlay maps and / or on-screen instructions. Specifically, in one implementation, application 26 can cause smartphone 18 to obtain 3D point location data 70 associated with an image including the first image dataset 14 from LiDAR sensor 68, and transfer the first image dataset 14 and the 3D point location data 70 from memory 74 of portable electronics 18 to first processor 12.

[0053] Then, application 26 can confirm that the necessary images and information have been obtained before transmitting the first image dataset 14 (step 124), and optionally, transmit the associated metadata and point location data 70 to processor 12 via wireless communication module 64 (step 126). Upon receiving this information, processor 12 can perform the desired measurements using at least the first image dataset 14, according to one or more of the specific processes discussed above.

[0054] It should be understood that changes and modifications may be made to the foregoing structures without departing from the concept of this disclosure, and it should also be understood that such concepts are intended to be covered by the following claims unless otherwise expressly stated in their language.

[0055] Equally important, it should be noted that the construction and arrangement of the elements of this disclosure as illustrated in the exemplary embodiments are merely illustrative. While only a few embodiments of the invention have been described in detail in this disclosure, those skilled in the art will readily understand that many modifications are possible (e.g., variations in the size, dimensions, structure, shape and proportions, parameter values, mounting arrangements, use of materials, color, orientation, etc.) without substantially departing from the novel teachings and advantages of the subject matter. For example, an element shown as integrally formed may be composed of multiple parts, or elements shown as multiple parts may be integrally formed; the operation of interfaces may be reversed or otherwise altered; the length or width of structures and / or components, or connectors or other elements of the system, may be changed; the nature or number of adjustment positions provided between elements may be altered. It should be noted that the elements and / or assemblies of the system may be constructed from any of a variety of materials providing sufficient strength or durability in any of a variety of colors, textures, and combinations. Therefore, all such modifications are intended to be included within the scope of this invention. Other substitutions, modifications, alterations, and omissions may be made in the design, operation, and arrangement of the desired embodiments and other exemplary embodiments without departing from the spirit of this invention.

[0056] It should be understood that any described process or step within a described process may be combined with other disclosed processes or steps to form a structure within the scope of this disclosure. The exemplary structures and processes disclosed herein are for illustrative purposes and should not be construed as limiting.

[0057] According to the present invention, a trailer measurement system is provided, comprising: a processor, the processor: receiving a first image dataset obtained from a first imager included in a portable electronic device external to a vehicle; identifying the vehicle and a trailer connected to the vehicle in the first image dataset; identifying a reference feature having at least one known size in the first image dataset; deriving a scaling factor of the first image dataset by comparing the size of the reference feature relative to the first image dataset with the at least one known size; and deriving at least one trailer feature size from the trailer identified in the first image dataset using the scaling factor.

[0058] According to an embodiment, the reference feature is one of the following: taillight width or height, rear window opening width or height, distance from the wheel well to the adjacent top portion of the vehicle, or side panel length.

[0059] According to an embodiment, the at least one trailer characteristic dimension is one of the distance between the trailer's attachment point and the trailer's axle, the length of the trailer, or the width of the trailer.

[0060] According to an embodiment: the portable electronic device is a mobile phone; the first imager includes a camera included in the mobile phone; and the first image dataset includes visual image data received from the camera.

[0061] According to an embodiment: the first image dataset includes the visual image data as a plurality of images, the plurality of images including the vehicle and trailer from corresponding plurality of locations around the vehicle and trailer; and before identifying the reference features, the processor uses a photogrammetric process to construct a three-dimensional model of the vehicle and trailer from the first image dataset.

[0062] According to an embodiment, the plurality of images are photographs that are collectively received as the first image dataset.

[0063] According to an embodiment, the plurality of images are selected images from still frame images of a video received as the first image dataset.

[0064] According to an embodiment, the present invention is further characterized by a lidar sensor, wherein: the processor receives a three-dimensional point position dataset from the lidar sensor and uses the three-dimensional point position data in conjunction with the first image dataset that identifies the reference feature, thereby deriving the scaling factor and deriving the at least one trailer feature size.

[0065] According to the present invention, a trailer measurement system is provided, the trailer measurement system comprising: a first processor, the first processor: receiving a plurality of images as a first visual image dataset, the plurality of images including the vehicle and trailer from corresponding plurality of locations around the vehicle and trailer, the plurality of images being acquired from a camera in a portable electronic device included outside the vehicle; using a photogrammetric process to construct a three-dimensional model of the vehicle and trailer from the first image dataset; identifying the vehicle and the trailer connected to the vehicle in the three-dimensional model of the vehicle and trailer; identifying a reference feature having at least one known size in the three-dimensional model of the vehicle and trailer; and deriving at least one trailer feature size of the trailer identified in the first image dataset, including scaling the first image dataset based on the known size of the reference feature using the three-dimensional model of the vehicle and trailer, the at least one trailer feature size being measured in at least the scaled first image dataset.

[0066] According to an embodiment, the plurality of images are photographs that are collectively received as the first visual image dataset.

[0067] According to an embodiment, the plurality of images are selected images from still frame images of a video received as the first visual image dataset.

[0068] According to an embodiment, the invention is further characterized by a mobile processor running a program that transfers the first image dataset from the memory of the portable electronic device to the first processor.

[0069] According to an embodiment, the program guides the user through a series of steps to capture the multiple images using the camera of the portable electronic device.

[0070] According to an embodiment, the reference feature is one of the following: bumper height, wheel size, or side panel length.

[0071] According to an embodiment, the at least one trailer characteristic dimension is one of the distance between the trailer's attachment point and the trailer's axle, the length of the trailer, or the width of the trailer.

[0072] According to the present invention, a trailer measurement system for use with a vehicle is provided, comprising: a first processor, the first processor: receiving a first image dataset obtained from a first imager included in a portable electronic device external to the vehicle; receiving three-dimensional point position data from a lidar sensor included in the portable electronic device; identifying the vehicle and the trailer connected to the vehicle in the first image dataset; identifying a reference feature having at least one known size in the first image dataset; and deriving at least one trailer feature size from the trailer identified in the first image dataset, including scaling the first image dataset based on the known size of the reference feature using the first image dataset in conjunction with the three-dimensional point position data, the at least one trailer feature size being measured in at least the scaled first image dataset.

[0073] According to an embodiment, the processor also uses three-dimensional point location data when measuring at least one trailer feature dimension.

[0074] According to an embodiment, the present invention is further characterized by a mobile processor running a program that: guides a user to capture a first image dataset using a first imager of a portable electronic device; causes the portable electronic device to obtain the three-dimensional point position data associated with the first image dataset from the lidar sensor included in the portable electronic device; and transfers the first image dataset and the three-dimensional point position data from the memory of the portable electronic device to the first processor.

[0075] According to an embodiment, the reference feature is one of the following: bumper height, wheel size, or side panel length.

[0076] According to an embodiment, the at least one trailer characteristic dimension is one of the distance between the trailer's attachment point and the trailer's axle, the length of the trailer, or the width of the trailer.

Claims

1. A trailer measurement system, comprising: processor: Receive a first image dataset obtained from a first imager included in a portable electronic device outside the vehicle; Identify the vehicle and the trailer connected to the vehicle in the first image dataset; as well as Identify reference features with at least one known size in the first image dataset; The scaling factor of the first image dataset is derived by comparing the size of the reference feature relative to the size of the first image dataset with the at least one known size; as well as At least one trailer feature size is derived from the trailer identified in the first image dataset using the scaling factor.

2. The trailer measurement system of claim 1, wherein the reference feature is one of the following: taillight width or height, rear window opening width or height, distance from the wheel well to an adjacent top portion of the vehicle, or side panel length.

3. The trailer measurement system as claimed in claim 1 or claim 2, wherein the at least one trailer characteristic dimension is one of the distance between the trailer's attachment point and the trailer's axle, the length of the trailer, or the width of the trailer.

4. The trailer measurement system as claimed in claim 1 or claim 2, wherein the portable electronic device is a mobile phone.

5. The trailer measurement system of claim 4, wherein the first imager includes a camera included in the mobile phone.

6. The trailer measurement system of claim 5, wherein the first image dataset includes visual image data received from the camera.

7. The trailer measurement system of claim 4, wherein the first image dataset includes the virtual image data as a plurality of images, the plurality of images including the vehicle and trailer from corresponding plurality of locations around the vehicle and the trailer.

8. The trailer measurement system of claim 7, wherein, prior to identifying the reference features, the processor uses a photogrammetric process to construct a three-dimensional model of the vehicle and trailer from the first image dataset.

9. The trailer measurement system of claim 7, wherein the plurality of images are photographs collectively received as the first image dataset.

10. The trailer measurement system of claim 7, wherein the plurality of images are selected images from still frame images of a video received as the first image dataset.

11. The trailer measurement system as claimed in claim 1 or claim 2, further comprising a lidar sensor, wherein: The processor receives a three-dimensional point location dataset from the lidar sensor and uses the three-dimensional point location data in conjunction with the first image dataset that identifies the reference features to derive the scaling factor and the size of the at least one trailer feature.

Images