System and method for calibrating image capture module
Through modular calibration target and image analysis functions, on-site calibration of rail vehicle machine vision system is realized, solving the problem of inefficient calibration in the prior art and improving the adaptability and safety of the system.
Patent Information
- Application Number
- CN202380029097.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2022-03-23
- Filing Date
- 2023-02-27
- Publication Date
- 2024-11-08
AI Technical Summary
Rail vehicle machine vision systems are prone to lose synchronization during transportation, resulting in unclear image capture. The existing calibration methods need to be carried out in a bench test environment, which is inefficient and cumbersome, and cannot be calibrated on site.
Modular calibration target and image analysis functions are adopted to capture target images through the camera, calculate the pixel size of the object, and automatically calibrate the rotary encoder to achieve on-site calibration and eliminate dependence on the bench test environment.
It improves the calibration efficiency of the rail vehicle machine vision system, ensures driving safety, reduces operator time waste and opportunity costs, and enhances the adaptability and performance of the system.
Smart Images

Figure CN118922861A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates generally to in-situ calibration of image capture modules, and more particularly to systems and methods for calibrating image capture modules using a calibration pole.
[0002] background
[0003] The cameras and rotary encoders used in rail vehicle machine vision systems must be calibrated and synchronized to provide clear, high-contrast, and properly scaled images. The rotary encoder on the rail vehicle triggers the camera controller to capture images of the track. Therefore, the machine vision object detection model is only as good as the input image. During transportation, the machine vision system may change and no longer properly aim at the track or capture a clear image. In addition, rail vehicles often need to be replaced and maintained due to wear on their wheels and suspension, which further affects the placement of the machine vision system. Therefore, the machine vision system on the rail vehicle chassis needs to be calibrated to synchronize the rotary encoder with the camera to compensate for various image capture obstacles.
[0004] Calibrating a machine vision system on a rail vehicle chassis can require mostly operator intervention to remove the machine vision system and calibrate the system in a bench test environment. After calibration, the operator needs to reconnect the machine vision system and repeat redundant processes, including stopping the rail vehicle, reversing, and repeatedly passing over the same section of track to ensure proper images are collected. Despite this, the calibration process can be arduous for rail vehicle operators. Inefficient processes with modern calibration technology can result in expensive repetitive processes while also wasting the time of operators who require a bench test environment. Operators incur opportunity costs due to wasted time.
[0005] While organizational tools and processes exist to calibrate rail vehicle machine vision systems in a bench test environment, current processes are not suitable for field calibration. Additionally, rail vehicle operation in a variety of environmental conditions wears out rail vehicle components, causing machine vision system movement and image distortion. Currently, operators are unable to calibrate machine vision systems while rail vehicles are in the field. When operators need to calibrate machine vision systems, current methods require operators to remove the machine vision system and utilize the bench test environment to calibrate the machine vision system, which is inefficient and cumbersome for field operations. Summary of the invention
[0006] The present disclosure achieves technical advantages as a system and method for calibrating a machine vision system on a rail vehicle chassis while the rail vehicle is in the field. The system enables a rail vehicle operator to calibrate a machine vision system without removing the machine vision system from the rail vehicle chassis. In one embodiment, the rail vehicle can be a locomotive, a rail car, or other suitable rail vehicle. The system can simplify the calibration process for the operator. In addition, the present disclosure eliminates the need for a bench test environment to calibrate the machine vision system. The system can capture a target image, calculate the object pixel size of the image to ensure calibration of the longitudinal measurement, and calibrate the rotary encoder using the camera of the machine vision system. The system can achieve calibration of rail vehicle components on site to compensate for the misalignment of the machine vision system and ensure driving safety.
[0007] The present disclosure solves the technical problem that a bench test environment is required for the calibration of a machine vision system for a rail vehicle by providing a modular calibration target for field calibration of the machine vision system. The present disclosure is not limited to a simple calibration bench test, but also combines at least a modular calibration target and an image analysis function to achieve accurate calibration to enhance the safety of rail vehicles. The present disclosure can eliminate the requirement to calibrate a machine vision system using a roller in a bench test environment by replacing the roller and bench test environment with a modular calibration target, thereby allowing an operator to calibrate the machine vision system in the field. By eliminating the requirement for a bench test environment, operators can improve driving efficiency while ensuring rail vehicle safety.
[0008] The present disclosure improves the performance and functionality of the system itself by replacing the drum in a benchtop test environment with a modular calibration target. The modular calibration target includes the ability to adapt to various types of track sizes, which is contrary to the current process that lacks adaptability. In addition, the present disclosure includes software tools for calculating the calibration between the rotary encoder and the camera of the machine vision system based on the pixel count of the image. The rotary encoder can be based on the wheel motion of the rail vehicle, and when the wheel rotates, the camera captures the image. In addition, the system can also compare the captured image with a known distance value and calibrate the machine vision system based on the comparison results.
[0009] The system may include the ability to filter the electronic signal to trigger the camera controller to instruct the camera to capture an image. The rotary encoder of the machine vision system may include a software process to filter the trigger signal to eliminate asynchronous signals. Current methods are unable to filter the electronic trigger signal that causes image distortion. The quality of the image directly affects the effectiveness of the machine vision system. The higher the image quality, the higher the granularity, and thus the smaller the gaps between pixels in the machine vision system image.
[0010] An object of the present invention is to provide a method for calibrating an image capture module. Another object of the present invention is to provide a calibration rod for calibrating an image acquisition module. Another object of the present invention is to provide a computer-readable storage medium for calibrating an image capture module. Another object of the present invention is to provide a system for calibrating an image capture module. These and other objects are provided by at least one of the following embodiments.
[0011] In one embodiment, a method includes: capturing an image of a target through a camera of an image capture module attached to a vehicle, wherein the target is at least one calibration pole attached to a road; determining a number of lateral pixels in a lateral pitch of the target image; determining a lateral object pixel size based on the number of lateral pixels; determining a number of longitudinal pixels in a longitudinal pitch of the image; and comparing the number of lateral pixels to the number of longitudinal pixels. Wherein, the method also includes: determining that the number of lateral pixels matches the number of longitudinal pixels in response to comparing the number of lateral pixels to the number of longitudinal pixels; and calculating a vehicle encoder rate based on an on-site calibration assessment. Wherein, the on-site calibration assessment includes on-site calculation of the lateral object pixel size based on the number of lateral pixels and the determined number of pixels. Wherein, the image also includes programming a vehicle encoder rate into a vehicle encoder attached to a wheel of the vehicle. Wherein, the vehicle encoder rate is based on a second relationship between a trigger factor, a wheel circumference, and an object pixel size. Wherein, the method also includes: determining that the number of lateral pixels does not match the number of longitudinal pixels in response to comparing the number of lateral pixels to the number of longitudinal pixels; resolving the difference between the number of lateral pixels and the number of longitudinal pixels using new calibration information to calibrate the image capture module; and transmitting the new calibration information. The method further includes: focusing a camera of an image capture module on a target under constant illumination conditions; and obtaining a maximum contrast between two pixels identifying a boundary of a light and dark portion of the target. The target includes a pattern of a plurality of alternating black and white portions; the transverse pitch represents a width of an end plate coupled to the target; and the longitudinal pitch represents a length of one of the plurality of black and white portions.
[0012] In another embodiment, the calibration pole includes: at least two end plates; a target attached to the at least two end plates; an image capture module attached to a fixture, wherein the image capture module includes a camera, and the camera is operable to capture an image of the target; and one or more controllers communicatively coupled to the vehicle encoder and the camera, wherein the one or more controllers are operable to: determine the number of lateral pixels in the lateral pitch of the target image; determine the lateral object pixel size based on the number of lateral pixels; determine the number of longitudinal pixels in the longitudinal pitch of the image; and compare the number of lateral pixels to the number of longitudinal pixels. Wherein, the one or more controllers are further operable to: determine that the number of lateral pixels matches the number of longitudinal pixels in response to comparing the number of lateral pixels to the number of longitudinal pixels; and calculate a vehicle encoder rate based on an in-situ calibration assessment. Wherein, the in-situ calibration assessment includes an in-situ calculation of the lateral object pixel size based on the number of lateral pixels and the determined number of pixels. Wherein, the image also includes programming a vehicle encoder rate into a vehicle encoder attached to a wheel of the vehicle. Wherein, the vehicle encoder rate is based on a relationship between a trigger factor, a wheel circumference, and an object pixel size. Wherein, the one or more controllers are further operable to: determine, in response to comparing the number of lateral pixels with the number of longitudinal pixels, that the number of lateral pixels does not match the number of longitudinal pixels; use new calibration information to resolve the difference between the number of lateral pixels and the number of longitudinal pixels to calibrate the image capture module; and transmit the new calibration information. Wherein, the one or more controllers are further operable to program the vehicle encoder rate into a vehicle encoder attached to a wheel of the vehicle. Wherein, the camera of the image capture module is further operable to adjust its focus on the target under constant lighting conditions to obtain a maximum contrast between two pixels identifying a boundary of a light and dark portion of the target. Wherein: the target includes a pattern of alternating multiple black and white portions; the lateral pitch represents the width of an end plate coupled to the target; and the longitudinal pitch represents the length of one of the multiple black and white portions.
[0013] In another embodiment, one or more computer-readable storage media contain instructions that, when executed by a processor, cause the processor to perform operations including: capturing an image of a target through a camera of an image capture module attached to a vehicle, wherein the target is a calibration pole attached to a road; determining a number of lateral pixels in a lateral pitch of the target image; determining a lateral object pixel size based on the number of lateral pixels; determining a number of longitudinal pixels in a longitudinal pitch of the image; and comparing the number of lateral pixels to the number of longitudinal pixels. Wherein, the operations also include: in response to comparing the number of lateral pixels to the number of longitudinal pixels, determining that the number of lateral pixels matches the number of longitudinal pixels; and calculating a vehicle encoder rate based on an on-site calibration assessment. Wherein, the on-site calibration assessment includes an on-site calculation of the lateral object pixel size based on the number of lateral pixels and the determined number of pixels. Wherein, the image also includes programming a vehicle encoder rate into a vehicle encoder attached to a wheel of the vehicle. Wherein, the vehicle encoder rate is based on a relationship between a trigger factor, a wheel circumference, and an object pixel size. Wherein, the one or more controllers are further operable to: determine, in response to comparing the number of lateral pixels with the number of longitudinal pixels, that the number of lateral pixels does not match the number of longitudinal pixels; use new calibration information to resolve the difference between the number of lateral pixels and the number of longitudinal pixels to calibrate the image capture module; and transmit the new calibration information. Wherein, the operation also includes programming a vehicle encoder rate into a vehicle encoder attached to a wheel of the vehicle. Wherein, the operation also includes focusing a camera of the image capture module on a target under constant lighting conditions to obtain a maximum contrast between two pixels identifying a boundary between a light and dark portion of the target. Wherein: the target includes a pattern of alternating multiple black and white portions; the lateral pitch represents the width of an end plate coupled to the target; and the longitudinal pitch represents the length of one of the multiple black and white portions.
[0014] In another embodiment, a system includes: at least two end plates attached to a roadway, including visual markings perpendicular to the roadway; and a calibration rod attached to the at least two end plates, including visual markings alternating along the length of the calibration rod. Wherein at least two of the end plates can be magnetically attached to the roadway. Wherein at least two of the end plates can be used with multiple track sizes. Wherein the multiple track sizes include 112 pounds per yard track, 132 pounds per yard track, 136 pounds per yard track, and 141 pounds per yard track. Wherein the visual markings perpendicular to the lane are black stripes. Wherein each of the visual markings arranged alternately along the length of the calibration rod is the same length. Wherein the alternating visual markings along the length of the calibration rod alternate between black and white paint. Wherein the calibration rod is a rigid material. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] The present disclosure will be easily understood by the following detailed description in combination with the accompanying drawings, which illustrate the principles of the present disclosure by way of example. The accompanying drawings show the design and practicality of one or more exemplary embodiments of the present disclosure, wherein the same elements are represented by the same reference numerals or symbols. The objects and elements in the accompanying drawings are not necessarily drawn in proportion, ratio or precise positional relationship. Instead, the focus is on illustrating the principles of the present disclosure.
[0017] Figure 1 A system for calibrating an image capture module and a vehicle encoder in the field according to one or more exemplary embodiments of the present disclosure is shown;
[0018] Figure 2 An image capture module according to one or more exemplary embodiments of the present disclosure is shown;
[0019] Figure 3 A system for an adaptable calibration target according to one or more exemplary embodiments of the present disclosure is shown;
[0020] Figure 4 A flow chart for calibrating an image capture module in the field according to one or more exemplary embodiments of the present disclosure is shown;
[0021] Figure 5 A computer system that can be used to implement the systems and methods described herein is shown according to one or more exemplary embodiments of the present disclosure; and
[0022] Figure 6 An in-situ calibration system for calibrating an image capture module according to one or more exemplary embodiments of the present disclosure is shown.
[0023] Detailed description
[0024] The present disclosure and the various features and advantages details therein set forth in the description below will be more fully explained by reference to the non-limiting examples contained in the accompanying drawings and the detailed description that follows. Descriptions of well-known components are omitted herein so as not to unnecessarily obscure the main features described herein. The following examples are intended to facilitate understanding of the implementation and practice of the present disclosure. Those of ordinary skill in the art will understand that the present disclosure means that any suitable combination of the following functions or exemplary embodiments can be combined to achieve the claimed subject matter. The present disclosure includes the number of representative species within the genus, or structural features common to members of the genus, so that those of ordinary skill in the art can identify members of the genus. Therefore, these examples should not be construed as limiting the scope of the claims.
[0025] Figure 1An example system 100 for calibrating an image capture module 140 in the field is shown. The system 100 or portions thereof may be associated with an entity, which may include any entity, such as a business, a company (e.g., a railroad company, a transportation company, etc.), or a government agency (e.g., a transportation department, a public safety department, etc.) that calibrates the image capture module 140 in the field. Elements of the system 100 may be implemented using any suitable combination of hardware, firmware, and software. For example, the system 100 may be implemented using a hardware, firmware, and software package. Figure 1 One or more components of a computer system to implement elements of system 100. 5.
[0026] System 100 includes a vehicle 110 , a vehicle encoder 120 , a beam 130 , one or more image capture modules 140 , a computer 150 , a network 160 , a target 170 , and an end plate 180 .
[0027] The vehicle 110 may include any machine capable of autonomous movement. The vehicle 110 may be a car, a train car, a truck, a bus, an airplane, or any other machine suitable for movement. The vehicle 110 may be operated at any speed to allow one or more components (e.g., sensors, cameras, etc.) of the beam 130 to capture images. For example, the vehicle 110 may be a rail vehicle traveling at 65 miles per hour (mph).
[0028] Road 112 may include any suitable road for vehicle 110. For example, vehicle 110 may travel along road 112. Road 112 may include a road, a highway, a railroad track, a waterway, and the like.
[0029] The vehicle encoder 120 may include a rotary encoder or other timing device for measuring the rotation of an axle. The vehicle encoder 120 may measure the number of times the axle rotates. The vehicle encoder 120 may be attached to an axle of the vehicle 110. The vehicle encoder 120 may be physically and / or logically connected to one or more components of the system 100. For example, the vehicle encoder 120 may be physically and / or logically connected to one or more cameras and / or sensors of the image capture module 140. As another example, the vehicle encoder 120 may be physically and / or logically connected to the computer 150.
[0030] The vehicle encoder 120 can communicate with the camera controller of the image capture module 140 to ensure that the camera captures images with the same perspective and scale regardless of the speed of the vehicle 110. For example, the vehicle encoder 120 can be synchronized with multiple cameras of the image capture module 140 to ensure that all cameras capture images at the same time. As another example, the vehicle encoder 120 can be synchronized with the cameras of the image capture module 140 to ensure that the images captured by the camera traveling with the vehicle 110 at a first speed (e.g., 10 miles per hour) have the same perspective and scale as the camera traveling with the vehicle 110 at a second speed (e.g., 65 miles per hour).
[0031] In another embodiment, the vehicle encoder 120 can be mechanically coupled to the vehicle 110 to reduce or eliminate lost motion that causes undesirable artifacts in the images generated by the image capture module 140. For example, lost motion may include loosening of the mechanical coupling, thereby causing image distortion. In another embodiment, lost motion can be mechanically reduced using components specially machined for the vehicle encoder. For example, components specially machined for the vehicle encoder can ensure a flexible and rigid fit to minimize vibration and other mechanical disturbances that cause stalling.
[0032] In another embodiment, the vehicle encoder 120 may be electrically coupled to the image acquisition module 140 including an electronic filter. For example, the electronic filter may filter the trigger signal sent to the camera of the image capture module 140 to smooth the trigger filter signal to compensate for asynchronous signal elements. In one embodiment, an averaging filter may be used to pass the trigger signal value within a user-defined time frame to smooth the asynchronous signal elements. For example, the averaging filter may recreate a smoothed trigger signal for distribution to the camera of the image capture module 140. In another embodiment, the electronic filter is implemented on the encoder controller and receives a user-defined number of pulses from the vehicle encoder 120. In one embodiment, the electronic filter is implemented on the encoder and receives a variable number of pulses within a user-defined time frame.
[0033] The beam 130 may include a structure that contains and orients components for capturing images (e.g., image capture module 140). In certain embodiments, the operation of the beam 130 is similar to a flatbed document scanner, except that the beam 130 is in motion when capturing images of stationary physical objects. The beam 130 may be engaged with the vehicle 110. For example, the beam 130 may be bolted to a subframe attached to the vehicle 110. Figure 1In the embodiment shown in , the beam 130 has three parts, including two end parts and a center part. The beam 130 has a gull-wing structure, so that the center part is bent inwardly toward the center of the beam 130. The gull-wing configuration allows the image capture components (e.g., sensors, cameras, etc.) of the image capture module 140 within the beam 130 to be correctly oriented relative to the physical object being captured. In some embodiments, the center part of the beam 130 is omitted, and each end part is connected to the vehicle 110. The beam 130 can be made of metal (e.g., steel or aluminum), plastic, or any other material suitable for accommodating the components of the beam 130 and attaching the beam 130 to the vehicle 110.
[0034] The beam 130 may include one or more openings. The openings may be used to place the image capture module 140 within the beam 130. The openings may be used to install, adjust, and maintain the image capture module 140. Figure 1 1 and 2, the beam 130 is shown as having a particular size and shape, but in reality the beam 130 can have any size and shape suitable for accommodating and positioning the image capture module 140. Other factors that may contribute to the design of the beam 130 include impact resistance, vibration resistance, weatherproofing considerations, durability, ease of maintenance, calibration considerations, and ease of installation.
[0035] In another embodiment, the beam 130 may include multiple sub-beams. For example, the beam 130 may include two independent sub-beams, each sub-beam including multiple cameras. In one embodiment, the system 100 with multiple sub-beams may reduce the complexity of maintenance and simplify the construction of each sub-beam. In another embodiment, the system 100 with multiple sub-beams may reduce the complexity of maintenance by reducing the number of personnel required, thereby maintaining control over construction tolerances. For example, welding and cutting of sub-beams may be reduced by 33% compared to full beams.
[0036] The image capture module 140 of the system 100 is used to capture images while the vehicle 110 is traveling. Each image capture module 140 may include one or more sensors, one or more cameras, etc. One or more image capture modules 140 may be attached to any location of the vehicle 110 so that the image capture module 140 can capture images of the environment surrounding the vehicle 110. Figure 1 In the embodiment shown in , the image capture module 140 is located within the beam 130 .
[0037] In certain embodiments, each end portion of the beam 130 accommodates one or more image capture modules 140. For example, a first end portion of the beam 130 may accommodate an image capture module 140 that includes two downward facing cameras for capturing images of the railroad sleeper and ballast areas of the rail. The first end portion of the beam 130 may accommodate two downward facing cameras located in a portion of the first end portion that is substantially horizontal with the rail. A second end of the beam 130, opposite the first end, may accommodate two image capture modules 140 that each include two angled cameras for capturing images of both sides of the rail and the rail fastening system. The second end of the beam 130 may accommodate four angle cameras located in a portion of the second end that is at an angle (e.g., a 45 degree angle) to the rail.
[0038] The image capture module 140 may include various types of sensors depending on the sensing and / or measurement requirements. In one embodiment, the sensors housed by the image capture module 140 may include optical sensors (e.g., cameras for visible light (monochrome and color), infrared, ultraviolet, and / or thermal), motion sensors (e.g., gyroscopes and accelerometers), light detection and ranging (LIDAR) sensors, hyperspectral sensors, global positioning system (GPS) sensors, etc. Optical sensors and lasers can be used together for laser triangulation to measure deflection or profile. LIDAR sensors can be used to generate three-dimensional (3D) point cloud data. Hyperspectral sensors can be used for specific wavelength responses. An example of the image capture module 140 is shown below. Figure 2 Described in .
[0039] The computer 150 may represent any suitable computing component that can be used to process information for the system 100. In one embodiment, the computer 150 may coordinate one or more components of the system 100. In another embodiment, the computer 150 may receive data from the image capture module 140 and / or the vehicle encoder 120. The computer 150 may monitor the input and / or output of the image capture module 140 and / or the vehicle encoder 120. In another embodiment, the computer 150 may include communication functionality that allows a user (e.g., a technician) to directly engage with the system 100. For example, the user may access the computer 150 through an interface (e.g., a screen, a graphical user interface (GUI) or a panel) of the computer 150. The computer 150 may be a laptop, a desktop computer, a smartphone, a tablet computer, a personal digital assistant, a wearable device, etc. The computer 150 may be located inside or outside the vehicle 110. The computer 150 may communicate with one or more components of the system 100 via a network 160.
[0040] The network 160 may include any type of network that facilitates communication between components of the system 100. One or more portions of the network 160 may include an ad hoc network, an intranet, an extranet, a virtual private network (VPN), a local area network (LAN), a wireless local area network (WLAN), a wide area network (WAN), a wireless wide area network (WWAN), a metropolitan area network (MAN), a portion of the Internet, a portion of the public switched telephone network (PSTN), a cellular telephone network, a 3G network, a 4G network, a 5G network, a long term evolution (LTE) cellular network, a combination of two or more of these, or other suitable types of networks. One or more portions of the network 160 may include one or more access (e.g., mobile access), core and / or edge networks. The network 160 may be any communication network, such as a private network, a public network, a connection through the Internet, a mobile network, a Wi-Fi network, a Bluetooth network, etc. One or more components of the system 100 may communicate via the network 160. For example, the computer 150 may communicate via the network 160, including receiving information from the image capture module 140 and / or the vehicle encoder 120.
[0041] Target 170 may include an object for calibrating image capture module 140 and / or vehicle encoder 120. For example, target 170 may include a calibration rod. In another embodiment, the calibration rod may include a cylindrical object made of a solid material. For example, the solid material may include aluminum (or other metals), PVC, wood, or other materials suitable for stabilizing the calibration rod. In another embodiment, target 170 may include a calibration pattern, which may be any suitable size, shape, and / or design. For example, the calibration pattern design may include alternating solid colors, a checkerboard pattern, a chessboard pattern, a circular grid pattern, a CharucoBoard pattern, and the like. For example, the calibration pattern may be a printed black and white alternating pattern including a plurality of black and white portions. In another embodiment, the calibration pattern may include units with unequal aspect ratios. For example, the length of each unit may be twice the width of each unit.
[0042] The end plate 180 may include at least one object to which the target 170 is attached. For example, the end plate 180 may include a structure for stabilizing the position of the target 170. In one embodiment, the structure may be metal (e.g., aluminum), plastic, wood, or some other sturdy material for stabilizing the target 170. In another embodiment, the end plate 180 may be used to stabilize the target 170 in order to calibrate the image capture module 140. In one embodiment, the end plate 180 is placed along the track by an operator. For example, the end plate 180 may include a component that is small enough and light enough so that the operator can move it. Alternatively, the end plate 180 may be fixed as part of the track.
[0043] In operation, the vehicle encoder rate is programmed into the vehicle encoder 120. The vehicle encoder rate is the number of electrical pulses generated by the vehicle encoder 120 when the shaft of the vehicle encoder 120 rotates one revolution. The vehicle encoder rate can be determined based on calibration data previously generated during the calibration process, as shown below Figure 3 and 5 If there is no calibration data, an arbitrary initial value of the vehicle encoder rate can be programmed into the vehicle encoder 120. In some embodiments, the vehicle encoder rate programmed into the vehicle encoder 120 is an integer. In some embodiments, an operator programs the vehicle encoder rate into the vehicle encoder 120.
[0044] The vehicle encoder 120 and image capture module 140 of the system 100 are fixed to the vehicle 110. A target 170 may be fixed to the road 112 to facilitate the field of view of the camera of the image capture module 140 for calibration. The target 170 is arranged perpendicular to the axis of the camera of the image capture module 140. The camera of the image capture module 140 is activated and the operator observes the current focus of the camera under constant lighting conditions. If the contrast between two pixels marking the boundary of the light and dark parts of the target 170 is less than the maximum contrast obtainable (or less than the contrast observed during the bench calibration procedure), the operator will unlock the focus mechanism of the camera and adjust the focus until the maximum contrast is achieved. The focus mechanism is then locked.
[0045] The image capture module 140 is connected to the computer 150 via the network 160. The computer 150 includes an image capture software. The image capture module 140 is used to capture a first image of the target 170 and display it on the computer 150. The operator determines the number of lateral pixels in the lateral pitch distance of the first image of the target 170, and determines the lateral object pixel size (OPS) by dividing the pitch of the target 170 by the number of lateral pixels in the pitch area. The test vehicle encoder rate is then determined by dividing the wheel circumference of the vehicle 110 by the lateral OPS. If the test vehicle encoder rate is different from the initial vehicle encoder rate programmed into the vehicle encoder 120, the test vehicle encoder rate is programmed into the vehicle encoder 120. The image capture software of the computer 150 is triggered by the vehicle encoder 120, and the vehicle 110 moves forward or backward over the target 170.
[0046] As the vehicle 110 moves over the target 170, the image capture device 140 captures a second image of the target 170. The operator of the computer 150 determines (e.g., counts) the number of light or dark longitudinal pixels within one longitudinal interval of each second image, and compares the number of lateral pixels to the number of longitudinal pixels. If the number of lateral pixels matches the number of longitudinal pixels, the image capture module 140 and the vehicle encoder 120 are calibrated. If the number of lateral pixels is different from the number of longitudinal pixels, the vehicle encoder rate is adjusted until the number of lateral pixels matches the number of longitudinal pixels. Thus, the system 100 can be used to calibrate the image capture module 140 and the vehicle encoder 120 to ensure that the system 100 captures enough images that can be used to accurately identify objects in the environment surrounding the vehicle 110.
[0047] Although Figure 1 A particular arrangement of vehicle 110, vehicle encoder 120, beam 130, image capture module 140, computer 150, network 160, and target 170 is shown, but the present disclosure contemplates any suitable arrangement of vehicle 110, vehicle encoder 120, beam 130, image capture module 140, computer 150, network 160, target 170, and end plate 180. For example, computer 150 may be located inside vehicle 110. Vehicle 110, vehicle encoder 120, beam 130, image capture module 140, and computer 150 may be physically or logically co-located in whole or in part.
[0048] Although Figure 1 A particular number of vehicles 110, vehicle encoders 120, beams 130, image capture modules 140, computers 150, networks 160, and targets 170 are shown, but the present disclosure contemplates any suitable number of vehicles 110, vehicle encoders 120, beams 130, image capture modules 140, computers 150, networks 160, targets 170, and end plates 180. For example, system 100 may include a first beam located at a front end of vehicle 110 and a second beam located at a rear end of vehicle 110. As another example, system 100 may include multiple computers 150. One or more components of system 100 may be configured using Figure 1 It is implemented by one or more components of a computer system. 5.
[0049] Figure 2 2 shows an example image capture module 140 that can be used by the system 100. The image capture module 140 includes a camera 210, a lens 220, a top plate 230, a bottom plate 240, a cover plate 250, a bolt 260, and an opening 270. The camera 210 is any device capable of capturing an image. For example, the camera 210 can capture Figure 11. As another example, the camera 210 can capture images of track components (e.g., track joints, switches, turnouts, fasteners, ballast, rail heads, and / or sleepers). In some embodiments, the camera 210 includes one or more sensors.
[0050] One or more cameras 210 may capture images from different angles. For example, one or more cameras 210 may capture images of two tracks of a railway system at any given location. Each beam (e.g., Figure 1 The beam 130 in the embodiment may include a plurality of cameras 210. The beam may include a first camera 210 that is aimed directly downward to capture a target (e.g., Figure 1 The beam may include a second camera 210 that is aimed downward and outward to capture an oblique view image of a target, physical object, etc.
[0051] The camera 210 can be a line scan camera. A line scan camera contains one row of pixels. The camera 210 can be a dual line scan camera. A dual line scan camera contains two rows of pixels that can be captured and / or processed simultaneously. As the camera 210 moves over the physical object, the camera 210 can capture images so that a complete image of the physical object can be reconstructed line by line in software. The capture rate of the camera 210 can be as high as 140 kHz. The camera 210 can have a resolution and an optical system to detect physical objects at least 1 / 16 inch in size. In one or more embodiments, the camera 210 includes a lens 220 that focuses and directs incident light to a sensor of the camera 210. The lens 220 can be a piece of glass or other transparent material. The lens 220 can be made of any suitable material (e.g., steel, aluminum, glass, plastic, or a combination thereof).
[0052] The top plate 230 and the bottom plate 240 are structural elements for positioning, supporting and / or stabilizing one or more components (e.g., the camera 210 or the sensor) of the image capture module 140. The top plate 230 and the bottom plate 540 can be made of any suitable material (e.g., steel, aluminum, plastic, glass, etc.). The top plate 230 can be connected to the bottom plate 240 by one or more bolts 260. The bolts 260 (e.g., jack bolts) can be used to change the pitch and / or roll direction of the camera 210. For example, the bolts 260 can be used to change the effective height between the top plate 230 and the bottom plate 240. The top plate 230 and / or the bottom plate 240 can be adjusted to reduce vibration and / or shock of the image capture module 140. The top plate 230 and / or the bottom plate 240 can include a resistive heating element to provide a warm environment for the operation of the camera 210 and the lens 220 during colder weather.
[0053] The cover plate 250 may be a plate covering the base plate 240. The cover plate 250 may be made of any suitable material (e.g., glass, steel, aluminum, etc.). The cover plate 250 includes an opening 270. The opening 270 may be used as an aperture for the lens of the camera 210 to observe physical objects. The opening 270 allows the signal sensed from the surrounding environment to be transmitted to the sensor of the camera 210. The opening 270 may be any suitable size (e.g., oval, rectangular, etc.) to accommodate the view of the camera 210. The lens 220 of the camera 210 may be placed directly above the opening 270.
[0054] Although Figure 2 While a particular arrangement of camera 210, lens 220, top plate 230, bottom plate 240, cover plate 250, bolts 260, and opening 270 is shown, the present disclosure contemplates any suitable arrangement of camera 210, lens 220, top plate 230, bottom plate 240, cover plate 250, bolts 260, and opening 270. Figure 2 A particular number of cameras 210, lenses 220, top plates 230, bottom plates 240, covers 250, bolts 260, and openings 270 are shown, but the present disclosure contemplates any suitable number of cameras 210, lenses 220, top plates 230, bottom plates 240, covers 250, bolts 260, and openings 270. For example, image capture module 140 may include multiple cameras 210. As another example, in some embodiments, image capture module 140 may not include Figure 2 One or more components of the image capture module 140 may be implemented using Figure 5 It is implemented by one or more elements of a computer system.
[0055] Figure 3 An example system 300 that can accommodate calibration targets is shown. The system 300 includes a road (e.g., Figure 1 The system 300 or a portion thereof may be associated with an entity, which may include any entity, such as a business, a company (e.g., a railroad company, a transportation company, etc.), or a government agency (e.g., a transportation department, a public safety department, etc.) that calibrates the image capture module on site. Figure 3 The system 300 in FIG. 1 includes the target 170 , the end plate 180 , a nut 302 , a marking strip 304 , an attachment device 306 , and a fastener hole 308 .
[0056] Nuts 302 may couple the target 170 to the end plates 180. For example, nuts 302 may include a mechanical coupler, such as a screw, bolt, cotter pin, or another mechanical coupler. In one embodiment, an operator of the rail vehicle will exit the rail vehicle, attach the end plates 180 to the rails, and couple each end of the target 170 to each end plate 180 using nuts 302. In another embodiment, an operator may rotate nuts 302 to connect and disconnect the system in the field.
[0057] The marker strip 304 may include a solid black strip on top of the end plate. For example, the marker strip 304 may include a strip of known length for calibration purposes. In one embodiment, the marker strip 304 may be used to calibrate the image system on the rail vehicle by providing a known lateral distance. For example, the marker strip 304 may be 5 inches in length. In another embodiment, the image system may capture an image of the marker strip 304 and analyze the image to determine if the image system is calibrated. In another embodiment, the image with the marker strip 304 may provide a plurality of lateral pixels for analysis.
[0058] The attachment device 306 can couple the end plate 180 to the track. For example, the attachment device 306 can couple the end plate 180 to the track mechanically, electrically, or magnetically. In one embodiment, the attachment device 306 may include a mechanical component to couple the end plate 180 to the track. For example, the mechanical component may include a clamp, a bolt, a screw, a cotter pin, or some other mechanical coupler. In another embodiment, the attachment device 306 may include an electrical component to couple the end plate 180 to the track. For example, the electrical component may include an electromechanical clamp, an electromagnetic coupler, or some other electrical coupler. In another embodiment, the attachment device 306 may include a magnetic component to couple the end plate 180 to the track. For example, the magnetic component may include a disk, a strip, or a paint manually placed by an operator. In another embodiment, the attachment device 306 can be detachable from the end plate 180. Alternatively, the attachment device 306 may be permanently attached to the end plate 180.
[0059] The fastener holes 308 may couple the target 170 to the end plate 180. For example, the fastener holes 308 may interconnect the target 170 with the nut 302. In another embodiment, the fastener holes 308 may be a portion of the end plate 180 or another base to which the target 170 is applied.
[0060] In operation, a user (eg, an operator) installs an image capture module (eg, image capture module 140 or a portion thereof, such as Figure 2The user may also include a camera 210 of the image capture module. The user may ...
[0061] In one embodiment, the operator may perform an on-site calibration assessment as discussed below. For example, the on-site calibration assessment may include successful focus achieved when maximum contrast is obtained between two pixels at the boundary of the light and dark portions of a calibration pattern identifying target 170 (e.g., alternating colors or a checkerboard pattern). In one embodiment, the user then locks the focus mechanism of the image capture module. In another embodiment, the operator may identify an image displayed on a computer, with the user observing a black or white area on target 170 in the middle of the camera's field of view. For example, the field of view may represent the angle at which the camera of the image capture module picks up electromagnetic radiation. In one embodiment, the field of view may be limited by an area of the image displayed on the computer.
[0062] In another embodiment, a computer operator may calculate the number of bright pixels or dark pixels in a first direction of the lateral pitch of the end plate 180. In one embodiment, the first direction is parallel to the axis of the end plate 180. In another embodiment, the lateral OPS is calculated by dividing the lateral pitch by the number of pixels in the lateral pitch. For example, if the lateral pitch is equal to one inch and the number of pixels in a one inch pitch is 52, then the lateral OPS is equal to one inch divided by 52, or 0.01923 inches per pixel. In one embodiment, the lateral OPS may indicate the actual physical size represented by one pixel at a specified working distance. For example, the working distance may include the distance between a camera and a target. In another embodiment, the lateral OPS may be determined based on an on-site calculation as follows:
[0063]
[0064] Where P target is the target spacing in length units, n pixels is the determined number of pixels. For example, the determined number of pixels may include the number of pixels counted by an operator.
[0065] Alternatively, the determined number of pixels may include a number of pixels based on camera characteristics, such as image size, lens size, and image resolution. In one embodiment, measuring and calibrating the lateral OPS ensures that objects depicted in images captured by the image capture module are of proper scale and that no data is lost between pixels when the image capture module is in field operation. In another embodiment, the pixels are square or approximately square (e.g., having equal length and width within a tolerance of two percent). For example, a certain amplitude may be allowed due to limitations of the camera of the image capture module and / or a vehicle encoder (e.g., vehicle encoder 120).
[0066] In another embodiment, the field calibration assessment may include determining a vehicle encoder rate of the vehicle encoder based on the lateral OPS. In one embodiment, the vehicle encoder rate may be equal to the number of electrical pulses generated by the vehicle encoder when the wheel axle rotates one revolution. For example, the vehicle encoder rate may be calculated by dividing the wheel circumference by the lateral OPS. In another embodiment, the vehicle encoder rate of the vehicle encoder is determined based on the lateral OPS. For example, the encoder rate is based on the following formula:
[0067]
[0068] where k fg is the trigger factor set in the camera or software, c wheel is the circumference of the wheel.
[0069] In some embodiments, the encoder rate is programmed into the vehicle encoder as an integer value. For example, the vehicle encoder rate is programmed into the vehicle encoder as an integer value. In one embodiment, the vehicle encoder may be programmed to 1715 or 1716 pulses per revolution. For example, an operator may drive a vehicle (e.g., vehicle 110) over target 170 at a low speed. In one embodiment, the low speed may be in the range of 5 to 20 mph (e.g., 10 mph). In another embodiment, an image capture module captures images while the rail vehicle is traveling at a low speed and transmits the collected images to a computer. In one embodiment, a computer operator determines (e.g., counts) the number of bright pixels or dark pixels in a second direction within a longitudinal spacing on target 170. For example, in Figure 3 In the illustrated embodiment, the second direction is parallel to the axis of the target 170 .
[0070] The operator then drives the vehicle at a high speed. The high speed can be in the range of fifty to eighty miles per hour (mph) (e.g., 65 mph). The high speed can represent the maximum speed of the vehicle. The image capture module captures images while the vehicle is traveling at a high speed and transmits the captured images to a computer. The computer operator determines (e.g., counts) the number of bright pixels or dark pixels within a spacing distance on the target 170 in the second direction. The high-speed and low-speed longitudinal pixel counts are compared with the horizontal pixel counts to determine whether the camera pixels represent the physical space equally in the horizontal and longitudinal directions. If the longitudinal pixel count is different from the horizontal pixel count, a different encoder rate can be programmed into the vehicle encoder, and the above process is repeated to compare the effect of the new encoder rate on the horizontal and longitudinal pixel counts.
[0071] Figure 4 A flow chart of an example field calibration control logic 400 according to one or more embodiments of the present disclosure is shown. The field calibration control logic 400 can be implemented as an algorithm on a server, a machine learning module, a client, a database, or other suitable system. In addition, the field calibration control logic 400 can implement or include one or more features of the image capture module 140. The field calibration control logic 400 can be implemented by software, hardware, an application programming interface (API), a network connection, a network transmission protocol, HTML, DHTML, JavaScript, Dojo, Ruby, Rails, other suitable applications, or a suitable combination thereof.
[0072] The field calibration control logic 400 can take advantage of the capabilities of the computer platform to generate multiple processes and threads by processing data simultaneously. By instantiating multiple processes to implement data lifecycle management, the speed and efficiency of the field calibration control logic 400 can be greatly improved. However, those skilled in the art of programming will recognize that a single processing thread may also be used and this is within the scope of the present disclosure.
[0073] In one embodiment, the command or data may be received via user input generated on the client or server, such as a screen click, a swipe, a mouse click, a key press, a voice command, or other suitable mechanism. In another embodiment, the inspection command or data may include inspection data having one or more fields, parameters, characteristics, or metadata related to the inspection. The field calibration control logic 400 then proceeds to step 410.
[0074] In step 410, in one embodiment, the control logic 400 may capture a first image of a target. For example, a camera in an image capture module (e.g., Figure 2 The camera 210 of the image capture module 140 in the image capture module 140 captures the target (e.g., Figure 1In one embodiment, the image capture module may be fixed to a vehicle (e.g., Figure 1 110), the target may be fixed on the road (e.g., Figure 1 In another embodiment, the target is perpendicular to the axis of the camera of the image capture module. In one embodiment, the image captured by the camera of the image capture module can be displayed on a computer (e.g., Figure 1 The control logic 400 proceeds to step 415.
[0075] In step 415, in one embodiment, the control logic 400 may determine the number of lateral pixels in the lateral spacing of the target image. For example, the number of lateral pixels may correspond to a known distance, such as the length of the marker strip 304. In one embodiment, the control logic 400 may include an operator's determination of the number of lateral pixels in the lateral spacing of the target image. Alternatively, in one embodiment, the number of lateral pixels is automatically determined using a software tool. For example, the software tool may identify a characteristic of the camera to determine the width of the image pixels and calculate the number of lateral pixels based on the ratio of the lateral spacing to the image width. In one embodiment, the characteristic of the camera is the resolution of the camera, such as the number of pixels in the image. In one embodiment, the operator may observe the current focus of the camera under constant lighting conditions. In another embodiment, if the contrast between two pixels identifying the boundary of the light and dark portions of the focus target is less than the contrast observed in the bench test, the operator may unlock the focus mechanism and adjust the focus until a satisfactory result is obtained. The focus mechanism is then locked. In another embodiment, the operator may calculate the number of bright pixels or dark pixels within the target lateral spacing at the center of the camera's field of view. The control logic 400 proceeds to step 420.
[0076] In step 420, in one embodiment, the control logic 400 may determine the lateral OPS. For example, the lateral OPS may be determined using the determined number of lateral pixels. In one embodiment, the control logic 400 may include an operator calculating the lateral OPS by dividing the pitch of the target 170 (e.g., one inch) by the number of lateral pixels in the pitch area. In one embodiment, the computer may calculate the vehicle encoder rate based on the OPS of the image. In another embodiment, the vehicle encoder rate may be calculated based on the operator observing a black or white area on the target 170 in the middle of the camera field of view. For example, the field of view may represent the angle at which the camera of the image capture module picks up electromagnetic radiation. The field of view may be limited by the image area displayed on the computer. In one embodiment, the computer operator may calculate the number of bright pixels or dark pixels in a first direction of the lateral pitch of the end plate 180. For example, the first direction is parallel to the axis of the end plate 180. In one embodiment, the lateral OPS is calculated by dividing the lateral pitch by the number of pixels in the lateral pitch. For example, if the lateral pitch is equal to one inch and the number of pixels in the one inch pitch is 52, then the OPS is equal to one inch divided by 52, i.e., 0.01923 inches per pixel. In one embodiment, the lateral OPS indicates the actual physical size represented by one pixel at a specified working distance.
[0077] In another embodiment, the working distance may include the distance between the camera and the target. In another embodiment, the lateral OPS may be determined based on field calculations as follows:
[0078]
[0079] Where P target is the target spacing in length units, n pixels Is a determined number of pixels. For example, the determined number of pixels may include the number of pixels counted by an operator. Alternatively, the determined number of pixels may include the number of pixels based on camera characteristics, such as image size, lens size, and image resolution. In one embodiment, measuring and calibrating the lateral OPS ensures that the objects depicted in the image captured by the image capture module have the proper proportions and that no data is lost between pixels when the image capture module is in field operation. In another embodiment, the pixels are square or approximately square (e.g., having equal length and width within a tolerance of two percent). For example, due to limitations of the camera and / or vehicle encoder (e.g., vehicle encoder 120) of the image capture module, a certain amplitude may be allowed. Control logic 400 proceeds to step 425.
[0080] In step 425, in one embodiment, the control logic 400 may determine the number of longitudinal pixels in the longitudinal spacing of the image. For example, the number of longitudinal pixels may correspond to a known distance, such as the length of at least a portion of the target 170. In one embodiment, the control logic 400 may include an operator's determination of the number of longitudinal pixels in the longitudinal spacing of the target image. Alternatively, in one embodiment, the number of longitudinal pixels is automatically determined using a software tool. For example, the software tool may identify a characteristic of the camera to determine the pixel length of the image and calculate the number of longitudinal pixels based on the ratio of the longitudinal spacing to the image length. In one embodiment, the characteristic of the camera is the resolution of the camera, such as the number of pixels in the image. In one embodiment, the operator may observe the current focus of the camera under constant lighting conditions. In another embodiment, if the contrast between two pixels identifying the boundary of the light and dark portions of the focus target is less than the contrast observed in the bench test, the operator may unlock the focus mechanism and adjust the focus until a satisfactory result is obtained. The focus mechanism is then locked. In another embodiment, the operator may calculate the number of bright pixels or dark pixels within the longitudinal spacing at the center of the camera's field of view. The control logic 400 proceeds to step 430.
[0081] In step 430, in one embodiment, the control logic 400 may compare the number of horizontal pixels to the number of vertical pixels. For example, the control logic 400 may include a computer to compare the number of horizontal pixels to the number of vertical pixels. In one embodiment, the computer may determine whether the number of horizontal pixels is greater than the number of vertical pixels. Alternatively, the operator may count the pixels in the image to determine the number of horizontal pixels and the number of vertical pixels. In another embodiment, the camera is formatted to capture images with an equal number of horizontal pixels and vertical pixels. The control logic 400 proceeds to step 435.
[0082] In step 435, in one embodiment, the control logic 400 may compare the number of horizontal pixels to the number of vertical pixels. For example, the control logic 400 may include a computer to compare the number of horizontal pixels to the number of vertical pixels. In one embodiment, the computer may determine whether the number of horizontal pixels is greater than the number of vertical pixels. Alternatively, an operator may count the number of horizontal pixels and the number of vertical pixels to determine whether the number of pixels is different. In one embodiment, the image may include a difference between the number of horizontal pixels and the number of vertical pixels. For example, the difference in the number of pixels may indicate that further calibration procedures are needed. If the number of horizontal pixels is not equal to the number of vertical pixels, the control logic 400 proceeds to step 460. If the number of horizontal pixels is equal to the number of vertical pixels, the control logic 400 proceeds to step 440.
[0083] In step 440, in one embodiment, the control logic 400 may calculate a vehicle encoder rate. For example, the control logic 400 may include a computer to calculate the vehicle encoder rate. In one embodiment, the vehicle encoder rate may be equal to the number of electrical pulses generated by the vehicle encoder when the wheel axle rotates one revolution. For example, the vehicle encoder rate may be calculated by dividing the wheel circumference by the lateral OPS. In another embodiment, the vehicle encoder rate of the vehicle encoder is determined based on the lateral OPS. For example, the encoder rate is based on the following formula:
[0084]
[0085] where k fg is the trigger factor set in the camera or software, c wheel is the circumference of the wheel. Control logic 400 proceeds to step 445 .
[0086] In step 445, in one embodiment, the control logic 400 may program a vehicle encoder rate into the vehicle encoder. For example, the control logic 400 may include a computer to program the vehicle encoder rate into the vehicle encoder. For example, the vehicle encoder rate is programmed into the vehicle encoder as an integer value. In one embodiment, the vehicle encoder may be programmed to 1715 or 1716 pulses per revolution. For example, an operator may drive a vehicle (e.g., vehicle 110) over target 170 at a low speed. In one embodiment, the low speed may be in the range of 5 to 20 miles per hour (e.g., 10 miles per hour). In another embodiment, an image capture module captures images while the rail vehicle is traveling at a low speed and transmits the collected images to a computer. In one embodiment, a computer operator determines (e.g., counts) the number of bright pixels or dark pixels in a second direction within a longitudinal spacing on target 170. For example, in Figure 3 In the illustrated embodiment, the second direction is parallel to the axis of the target 170. The control logic 400 proceeds to step 450.
[0087] In step 450, in one embodiment, the control logic 400 may focus the camera on the target. For example, the control logic 400 may include a computer focusing the camera on the target under constant lighting conditions. In one embodiment, an independent lighting source may be set under the rail vehicle to illuminate the target. In another embodiment, the computer may focus the camera in a manual or automatic manner. For example, the manual focus method may include an operator generating a virtual focus on a computer display to control the camera. Alternatively, the automatic focus method of the camera may include a software tool to evaluate the image and determine the optimal focal length for the lighting environment. For example, the software tool may determine the optimal focal length based on physical factors of the camera and various environmental settings. In one embodiment, the physical factors may include the lens size of the camera, the resolution of the camera, and other camera factors. In another embodiment, the various environmental settings may include a low lighting environment, supplemented by a software filter to increase the contrast of the image pixels. The control logic 400 continues to step 455.
[0088] In step 455, in one embodiment, the control logic 400 can obtain a maximum contrast between two pixels of the image. For example, the control logic 400 can include a computer to obtain the maximum contrast between two pixels. For example, successful focus is achieved when the maximum contrast is obtained between two pixels at the boundary of the light and dark portions of a calibration pattern (e.g., alternating colors or a checkerboard pattern) of the recognition target 170. In one embodiment, an operator can lock the focus mechanism of the image capture module. For example, the operator can manually or automatically lock the focus mechanism. In one embodiment, the operator can manually lock the focus mechanism using various mechanical and electrical components, such as locking a torque drive cover to the image capture module, or other mechanical and electrical locking components.
[0089] In step 460, in one embodiment, the control logic 400 may resolve the pixel value difference between the number of horizontal pixels and the number of vertical pixels. For example, the control logic 400 may include a computer to resolve the difference between the number of horizontal pixels and the number of vertical pixels. In one embodiment, the computer may resolve the difference by adjusting the vehicle encoder rate to a new value to compensate for the difference in pixel values. In one embodiment, the rail vehicle may repeat the calibration process to capture images at various speeds and then compare the pixel values of the images to determine whether the image capture module is properly calibrated. In another embodiment, the purpose of measuring and calibrating the OPS is to ensure that the objects depicted in the image are properly scaled and that no real-world space is lost between the pixels of the image capture module when the image capture module is running. For example, the pixels may be roughly square or slightly larger in the horizontal direction. In one embodiment, by making the pixels square or slightly larger in the horizontal direction, no real-world space is lost. In another embodiment, some small margin is allowed for the horizontal to vertical pixel size, taking into account the required field of view, the actual working distance, and the limitations of the camera and the vehicle encoder. The control logic 400 continues to step 465.
[0090] In step 465, in one embodiment, the control logic 400 may transmit the new calibration information. For example, the control logic 400 may include a computer to transmit the new calibration information. For example, the new calibration information may correspond to the adjusted vehicle encoder rate. In another embodiment, the computer transmits the new calibration information over a network.
[0091] Can Figure 4 The method 400 shown in the figure may be modified, added or omitted. The method 400 may include more, less or other steps. For example, the method 400 may include programming the initial vehicle encoder rate into the vehicle encoder. As another example, the method 400 may include activating a camera of an image capture module. The various steps may be performed in parallel or in any suitable order. Although discussed as a specific component that completes the steps of the method 400, any suitable component may perform any step of the method 400.
[0092] Figure 5 An example computer system that can be used by the systems and methods described herein is shown. For example, Figure 1 One or more components of system 100 (e.g., computer 150) and / or Figure 3 The system 300 in the embodiment may include one or more interfaces 510, processing circuits 520, memory 530, and / or other suitable elements. The interface 510 receives input, sends output, processes input and / or output, and / or performs other suitable operations. The interface 510 may include hardware and / or software.
[0093] The processing circuit 520 performs or manages the operation of the component. The processing circuit 520 may include hardware and / or software. Examples of processing circuits include one or more computers, one or more microprocessors, one or more applications, etc. In some embodiments, the processing circuit 520 executes logic (e.g., instructions) to perform actions (e.g., operations), such as generating outputs from inputs. The logic executed by the processing circuit 520 may be encoded in one or more tangible, non-transitory computer-readable media (e.g., memory 530). For example, the logic may include a computer program, software, computer-executable instructions, and / or instructions that can be executed by a computer. In a particular embodiment, the operation of the embodiment may be performed by one or more computer-readable media storing, embodying and / or encoding a computer program and / or having a stored and / or encoded computer program.
[0094] The memory 530 (or storage unit) stores information. The memory 530 may include one or more non-transitory, tangible, computer-readable and / or computer-executable storage media. Examples of the memory 530 include computer memory (e.g., RAM or ROM), mass storage media (e.g., hard disk), removable storage media (e.g., compact disk (CD) or digital video disk (DVD)), database and / or network storage (e.g., server), and / or other computer-readable media.
[0095] Figure 6 An example field calibration system 600 for calibrating an image capture module 140 is shown. The system 600 may include a vehicle 110, a beam 130, one or more image capture modules 140 (not shown), a target 170, and an end plate 180. In one embodiment, the beam 130 may include a structure containing and orienting components (e.g., the image capture module 140) for capturing images. The system 600 may provide a method for passing a vehicle over a road 112, a target 170, and an end plate 180. For example, the vehicle 110 may travel along the road 112 with the target 170 and the end plate 180 located at the road 112. In one embodiment, the end plate 180 may position the target 170 at an appropriate location along the road 112 below the clearance level of the vehicle 110. In one embodiment, the road 112 may include a structure along the edge of the road 112 to provide support to hold the end plate 180. In this way, the end plate 180 may stabilize the target 170 to support the target 170 as the vehicle 110 passes by. For example, the end plate 180 may include fasteners to couple the end plate 180 to a support of the roadway 112 .
[0096] The image capture module 140 can capture a first image of the target 170 and can display it on a computer. The operator can determine the number of lateral pixels in the lateral pitch distance of the first image of the target 170 and determine the lateral OPS by dividing the pitch of the target 170 by the number of lateral pixels in the pitch area. The test vehicle encoder rate is then determined by dividing the wheel circumference of the vehicle 110 by the lateral OPS. If the test vehicle encoder rate is different from the initial vehicle encoder rate programmed into the vehicle encoder, the test vehicle encoder rate is programmed into the vehicle encoder. The image capture software of the computer is triggered according to the signal from the vehicle encoder, and the vehicle 110 moves forward or backward over the target 170.
[0097] As the vehicle 110 moves over the target 170, the image capture device 140 may capture a second image of the target 170. The computer operator may determine (e.g., count) the number of light or dark longitudinal pixels within one longitudinal spacing of each second image, and may compare the number of lateral pixels to the number of longitudinal pixels. If the number of lateral pixels matches the number of longitudinal pixels, the image capture module 140 and the vehicle encoder are calibrated. If the number of lateral pixels is different from the number of longitudinal pixels, the vehicle encoder rate may be adjusted until the number of lateral pixels matches the number of longitudinal pixels. Thus, the system 600 may be used to calibrate the image capture module 140 to ensure that the system 600 captures enough images that can be used to accurately identify objects in the environment surrounding the vehicle 110.
[0098] In one embodiment, the end plate 180 may include a component that is small enough and light enough so that the operator can move it. For example, the end plate 180 may be coupled to the road 112 by mechanical, electrical, or magnetic means. In one embodiment, the end plate 180 may include a mechanical component, such as a clamp, a bolt, a screw, a cotter pin, or some other mechanical coupler. In another embodiment, the end plate 180 may include an electrical component, such as an electromechanical clamp, an electromagnetic coupler, or some other electrical coupler. In another embodiment, the end plate 180 may include a magnetic component, such as a disk, strip, or paint, that is manually placed by an operator. In another embodiment, the end plate 180 may be removable from the road 112.
[0099] In one embodiment, the end plate 180 may provide sufficient structure to keep the target 170 parallel to the direction of travel of the road 112. For example, the image capture module 140 may capture images of the road 112 as the vehicle 110 drives past the target 170 and the end plate 180. In this manner, the image capture module 140 may capture at least one image of the target 170 and the end plate 180 along the road 112. The vehicle 110 may drive past the target 170 and the end plate 180 to capture subsequent images, thereby providing a certain number of images for calibrating the image capture module 140.
[0100] The present disclosure achieves at least the following advantages:
[0101] 1. Ability to accurately calibrate the image capture module at the site of use;
[0102] 2. Modular calibration based on calibration rods that can be installed and removed from the railway;
[0103] 3. Provide a portable calibration system to simplify current calibration techniques.
[0104] Those skilled in the art will readily appreciate that without the specific combination of computer hardware and other structural components and mechanisms assembled in the system of the present invention and described herein, the above advantages and purposes will not be possible to achieve. In addition, the algorithms, methods and processes disclosed herein improve and convert any general-purpose computer or processor disclosed in this specification and the accompanying drawings into a special-purpose computer, which is programmed to execute the disclosed algorithms, methods and processes to achieve the above functions, advantages and goals. It should also be understood that for those who are proficient in this field, there are a variety of programming tools that can be used to generate and implement the functions and operations described above. In addition, the specific selection of programming tools can be determined by the specific goals and constraints set by the implementation methods selected to implement the concepts set forth in this article and the appended claims.
[0105] The description in this patent document should not be construed as implying that any particular element, step, or function is an essential or critical element that must be included within the scope of the claims. In addition, unless the exact words "means for..." or "step for..." are expressly used in a particular claim and followed by a participatory phrase identifying the function, no claim is intended to invoke 35 U.S.C. §112(f) for any additional claim or claim element. Terms used in the claims (such as (but not limited to) "mechanism", "module", "device", "unit", "component", "element", "member", "device", "machine", "system", "processor", "processing device", or "controller") should be understood and intended to refer to structures known to those skilled in the relevant art and are further modified or enhanced by the features of the claims themselves, and are not intended to invoke 35 U.S.C. §112(f). Even under the broadest reasonable interpretation, in accordance with this paragraph of the specification, the claims are not intended to invoke 35 U.S.C. §112(f) in the absence of the above specific language.
[0106] The present disclosure may be embodied in other specific forms without departing from its spirit or essential characteristics. For example, each new structure described herein may be modified to accommodate specific local changes or requirements while retaining their basic configuration or structural relationship to each other, or while performing the same or similar functions described herein. Therefore, the present embodiments should be considered illustrative rather than restrictive in all respects. Therefore, the scope of the invention should be determined by the appended claims rather than by the foregoing description. Therefore, all changes within the meaning and equivalent scope of the claims should be included in the claims. In addition, the various elements in the claims are not well-known, conventional or traditional. Instead, the claims are directed to the unconventional inventive concepts described in the specification.
Claims
1. A method comprising: capturing an image of a target by a camera of an image capture module attached to the vehicle, wherein the target is at least one calibration pole attached to a roadway; Determine the number of horizontal pixels within the horizontal spacing of the target image; Determine the horizontal object pixel size according to the number of horizontal pixels; determining the number of vertical pixels within the vertical spacing of the image; and Compare the number of horizontal pixels to the number of vertical pixels.
2. The method according to claim 1, further comprising: In response to comparing the number of horizontal pixels to the number of vertical pixels, determining that the number of horizontal pixels matches the number of vertical pixels; and Calculate the vehicle encoder rate based on field calibration evaluation.
3. The method of claim 2, wherein the field calibration evaluation includes performing a field calculation of the lateral object pixel size based on the number of lateral pixels and the determined number of pixels. 4 . The method of claim 2 , wherein the image further comprises programming a vehicle encoder rate into a vehicle encoder attached to a wheel of the vehicle. 5 . The method of claim 4 , wherein the vehicle encoder rate is based on a second relationship among the trigger factor, the wheel circumference, and the object pixel size.
6. The method according to claim 1, further comprising: in response to comparing the number of horizontal pixels to the number of vertical pixels, determining that the number of horizontal pixels does not match the number of vertical pixels; calibrating the image capture module using the new calibration information to resolve the difference between the number of horizontal pixels and the number of vertical pixels; and Transfer new calibration information.
7. The method according to claim 1, further comprising: Under constant lighting conditions, focusing the camera of the image capture module on the target; as well as The maximum contrast between two pixels is obtained to identify the boundary between the light and dark parts of the object.
8. The method of claim 1, wherein: The target consists of multiple patterns with alternating black and white sections; The lateral spacing indicates the length of the marker bar coupled to the target; as well as The longitudinal pitch indicates the length of at least one of the plurality of black and white portions.
9. A calibration system comprising: at least two end plates; a target attached to at least two end plates; an image capture module attached to the fixture, wherein the image capture module includes a camera and the camera is operable to capture an image of the target; as well as One or more controllers are communicatively coupled to the vehicle encoder and the camera, wherein the one or more controllers are operable to: Determine the number of horizontal pixels within the horizontal spacing of the target image; Determine the horizontal object pixel size according to the number of horizontal pixels; determining the number of vertical pixels within the vertical spacing of the image; and Compare the number of horizontal pixels to the number of vertical pixels.
10. The calibration system of claim 9, wherein the one or more controllers are further configured to: In response to comparing the number of horizontal pixels to the number of vertical pixels, determining that the number of horizontal pixels matches the number of vertical pixels; and Calculate the vehicle encoder rate based on field calibration evaluation.
11. The calibration system of claim 10, wherein the in-situ calibration evaluation includes an in-situ calculation of a lateral object pixel size based on the number of lateral pixels and the determined number of pixels. 12 . The calibration system of claim 10 , wherein the one or more controllers are further configured to program a vehicle encoder rate into a vehicle encoder attached to a wheel of the vehicle.
13. The calibration system of claim 12, wherein the vehicle encoder rate is based on a relationship between a trigger factor, a wheel circumference, and an object pixel size.
14. The calibration system of claim 9, wherein the one or more controllers are further operable to: in response to comparing the number of horizontal pixels to the number of vertical pixels, determining that the number of horizontal pixels does not match the number of vertical pixels; calibrating the image capture module using the new calibration information to resolve the difference between the number of horizontal pixels and the number of vertical pixels; and Transfer new calibration information.
15. The calibration system of claim 9, wherein the one or more controllers are further operable to program a vehicle encoder rate into a vehicle encoder attached to a wheel of the vehicle.
16. The calibration system of claim 9, wherein the camera of the image capture module is further operable to adjust its focus on the target under constant lighting conditions to obtain a maximum contrast between two pixels identifying a boundary between light and dark portions of the target.
17. The calibration system of claim 9, wherein: The target consists of multiple patterns with alternating black and white sections; The lateral spacing represents the width of the end plate coupled to the target; and The longitudinal pitch indicates the length of one portion among a plurality of black and white portions.
18. One or more computer-readable storage media containing instructions that, when executed by a processor, cause the processor to perform operations including: The image of the target is captured by a camera of an image capture module attached to the vehicle, wherein: The target is a calibrated pole attached to the road; Determine the number of horizontal pixels within the horizontal spacing of the target image; Determine the horizontal object pixel size according to the number of horizontal pixels; determining the number of vertical pixels within the vertical spacing of the image; and Compare the number of horizontal pixels to the number of vertical pixels.
19. The one or more computer readable storage media of claim 18, wherein the operations further comprise: In response to comparing the number of horizontal pixels to the number of vertical pixels, determining that the number of horizontal pixels matches the number of vertical pixels; and Calculate the vehicle encoder rate based on field calibration evaluation.
20. The one or more computer-readable storage media of claim 19, wherein the in-situ calibration assessment comprises an in-situ calculation of a lateral object pixel size based on the number of lateral pixels and the determined number of pixels.
21. The one or more computer readable storage media of claim 17, wherein the image further comprises programming a vehicle encoder rate into a vehicle encoder attached to a wheel of the vehicle.
22. The one or more computer readable storage media of claim 21, wherein the vehicle encoder rate is based on a relationship between a trigger factor, a wheel circumference, and an object pixel size.
23. The one or more computer-readable storage media of claim 18, wherein the one or more controllers are further operable to: in response to comparing the number of horizontal pixels to the number of vertical pixels, determining that the number of horizontal pixels does not match the number of vertical pixels; calibrating the image capture module using the new calibration information to resolve the difference between the number of horizontal pixels and the number of vertical pixels; and Transfer new calibration information.
24. The one or more computer-readable storage media of claim 18, operations further comprising programming a vehicle encoder rate into a vehicle encoder attached to a wheel of the vehicle.
25. The one or more computer-readable storage media of claim 18, wherein operations further include focusing a camera of the image capture module on the target under constant lighting conditions to obtain a maximum contrast between two pixels identifying a boundary between a light portion and a dark portion of the target.
26. The one or more computer-readable storage media of claim 18, wherein: The target consists of multiple patterns with alternating black and white sections; The lateral spacing represents the width of the end plate coupled to the target; and The longitudinal pitch indicates the length of one portion among a plurality of black and white portions.
27. A calibration rod comprising: at least two end plates attached to the roadway including marking strips perpendicular to the roadway; as well as A target attached to at least two end plates includes visual markings alternating along the length of the target, wherein the calibration rod is a rigid material.
28. The system of claim 27, wherein at least two end plates are magnetically attached to the roadway and are usable with a variety of track sizes, wherein the variety of track sizes include 112 lb / yd track, 132 lb / yd track, 136 lb / yd track, or lb / yd track.
29. The system of claim 27, wherein the marking strip perpendicular to the road is a black strip.
30. The system of claim 27, wherein: Each of the alternating visual markings along the length of the calibration rod has a same length, wherein the alternating visual markings along the length of the calibration rod alternate between at least two colors, and wherein the at least two colors include black and white.
Citation Information
Patent Citations
Testing 3D imaging systems
CN107727364A
Detection system for straddle type monorail inspection engineering vehicle
CN111256586A
Systems and methods for calibrating image capturing modules
US20210287402A1
