A method and system for constructing a camera observation model for automatic driving of horizontal transport locomotives in rail engineering.

Abstract

This invention discloses a method and system for constructing a camera observation model for automatic driving of a horizontal transport locomotive in rail engineering, relating to the field of image processing technology. The method includes the following steps: determining the camera coordinate system and imaging plane coordinate system of a pinhole camera; generating similarity characteristics between the camera coordinate system and imaging plane coordinate system based on camera parameters; constructing a pixel plane and generating a pixel plane coordinate system; generating an initial model based on the similarity characteristics; determining a camera spherical model and its corresponding reference plane; determining the optical axis offset between the target image and the initial model based on the camera model parameters; constructing a camera distortion model based on the reference plane and the camera optical axis offset; and generating a camera observation model based on the initial model. This invention enables accurate projection of a three-dimensional target onto image pixels, providing reliable visual modeling support for the environmental perception module of the automatic driving system of a horizontal transport locomotive in rail engineering.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and system for constructing a camera observation model for automatic driving of a horizontal transport locomotive in rail engineering. Background Technology

[0002] In the driving scenario of horizontal transport locomotives in rail engineering, the traditional manual driving mode has problems such as high workload, high risk of misjudgment, and insufficient operation accuracy, which is no longer suitable for the high-precision and high-safety operation and maintenance requirements of modern rail transit.

[0003] Autonomous driving technology has provided an opportunity for the upgrade of engineering horizontal transport locomotives. Environmental perception is the core support of the autonomous driving system, and cameras, with their advantages of low cost and high information density, have become the core equipment of the autonomous driving perception system of rail engineering horizontal transport locomotives. They need to achieve accurate perception of key targets such as tracks, switches, and obstacles.

[0004] Traditional camera observation model research has largely focused on general autonomous driving and robotic SLAM, with very limited models available for specialized engineering horizontal transport locomotives. Furthermore, existing camera observation models still suffer from cumbersome model construction processes and insufficient compensation for optical axis misalignment when facing the complex conditions of track engineering, making it difficult to simultaneously meet the requirements of high precision and real-time performance.

[0005] Therefore, providing a method and system for constructing a camera observation model for automatic driving of horizontal transport locomotives in rail engineering to overcome the difficulties in the existing technology is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] In view of this, the present invention provides a method and system for constructing a camera observation model for automatic driving of a horizontal transport locomotive in rail engineering, which can realize the accurate projection of three-dimensional spatial targets to image pixels, and provide reliable visual modeling support for the environmental perception module of the automatic driving system of the horizontal transport locomotive in rail engineering.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: A method for constructing a camera observation model for automatic driving of a horizontal transport locomotive in rail engineering includes the following steps: Determine the camera coordinate system and imaging plane coordinate system of the pinhole camera, and generate similarity characteristics between the camera coordinate system and imaging plane coordinate system by combining camera parameters; Construct a pixel plane and generate a pixel plane coordinate system, and generate an initial model by combining similarity characteristics; Determine the camera spherical model and its corresponding reference plane, and determine the optical axis offset between the target image and the initial model based on the camera model parameters; A camera distortion model is constructed based on the reference plane and the camera optical axis offset, and a camera observation model is generated by combining the initial model.

[0008] Optionally, generating the initial model includes: Let physical imaging coordinates Convert to pixel coordinate system The conversion process includes scaling and origin adjustment. Translation, where scaling is used to convert physical dimensions into pixel units, including... On the axis Scaling and On the axis The scaling factor is expressed as: , ; make , Construct the initial model, with the expression: , in, Let be a point in the camera coordinate system with coordinates (X, Y, Z). This refers to the camera's internal parameters.

[0009] Optionally, determining the optical axis offset includes: The lens is equivalent to a sphere centered on the optical center. A camera sphere model is constructed, and the intersection of the camera coordinate system and the outer surface of the camera sphere model is obtained. Using the intersection point as the tangent point, obtain the target plane that is tangent to the camera sphere model, and perform image normalization on the target plane to obtain the reference plane. The reference plane is the set of three-dimensional points corresponding to the image pixels in the coordinate system of the camera sphere model. Based on the actual model parameters obtained from camera calibration, the optical axis offset of the target image relative to the initial model is calculated, including the optical axis tilt angle and offset orientation.

[0010] Optionally, constructing a camera distortion model includes: Normalize the points in the camera coordinate system to the plane with z=1, and convert the normalized plane to polar coordinates. Radial and tangential distortions are introduced, and a camera distortion model is constructed based on these distortions. The radial distortion is expressed in polar coordinates. Changes, tangential distortion is The changes.

[0011] Optionally, radial distortion is represented as: , , in, For normalized coordinates, These are the model coefficients; Tangential distortion is represented as: , , in, These are the coefficients of the tangential distortion model.

[0012] Optionally, the generated camera observation model includes: Projecting three-dimensional spatial points onto a normalized camera coordinate system yields a normalized polar coordinate representation. Input the normalized polar coordinate points into the camera distortion model, calculate and remove radial and tangential distortion, and generate distortion-free polar coordinate points. Using the camera's intrinsic parameter matrix, the distortion-free polar coordinate points are projected onto the pixel plane.

[0013] A camera observation model construction system for automatic driving of a horizontal transport locomotive in rail engineering is provided to implement the camera observation model construction method for automatic driving of a horizontal transport locomotive in rail engineering as described above. The system includes a data preprocessing module, an initial model generation module, an offset calculation module, and a model construction module connected in sequence. Data preprocessing module: Determines the camera coordinate system and imaging plane coordinate system of the pinhole camera, and generates similarity characteristics between the camera coordinate system and imaging plane coordinate system by combining camera parameters; Initial model generation module: Constructs a pixel plane and generates a pixel plane coordinate system, and generates an initial model by combining similarity characteristics; Offset calculation module: Determines the camera sphere model and its corresponding reference plane, and determines the optical axis offset between the target image and the initial model based on the camera model parameters; Model building module: Constructs a camera distortion model based on the reference plane and camera optical axis offset, and generates a camera observation model by combining the initial model.

[0014] As can be seen from the above technical solutions, compared with the prior art, the present invention provides a camera observation model construction method and system for automatic driving of rail engineering horizontal transport locomotives, which has the following beneficial effects: the present invention can be adapted to different types of industrial cameras, can cope with complex working conditions such as vibration and light changes during the operation of rail engineering horizontal transport locomotives, and can be directly integrated into the perception and decision-making system of automatic driving of rail engineering horizontal transport locomotives, providing accurate visual input for subsequent target detection, path planning, obstacle avoidance control and other modules. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0016] Figure 1 This is a flowchart of a camera observation model construction method for automatic driving of a horizontal transport locomotive in rail engineering, as disclosed in this invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] Reference Figure 1 As shown, this invention discloses a method for constructing a camera observation model for automatic driving of a horizontal transport locomotive in rail engineering, comprising the following steps: Determine the camera coordinate system and imaging plane coordinate system of the pinhole camera, and generate similarity characteristics between the camera coordinate system and imaging plane coordinate system by combining camera parameters; Construct a pixel plane and generate a pixel plane coordinate system, and generate an initial model by combining similarity characteristics; Determine the camera spherical model and its corresponding reference plane, and determine the optical axis offset between the target image and the initial model based on the camera model parameters; A camera distortion model is constructed based on the reference plane and the camera optical axis offset, and a camera observation model is generated by combining the initial model.

[0019] Furthermore, the coordinate system is constructed as follows: Through camera calibration experiments, feature point matching of calibration board images is used to solve the camera's intrinsic parameters and determine the spatial position of the camera's optical center, which serves as the origin of the camera's coordinate system. With the shooting direction of the camera lens as the positive Z-axis, the initial attitude of the camera is obtained through the camera attitude sensor to ensure that the Z-axis is consistent with the optical axis of the lens; With the horizontal direction of the camera sensor as the positive X-axis and the vertical direction as the positive Y-axis, the coordinate axis directions are calibrated using a camera calibration tool to ensure that the X, Y, and Z axes are perpendicular to each other, thus forming a standard right-handed coordinate system and obtaining the camera coordinate system. With the intersection of the camera's optical axis and the imaging plane as the origin, let the x-axis be parallel to the X-axis, the y-axis be parallel to the Y-axis, and the imaging plane be perpendicular to the Z-axis, thus forming an imaging plane coordinate system.

[0020] Furthermore, camera calibration includes: Secure the camera to the designated position on the track engineering horizontal transport locomotive, ensuring a firm installation to prevent loosening during operation; adjust the camera's orientation to ensure the shooting range covers the critical areas required for the track engineering horizontal transport locomotive's operation. Use a checkerboard calibration board to ensure that the surface of the calibration board is flat and non-reflective. Choose an environment with uniform and unobstructed light for calibration to avoid direct sunlight or excessively dark light that may cause image blurring. Take no fewer than 20 images of the calibration board in different poses. The poses of the calibration board need to be diverse, and ensure that the feature points of the calibration board are clearly distinguishable in each image, without occlusion or blur. Using the OpenCV calibration module or Matlab camera calibration tool, import the acquired calibration board image, extract feature points, and solve for the camera's intrinsic, extrinsic, and distortion coefficients. Verify the calibration results, calculate the reprojection error, and if the reprojection error exceeds the threshold, re-acquire the image for calibration. Save all the parameters obtained from the calibration.

[0021] Furthermore, the similarity between the camera coordinate system and the imaging plane coordinate system is based on the pinhole imaging principle of a pinhole camera, that is, the projection of a point in three-dimensional space onto the camera coordinate system satisfies a similar triangle relationship, namely: , The mapping relationship between the imaging plane coordinates and the camera coordinate system coordinates can be obtained by rearranging the expressions: , .

[0022] Furthermore, generating the initial model includes: Let physical imaging coordinates Convert to pixel coordinate system The conversion process includes scaling and origin adjustment. Translation, where scaling is used to convert physical dimensions into pixel units, including... On the axis Scaling and On the axis The scaling factor is expressed as: , ; make , Construct the initial model, with the expression: , in, Let be a point in the camera coordinate system with coordinates (X, Y, Z). This refers to the camera's internal parameters.

[0023] Furthermore, without knowing the details... In the case of the camera's position in the camera coordinate system, the camera needs to be located in its position in world coordinates. The expression is obtained by performing the corresponding attitude transformation: ,in, .

[0024] Further, determining the optical axis offset includes: The lens is equivalent to a sphere centered on the optical center. A camera sphere model is constructed, and the intersection of the camera coordinate system and the outer surface of the camera sphere model is obtained. Using the intersection point as the tangent point, obtain the target plane that is tangent to the camera sphere model, and perform image normalization on the target plane to obtain the reference plane. The reference plane is the set of three-dimensional points corresponding to the image pixels in the coordinate system of the camera sphere model. Based on the actual model parameters obtained from camera calibration, the optical axis offset of the target image relative to the initial model is calculated, including the optical axis tilt angle and offset orientation.

[0025] Furthermore, constructing the camera sphere model includes: Determine the center of the sphere and align the center of the camera sphere model with the origin of the camera coordinate system, i.e., the camera's optical center; Let the radius r of the sphere be equal to the effective focal length of the camera lens, i.e., r = f, to ensure that the optical properties of the sphere surface are equivalent to those of the camera lens; The equations of the camera sphere model in the camera coordinate system are: .

[0026] Furthermore, by determining the intersection point of the Z-axis of the camera coordinate system and the outer surface of the sphere model, and substituting the coordinate characteristics on the Z-axis into the sphere equation, the coordinates of the intersection point can be obtained, which is the tangent point.

[0027] Furthermore, the optical properties of camera lenses can cause imaging distortion. Radial distortion is caused by the non-uniform curvature of the lens surface, while tangential distortion is caused by the lens not being parallel to the imaging plane. Therefore, constructing a camera distortion model includes: Normalize the points in the camera coordinate system to the plane with z=1, and convert the normalized plane to polar coordinates. Radial and tangential distortions are introduced, and a camera distortion model is constructed based on these distortions. The radial distortion is expressed in polar coordinates. Changes, tangential distortion is The changes.

[0028] Furthermore, radial distortion is the most prevalent type of distortion in camera imaging, manifesting as pixels at the image edges shifting towards the center or outwards, as shown below: , , in, For normalized coordinates, These are the model coefficients; Tangential distortion is caused by the camera lens not being parallel to the imaging plane, or by the lens being mounted at an angle. It manifests as a shift of image pixels along the tangential direction, as shown below: , , in, These are the coefficients of the tangential distortion model.

[0029] Furthermore, during camera imaging, radial and tangential distortions coexist. Therefore, it is necessary to integrate the radial and tangential distortion models to obtain a complete camera distortion model, achieving comprehensive compensation for imaging distortion. The integration method is as follows: normalized coordinates are first corrected for radial distortion, then for tangential distortion, finally yielding the distortion-free normalized coordinates, expressed as: , .

[0030] Furthermore, the generation of camera observation models includes: Projecting three-dimensional spatial points onto a normalized camera coordinate system yields a normalized polar coordinate representation. Input the normalized polar coordinate points into the camera distortion model, calculate and remove radial and tangential distortion, and generate distortion-free polar coordinate points. Using the camera's intrinsic parameter matrix, the distortion-removed polar coordinate points are projected onto the pixel plane, with the corresponding expression being: , .

[0031] A camera observation model construction system for automatic driving of a horizontal transport locomotive in rail engineering is provided to implement the camera observation model construction method for automatic driving of a horizontal transport locomotive in rail engineering as described above. The system includes a data preprocessing module, an initial model generation module, an offset calculation module, and a model construction module connected in sequence. Data preprocessing module: Determines the camera coordinate system and imaging plane coordinate system of the pinhole camera, and generates similarity characteristics between the camera coordinate system and imaging plane coordinate system by combining camera parameters; Initial model generation module: Constructs a pixel plane and generates a pixel plane coordinate system, and generates an initial model by combining similarity characteristics; Offset calculation module: Determines the camera sphere model and its corresponding reference plane, and determines the optical axis offset between the target image and the initial model based on the camera model parameters; Model building module: Constructs a camera distortion model based on the reference plane and camera optical axis offset, and generates a camera observation model by combining the initial model.

[0032] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for constructing a camera observation model for automatic driving of a horizontal transport locomotive in rail engineering, characterized in that, Includes the following steps: Determine the camera coordinate system and imaging plane coordinate system of the pinhole camera, and generate similarity characteristics between the camera coordinate system and imaging plane coordinate system by combining camera parameters; Construct a pixel plane and generate a pixel plane coordinate system, and generate an initial model by combining similarity characteristics; Determine the camera spherical model and its corresponding reference plane, and determine the optical axis offset between the target image and the initial model based on the camera model parameters; A camera distortion model is constructed based on the reference plane and the camera optical axis offset, and a camera observation model is generated by combining the initial model.

2. The method for constructing a camera observation model for automatic driving of a horizontal transport locomotive in rail engineering according to claim 1, characterized in that, Generating the initial model includes: Let physical imaging coordinates Convert to pixel coordinate system The conversion process includes scaling and origin adjustment. Translation, where scaling is used to convert physical dimensions into pixel units, including... On the axis Scaling and On the axis The scaling factor is expressed as: ， ； make , Construct the initial model, with the expression: ， in, Let be a point in the camera coordinate system with coordinates (X, Y, Z). This refers to the camera's internal parameters.

3. The method for constructing a camera observation model for automatic driving of a horizontal transport locomotive in rail engineering according to claim 1, characterized in that, Determining the optical axis offset includes: The lens is equivalent to a sphere centered on the optical center. A camera sphere model is constructed, and the intersection of the camera coordinate system and the outer surface of the camera sphere model is obtained. Using the intersection point as the tangent point, obtain the target plane that is tangent to the camera sphere model, and perform image normalization on the target plane to obtain the reference plane. The reference plane is the set of three-dimensional points corresponding to the image pixels in the coordinate system of the camera sphere model. Based on the actual model parameters obtained from camera calibration, the optical axis offset of the target image relative to the initial model is calculated, including the optical axis tilt angle and offset orientation.

4. The method for constructing a camera observation model for automatic driving of a horizontal transport locomotive in rail engineering according to claim 1, characterized in that, Constructing a camera distortion model includes: Normalize the points in the camera coordinate system to the plane with z=1, and convert the normalized plane to polar coordinates. Radial and tangential distortions are introduced, and a camera distortion model is constructed based on these distortions. The radial distortion is expressed in polar coordinates. Changes, tangential distortion is The changes.

5. The method for constructing a camera observation model for automatic driving of a horizontal transport locomotive in rail engineering according to claim 4, characterized in that, Radial distortion is represented as: ， ， in, For normalized coordinates, These are the model coefficients; Tangential distortion is represented as: ， ， in, These are the coefficients of the tangential distortion model.

6. The method for constructing a camera observation model for automatic driving of a horizontal transport locomotive in rail engineering according to claim 1, characterized in that, The generated camera observation model includes: Projecting three-dimensional spatial points onto a normalized camera coordinate system yields a normalized polar coordinate representation. Input the normalized polar coordinate points into the camera distortion model, calculate and remove radial and tangential distortion, and generate distortion-free polar coordinate points. Using the camera's intrinsic parameter matrix, the distortion-free polar coordinate points are projected onto the pixel plane.

7. A camera observation model construction system for automatic driving of a horizontal transport locomotive in rail engineering, used to implement the camera observation model construction method for automatic driving of a horizontal transport locomotive in rail engineering as described in any one of claims 1-6, characterized in that, It includes a data preprocessing module, an initial model generation module, an offset calculation module, and a model building module, which are connected in sequence. Data preprocessing module: Determines the camera coordinate system and imaging plane coordinate system of the pinhole camera, and generates similarity characteristics between the camera coordinate system and imaging plane coordinate system by combining camera parameters; Initial model generation module: Constructs a pixel plane and generates a pixel plane coordinate system, and generates an initial model by combining similarity characteristics; Offset calculation module: Determines the camera sphere model and its corresponding reference plane, and determines the optical axis offset between the target image and the initial model based on the camera model parameters; Model building module: Constructs a camera distortion model based on the reference plane and camera optical axis offset, and generates a camera observation model by combining the initial model.

Images