Method for mounting camera module and mobile platform

Abstract

A kind of installation method of camera module and mobile platform, for automatic driving or intelligent driving.It includes: camera module is installed on mobile platform, camera module is parallel with the ground where mobile platform is;Wherein camera module includes lens group and image sensor, lens group includes at least one lens;The projection of the center of lens group in image sensor plane is first position;The distance between first position and image sensor center is greater than first threshold value, and first threshold value is greater than 0.In this way, since camera module can be parallel with the ground where mobile platform is, it can be guaranteed that the motion direction is perpendicular to image sensor plane, the accuracy and robustness of camera motion estimation can be improved, to reduce target positioning error, and without tilting installation camera module, the required detection range can be met.The method can be applied to Internet of vehicles, such as vehicle external connection V2X, inter-vehicle communication long term evolution LTE-V, vehicle-to-vehicle V2V, etc.

Description

[0001] This application is a divisional application. The application number of the original application is 202180002256.4, and the original application date is March 24, 2021. The entire content of the original application is incorporated into this application by reference. Technical Field

[0002] The present application relates to the field of sensor technology, and in particular to a camera module installation method and a mobile platform. Background Art

[0003] The lens assembly and image sensor are two key components of a camera module. The lens assembly can be a set of convex or concave lenses, while the image sensor is the imaging surface. The image sensor converts light transmitted from the lens assembly into electrical signals, which are then converted internally through analog-to-digital conversion to form digital signals, forming an image.

[0004] Currently, camera modules can be used to detect the surrounding environment and then perform three-dimensional reconstruction of the surrounding environment to achieve target positioning. For example, when a camera module is used in an in-vehicle perception system, the functions of the camera module can mainly include the detection and recognition of vehicles, pedestrians, general obstacles, lane lines, road markings, traffic signs and other targets around the vehicle, as well as the distance and speed measurement of the above-mentioned detected targets, and the estimation of camera motion (i.e., vehicle motion) (including rotation and translation), and the three-dimensional reconstruction of the surrounding environment based on this, thereby achieving target positioning.

[0005] The installation of camera modules on mobile platforms, such as vehicles and other moving objects, also needs to consider the accuracy and robustness of camera motion estimation to reduce target positioning errors. Summary of the Invention

[0006] The present application provides a camera module installation method and mobile platform to solve the problem that in order to meet the required detection range, the installation method will affect the accuracy and robustness of the camera motion estimation, thereby leading to large target positioning errors.

[0007] In a first aspect, the present application provides a method for installing a camera module, which may specifically include: installing the camera module on a mobile platform, wherein the camera module is parallel to the ground where the mobile platform is located; wherein the camera module includes a lens group and an image sensor, and the lens group includes at least one lens; the center of the lens group is projected on the plane of the image sensor as a first position; the distance between the first position and the center of the image sensor is greater than a first threshold, and the first threshold is greater than 0.

[0008] Through the above-mentioned installation method, since the camera module can be parallel to the ground where the mobile platform is located, the movement direction can be ensured to be perpendicular to the image sensor plane, which can improve the accuracy and robustness of the camera motion estimation, thereby reducing the target positioning error. In addition, by ensuring that the distance between the first position of the projection of the center of the lens group on the image sensor plane and the center of the image sensor is greater than the first threshold, the perception ability of the part above or below the center of the lens group can be increased, thereby meeting the required detection range without the need to install the camera module at an angle.

[0009] In one possible design, the optical axis of the lens assembly and the normal of the image sensor plane are both parallel to the ground on which the mobile platform rests. This ensures that the direction of motion is perpendicular to the image sensor plane, thereby improving the accuracy and robustness of camera motion estimation.

[0010] In one possible design, the line connecting the first position and the center of the image sensor is perpendicular to the horizontal axis of a first coordinate system. The first coordinate system is a rectangular coordinate system established with the center of the image sensor as its origin, with the horizontal positive axis pointing rightward and the vertical positive axis pointing downward. This ensures that the first position is separated from the center of the image sensor only in the vertical direction, and not in the horizontal direction, thereby ensuring the detection performance of the camera module.

[0011] In one possible design, the distance between the first position and the center of the image sensor is related to a first angle, which is the angle between the bisector of the vertical field of view (VFOV) angle of the camera module and the optical axis of the lens assembly. Specifically, the first angle can represent the direction of the VFOV of the camera module, and thus can indicate the position of the actual detection range. The desired detection range can be obtained by adjusting the first angle. In this way, the distance between the first position and the center of the image sensor can be determined according to actual detection requirements.

[0012] In one possible design, the first angle is greater than 0 degrees and less than or equal to half the supplementary angle of the camera module's VFOV. The degree of the first angle to be used can be determined based on the actual required detection range to determine the direction of the camera module's VFOV, and further determine the distance between the first position and the center of the image sensor to meet actual detection requirements.

[0013] In one possible design, the first position is higher than the center of the image sensor, thereby increasing the perception capability of the portion below the center of the lens group, that is, more information in this area that was originally not imaged on the image sensor can be received by the image sensor through the lens group; or, the first position is lower than the center of the image sensor, thereby increasing the perception capability of the portion above the center of the lens group, that is, more information in this area that was originally not imaged on the image sensor can be received by the image sensor through the lens group.

[0014] In one possible design, the absolute value of the ordinate y0 of the optical center or expanded focus FOE corresponding to the camera module in the first coordinate system satisfies the following formula:

[0015] |y0|=|f y h(θ)|

[0016] Among them, f y is the focal length of the center of the camera module, θ is the first angle, h(θ) = tan(θ), or h(θ) is a univariate Nth-order function of θ, N is an integer greater than 0, |*| is the absolute value of the parameter, tan(*) represents the tangent function, and the first coordinate system is a rectangular coordinate system established with the center of the image sensor as the origin, the horizontal right direction as the positive direction of the horizontal axis, and the vertical downward direction as the positive direction of the vertical axis.

[0017] In one possible design, the vertical coordinate y0 of the optical center or expanded focus FOE corresponding to the camera module in the first coordinate system meets the following formula:

[0018]

[0019] Among them, f y is the focal length of the center of the camera module, is the VFOV of the camera module, and the first coordinate system is a rectangular coordinate system established with the center of the image sensor as the origin, the horizontal rightward direction as the positive direction of the horizontal axis, and the vertical downward direction as the positive direction of the vertical axis.

[0020] In one possible design, the vertical coordinate y0 of the optical center or expanded focus FOE corresponding to the camera module in the first coordinate system meets the following formula:

[0021] y0=-f y θ d

[0022] Among them, f y is the focal length of the camera module center, θ d Related to θ1, θ d It can be a function g(θ1) related to θ1, is the VFOV of the camera module, and the first coordinate system is a rectangular coordinate system established with the center of the image sensor as the origin, the horizontal rightward direction as the positive direction of the horizontal axis, and the vertical downward direction as the positive direction of the vertical axis.

[0023] In one possible design, the vertical coordinate y0 of the optical center or expanded focus FOE corresponding to the camera module in the first coordinate system meets the following formula:

[0024]

[0025] Among them, f y is the focal length of the center of the camera module, θ0 is the preset degree, is the VFOV of the camera module, and the first coordinate system is a rectangular coordinate system established with the center of the image sensor as the origin, the horizontal rightward direction as the positive direction of the horizontal axis, and the vertical downward direction as the positive direction of the vertical axis.

[0026] In one possible design, the vertical coordinate y0 of the optical center or expanded focus FOE corresponding to the camera module in the first coordinate system meets the following formula:

[0027] y0=f y θ s

[0028] Among them, f y is the focal length of the camera module center, θ s Related to θ2, θ s It can be a function g(θ2) related to θ2, is the VFOV of the camera module, θ0 is a preset degree, and the first coordinate system is a rectangular coordinate system established with the center of the image sensor as the origin, the horizontal right direction as the positive direction of the horizontal axis, and the vertical downward direction as the positive direction of the vertical axis.

[0029] In a second aspect, the present application provides a mobile platform, which is equipped with a camera module parallel to the ground on which it is located, the camera module including a lens group and an image sensor, the lens group including at least one lens; wherein: the center of the lens group is projected on the plane of the image sensor as a first position; the distance between the first position and the center of the image sensor is greater than a first threshold, and the first threshold is greater than 0.

[0030] In this way, since the camera module can be parallel to the ground where the mobile platform is located, the direction of movement can be ensured to be perpendicular to the image sensor plane, which can improve the accuracy and robustness of the camera motion estimation, thereby reducing the target positioning error. In addition, by ensuring that the distance between the first position of the projection of the center of the lens group on the image sensor plane and the center of the image sensor is greater than the first threshold, the perception ability of the part above or below the center of the lens group can be increased, thereby meeting the required detection range without the need to install the camera module at an angle.

[0031] In one possible design, the optical axis of the lens assembly and the normal of the image sensor plane are both parallel to the ground on which the mobile platform rests. This ensures that the direction of motion is perpendicular to the image sensor plane, thereby improving the accuracy and robustness of camera motion estimation.

[0032] In one possible design, a line connecting the center of the lens assembly and the center of the image sensor is perpendicular to the horizontal axis of a first coordinate system. The first coordinate system is a rectangular coordinate system established with the center of the image sensor as its origin, with the horizontal axis pointing rightward and the vertical axis pointing downward. This ensures that the first position is only vertically distant from the center of the image sensor, with no horizontal distance, thereby ensuring the performance of the camera module.

[0033] In one possible design, the distance between the first position and the center of the image sensor is related to a first angle, which is the angle between the angular bisector of the vertical field of view (VFOV) of the camera module and the optical axis of the lens assembly. Specifically, the first angle can represent the direction of the camera module's VFOV, and thus indicate the position of the actual detection range. The desired detection range can be obtained by adjusting the first angle. In this way, the distance between the first position and the center of the image sensor can be determined based on actual detection requirements.

[0034] In one possible design, the first angle is greater than 0 degrees and less than or equal to half the supplementary angle of the camera module's VFOV. The degree of the first angle to be used can be determined based on the actual required detection range to determine the direction of the camera module's VFOV, and further determine the distance between the first position and the center of the image sensor to meet actual detection requirements.

[0035] In one possible design, the first position is higher than the center of the image sensor, thereby increasing the perception capability of the portion below the center of the lens group, that is, more information in this area that was originally not imaged on the image sensor can be received by the image sensor through the lens group; or, the first position is lower than the center of the image sensor, thereby increasing the perception capability of the portion above the center of the lens group, that is, more information in this area that was originally not imaged on the image sensor can be received by the image sensor through the lens group.

[0036] In one possible design, the absolute value of the ordinate y0 of the optical center or expanded focus FOE corresponding to the camera module in the first coordinate system satisfies the following formula:

[0037] |y0|=|f y h(θ)|

[0038] Among them, f y is the focal length of the center of the camera module, θ is the first angle, h(θ) = tan(θ), or h(θ) is a univariate Nth-order function of θ, N is an integer greater than 0, and the first coordinate system is a rectangular coordinate system established with the center of the image sensor as the origin, the horizontal right direction as the positive direction of the horizontal axis, and the vertical downward direction as the positive direction of the vertical axis.

[0039] In one possible design, the vertical coordinate y0 of the optical center or expanded focus FOE corresponding to the camera module in the first coordinate system meets the following formula:

[0040]

[0041] Among them, f y is the focal length of the center of the camera module, is the VFOV of the camera module, and the first coordinate system is a rectangular coordinate system established with the center of the image sensor as the origin, the horizontal rightward direction as the positive direction of the horizontal axis, and the vertical downward direction as the positive direction of the vertical axis.

[0042] In one possible design, the vertical coordinate y0 of the optical center or expanded focus FOE corresponding to the camera module in the first coordinate system meets the following formula:

[0043] y0=-f y θ d

[0044] Among them, f y is the focal length of the camera module center, θ d Related to θ1, for example, θ d It can be a function g(θ1) related to θ1, is the VFOV of the camera module, and the first coordinate system is a rectangular coordinate system established with the center of the image sensor as the origin, the horizontal rightward direction as the positive direction of the horizontal axis, and the vertical downward direction as the positive direction of the vertical axis.

[0045] In one possible design, the vertical coordinate y0 of the optical center or expanded focus FOE corresponding to the camera module in the first coordinate system meets the following formula:

[0046]

[0047] Among them, f y is the focal length of the center of the camera module, θ0 is the preset degree, is the VFOV of the camera module, and the first coordinate system is a rectangular coordinate system established with the center of the image sensor as the origin, the horizontal rightward direction as the positive direction of the horizontal axis, and the vertical downward direction as the positive direction of the vertical axis.

[0048] In one possible design, the vertical coordinate y0 of the optical center or expanded focus FOE corresponding to the camera module in the first coordinate system meets the following formula:

[0049] y0=f y θ s

[0050] Among them, f y is the focal length of the camera module center, θ s Related to θ2, θ s It can be a function g(θ2) related to θ2, is the VFOV of the camera module, θ0 is a preset degree, and the first coordinate system is a rectangular coordinate system established with the center of the image sensor as the origin, the horizontal right direction as the positive direction of the horizontal axis, and the vertical downward direction as the positive direction of the vertical axis. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 A schematic diagram of an exploded view of a camera module provided in this application;

[0052] Figure 2 A schematic diagram of the assembly position of a lens group and an image sensor in the prior art;

[0053] Figure 3 A schematic diagram of the perceivable VFOV and horizontal field of view of a camera module in the prior art;

[0054] Figure 4 A schematic diagram of the installation of a front-view camera in a vehicle-mounted surround view perception system in the prior art;

[0055] Figure 5A schematic diagram of the installation of a forward-looking camera in a vehicle-mounted forward-looking perception system in the prior art;

[0056] Figure 6 A schematic diagram of the installation of a camera module provided in this application;

[0057] Figure 7 A side view of a camera module provided in this application;

[0058] Figure 8 A projection diagram of a camera module provided in this application;

[0059] Figure 9 A projection diagram of another camera module provided for this application;

[0060] Figure 10 A schematic diagram of the installation of another camera module provided for this application;

[0061] Figure 11 A schematic diagram of the installation of another camera module provided for this application;

[0062] Figure 12 This is a schematic diagram of a method for estimating camera motion by matching feature points in two frames of images taken by a camera module, provided in this application;

[0063] Figure 13 A schematic diagram of a solution result provided for this application;

[0064] Figure 14 A schematic diagram of the impact of the camera module translation direction on translation vector estimation provided in this application. DETAILED DESCRIPTION

[0065] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0066] The embodiments of the present application provide a camera module installation method and mobile platform to solve the problem that the installation method will affect the accuracy and robustness of camera motion estimation in order to achieve the required detection range, thereby leading to large target positioning errors.

[0067] In this application, at least one refers to one or more; a plurality refers to two or more.

[0068] In the description of this application, words such as "first" and "second" are only used for the purpose of distinguishing the description and cannot be understood as indicating or implying relative importance or order.

[0069] The main components of the current camera module (camera compact module, CCM) can be as follows Figure 1The exploded view of the camera module shows the components. Figure 1 In the embodiment, the camera module mainly includes a lens group and an image sensor. The lens group may include at least one lens (lens), such as Figure 1 The lens group may include lens 1, lens 2 and lens 3. At least one lens included in the lens group may be a convex (concave) lens. Specifically, Figure 1 As shown in FIG, the lens assembly can be fixed by a lens barrel and a lens mount, with a filter disposed between the lens assembly and the lens mount. The image sensor can be a semiconductor chip comprising a photosensitive area. Specifically, the image sensor can convert light transmitted from the lens assembly into an electrical signal, which is then converted into a digital signal through internal analog-to-digital conversion to form an image.

[0070] For example, Figure 1 As shown, the camera module may further include a circuit board, which is a support for electronic components and a carrier for electrical connections between electronic components.

[0071] It should be noted that Figure 1 The number of lenses included in the lens group shown is only an example. In practice, the lens group may include more or fewer lenses, and this application does not limit this.

[0072] Currently, during the assembly process of the camera module, it is usually ensured that the center of the lens group and the center of the image sensor are aligned, for example Figure 2 The assembly position diagram of the lens group and image sensor shown in FIG. Figure 3 As can be seen from the projection diagram of the camera module shown, the projection of the center of the lens group on the image sensor plane coincides with the center of the image sensor (i.e., the centers are aligned). In this assembly method, due to the limitation of the aspect ratio of the image sensor, the vertical field of view (VFOV) that the camera module can actually perceive is usually small. For example, the perceptible VFOV and horizontal field of view (HFOV) of the camera module can be as follows Figure 3 As shown in , where VFOV is the lens group's opening angle to height, HFOV is the lens group's opening angle to width, and Figure 3 Only the height and width are shown as examples.

[0073] The camera module can be widely used in various scenarios that require pose estimation and three-dimensional reconstruction of the target object, so as to realize the detection of the surrounding environment through the camera module, perform three-dimensional reconstruction of the surrounding environment, and thus realize target positioning. For example, the camera module can be applied to a mobile platform, etc. to realize the detection of the surrounding environment of the mobile platform, etc. For example, the camera module can be applied to a vehicle, and specifically can be applied to an on-board perception system. For example, the camera module can be used for the front-view camera of an on-board forward-view perception system, and can also be used for the front-view camera, side-view camera or rear-view camera of an on-board surround-view perception system. Among them, the field of view (FOV) of the front-view camera, side-view camera or rear-view camera can be a general specification of 40 to 60 degrees, or a narrower 23 to 40 degrees, or a wider 100 to 180 degrees, etc. Exemplarily, the front-view camera, side-view camera or rear-view camera can be a monocular camera, a binocular camera or a fisheye camera.

[0074] For example, when a camera module is applied to an in-vehicle perception system, the functions of the camera module may mainly include the detection and recognition of targets such as vehicles, pedestrians, general obstacles, lane lines, road markings, traffic signs, etc. around the vehicle, as well as the distance and speed measurement of the above-mentioned detected targets, and the estimation of camera motion (including rotation and translation), as well as the three-dimensional reconstruction of the surrounding environment based on this, thereby achieving target positioning.

[0075] In a specific example, when a camera module is used in a vehicle-mounted surround view perception system, the camera module detects the surrounding environment of the vehicle body. In order to ensure that the blind spot around the vehicle body is small enough, the camera module is tilted downward to point to the ground during installation to ensure that the bottom edge of its VFOV is close to the edge of the vehicle body. For example, Figure 4 As shown in the schematic diagram of the installation of the front-view camera in the vehicle-mounted surround view perception system, assuming that the VFOV of the front-view camera is 120 degrees, in order to meet the detection range, the camera module can be tilted downward by 30 degrees (that is, the optical axis of the camera module is tilted downward by 30 degrees) to ensure that the lower edge of its VFOV is close to the front guard of the vehicle.

[0076] It should be noted that the installation principles of the side view cameras and rear view cameras in the vehicle-mounted surround view perception system are the same as the installation principles of the front view camera mentioned above, and they can be referenced to each other and will not be described in detail here.

[0077] In another specific example, when the camera module is used in a vehicle-mounted forward-view perception system, the camera module detects targets such as traffic signs and traffic lights. For example, Figure 5As shown in the front-view camera installation diagram of the vehicle-mounted front-view perception system, assuming that the VFOV of the front-view camera is 40 degrees, the horizontal downward part of the field of view is about 12 degrees due to the hood of the vehicle. In order to meet the detection range, the camera module can be tilted upward by 8 degrees (that is, the optical axis of the camera module is tilted upward by 8 degrees) during installation.

[0078] In Figure 4 and Figure 5 In the two examples shown, in order to meet the detection range, the camera module is tilted and installed. This will cause the motion direction and the image sensor plane not to be perpendicular, thereby affecting the accuracy and robustness of camera motion estimation, and further causing large target positioning errors. Based on this, the application provides a camera module installation method and a mobile platform to improve the accuracy and robustness of camera motion estimation, thereby reducing target positioning errors.

[0079] In order to more clearly describe the technical solutions of the embodiments of the application, the camera module installation method and the mobile platform provided by the embodiments of the application will be described in detail below with reference to the drawings.

[0080] The application provides a camera module installation method, which can be specifically: installing the camera module on a mobile platform, the camera module being parallel to the ground on which the mobile platform is located, for example, the installation diagram of the camera module can be as shown in Figure 6 It should be noted that Figure 6 The shape of the camera module and the installation position on the mobile platform in the above are only examples, and the application does not limit them. The camera module can include a lens group and an image sensor, the lens group including at least one lens; the projection of the center of the lens group on the image sensor plane is a first position; the distance between the first position and the center of the image sensor is greater than a first threshold, and the first threshold is greater than 0. For example, the side view of the camera module can be as shown in Figure 7 (a) or Figure 7 (b) of the above.

[0081] In an embodiment, without considering assembly errors, the center of the lens group in the existing camera module and the center of the image sensor are absolutely aligned, at this time, the projection of the center of the lens group on the image sensor plane coincides with the center of the image sensor, that is, the distance between the projection position and the center of the image sensor is 0. That is, without considering assembly errors, the first threshold in the camera module involved in the application is greater than 0.

[0082] In another embodiment, in consideration of assembly error, the center of the lens group and the center of the image sensor in an existing camera module are aligned, and there is an error value, and the center of the lens group is considered to be aligned with the center of the image sensor when the projection position of the center of the lens group on the image sensor plane is at a distance of the error value from the center of the image sensor. That is, in consideration of assembly error, the first threshold value in the camera module involved in the present application is greater than the error value.

[0083] Specifically, the camera is mounted on the mobile platform, and when the camera module is parallel to the ground on which the mobile platform is located, the optical axis of the lens group and the normal line of the image sensor plane are both parallel to the ground on which the mobile platform is located. This can improve the accuracy and robustness of camera motion estimation, thereby reducing target positioning error and improving the accuracy of target positioning. For example, the mobile platform can be a motor vehicle, a drone, a rail vehicle, a bicycle, a signal light, a speed measuring device, or a network device (such as a base station or a terminal device in various systems), etc. For example, the camera module can be mounted on a movable device such as a transportation device, a household device, a robot, a gimbal, etc. The present application does not limit the type of terminal device on which the camera module is mounted and the function of the camera module.

[0084] For example, the line connecting the first position and the center of the image sensor can be perpendicular to the horizontal axis direction in the first coordinate system, and the first coordinate system is a rectangular coordinate system with the center of the image sensor as the origin, the horizontal right direction as the positive direction of the horizontal axis, and the vertical downward direction as the positive direction of the vertical axis. That is, the projection position of the center of the lens group on the image sensor plane is at a distance from the center of the image sensor in the vertical direction, and there is no distance in the horizontal direction.

[0085] In an optional embodiment, as shown in (a) of Figure 7 , the first position can be higher than the center of the image sensor. In this case, the projection diagram of the camera module can be as shown in Figure 8 , and it can be seen from Figure 8 that the projection position of the lens group on the image sensor plane (i.e., the first position) is higher than the center of the image sensor.

[0086] Specifically, Figure 8 , the camera module shown in Figure 3 , the perception ability of the part below the center of the lens group can be increased, that is, more information that was originally unable to be imaged on the image sensor can be received by the image sensor through the lens group.

[0087] In another optional embodiment, as shown inFigure 7 As shown in (b), the first position may be lower than the center of the image sensor. In this case, the projection of the camera module may be as follows: Figure 9 As shown, in Figure 9 It can be seen that the projection position of the lens group on the image sensor plane (ie, the first position) is lower than the center of the image sensor.

[0088] Specifically, Figure 9 The camera module shown is relative to Figure 3 The camera module shown can increase the perception capability of the portion above the center of the lens group, that is, more information in this area that was originally unable to be imaged on the image sensor can be received by the image sensor through the lens group.

[0089] In one embodiment, the distance between the first position and the center of the image sensor may be associated with a first angle, where the first angle may be the angle between an angle bisector of the camera module's VFOV and the optical axis of the lens assembly. Optionally, the first angle may be greater than 0 degrees and less than or equal to half the supplementary angle of the camera module's VFOV.

[0090] The distance between the first position and the center of the image sensor is the absolute value of the vertical coordinate of the optical center or focus of expansion (FOE) corresponding to the camera module involved below in the first coordinate system.

[0091] Specifically, the absolute value of the vertical coordinate y0 of the optical center or FOE corresponding to the camera module in the first coordinate system may conform to the following formula 1:

[0092] |y0|=|f y h(θ)| Formula 1;

[0093] Among them, f y is the focal length of the camera module center, θ is the first angle, h(θ)=tan(θ), or h(θ) is a univariate Nth-order function of θ, where N is an integer greater than 0. Wherein, |*| denotes the absolute value of the parameter, and tan(*) represents the tangent function.

[0094] Optionally, the FOE may be a convergence point of optical flows on a stationary target when the camera module moves based on a mobile platform.

[0095] In the above method, the first angle may be an angle to be adjusted, that is, the detection range detected by the camera module needs to adjust the detection range corresponding to the first angle on the basis of satisfying the VFOV.

[0096] In an optional embodiment, during actual imaging, as Figure 7 (a) and Figure 8 The optical center coordinates calibrated during the camera intrinsic parameter calibration of the camera module shown are located in the upper half of the image sensor plane. The coordinates of the FOE are also in the upper half of the image sensor plane. At this time, y0 is a negative value in the first coordinate system. Based on the above formula 1, it can be obtained that y0 can meet the following formula 2:

[0097] y0=-f y h(θ) Formula 2.

[0098] In another optional embodiment, during actual imaging, as Figure 7 (b) and Figure 9 The optical center coordinates calibrated during the camera intrinsic parameter calibration of the camera module shown are located in the lower half of the image sensor plane. The coordinates of the FOE are also in the lower half of the image sensor plane. At this time, y0 is a positive value in the first coordinate system. Based on the above formula 1, it can be obtained that y0 can meet the following formula 3:

[0099] y0=f y h(θ) Formula 3.

[0100] Optionally, due to different imaging models of the camera module, the function h(θ) of θ may also be different. For example, when the imaging model of the camera module is a pinhole imaging model, h(θ) = tan(θ). For another example, when the imaging model of the camera module is a fisheye imaging model, h(θ) may be a univariate Nth-order function of θ. Optionally, the univariate Nth-order function of θ may be a univariate 9th-order function of θ, for example, h(θ) = θ(1+k1θ 2 +k2θ 4 +k3θ 6 +k4θ 8), where k1, k2, k3, and k4 are the four coefficients in the fisheye imaging model. For example, the values ​​of k1, k2, k3, and k4 can be: -1.2101823606265119, 2.348159905176264, -2.8413822488946474, and 1.3818466241138192; for another example, the values ​​of k1, k2, k3, and k4 can be: -1.1529851704803267, and 2.1144 43595798193, -2.458009210238794, 1.1606670303240054; for another example, the values ​​of k1, k2, k3, and k4 can be: -1.1741024894366126, 2.1870282871688733, -2.5272904743180695, 1.170976436497773. Of course, the values ​​of k1, k2, k3, and k4 can also be other values, which are not listed here in this application.

[0101] It should be noted that the above-mentioned 9-order unary function of θ is only an example and is not intended to limit h(θ).

[0102] It should be noted that the above examples only illustrate possible h(θ) using two imaging models, and cannot limit h(θ). h(θ) can also be represented by many other functions, which are not listed one by one in this application.

[0103] In an optional embodiment, the distance between the first position and the center of the image sensor can be related to the VFOV of the camera module. Similarly, the distance between the first position and the center of the image sensor is the absolute value of the ordinate of the optical center or FOE corresponding to the camera module mentioned below in the first coordinate system.

[0104] In one example, the vertical coordinate y0 of the optical center or FOE corresponding to the camera module in the first coordinate system may conform to the following formula 4:

[0105]

[0106] Among them, f y is the focal length of the center of the camera module, is the VFOV of the camera module.

[0107] Optionally, when the imaging model of the camera module is a pinhole imaging model, the method of Formula 4 above may be used.

[0108] In another example, the longitudinal coordinate y0 of the optical center or FOE of the camera module in the first coordinate system can satisfy the following Formula Five:

[0109] y0 = -f y θ d Formula Five.

[0110] wherein f y is the focal length of the center of the camera module, θ d is related to θ1, for example, θ d may be a function g(θ1) related to θ1, is the VFOV of the camera module.

[0111] Optionally, when the imaging model of the camera module is a fisheye imaging model, the above Formula Five can be used. At this time, θ d = θ1(1+k1θ1 2 +k2θ1 4 +k3θ1 6 +k4θ1 8 ), k1, k2, k3, k4 are four coefficients in the fisheye imaging model. For example, the values of k1, k2, k3, k4 can be -1.2101823606265119, 2.348159905176264, -2.8413822488946474, 1.3818466241138192 respectively; for another example, the values of k1, k2, k3, k4 can be -1.1529851704803267, 2.114443595798193, -2.458009210238794, 1.1606670303240054 respectively; for another example, the values of k1, k2, k3, k4 can be -1.1741024894366126, 2.1870282871688733, -2.5272904743180695, 1.170976436497773 respectively. Of course, the values of k1, k2, k3, k4 can also be other values, which will not be listed one by one here.

[0112] It should be noted that the above calculation method of θ d is only one example, and there can be other ways, which are not limited by the present application.

[0113] In specific implementation, when the camera module is the camera module shown in (a) and (b) of FIG. 1, the above Formula Four and Formula Five can be used. Figure 7 Figure 8

[0114] ​​In yet another example, the longitudinal coordinate y0 of the optical center or FOE corresponding to the camera module in the first coordinate system can satisfy the following Formula Six:

[0115]

[0116] wherein f y is the focal length of the center of the camera module, θ0 is a preset degree, is the VFOV of the camera module.

[0117] Optionally, θ0 can be 0.21 radian (rad), or θ0 can be 12 degrees, of course, θ0 can also be other degrees, which is not limited in the present application.

[0118] Optionally, when the imaging model of the camera module is a pinhole imaging model, the method of Formula Six above can be adopted.

[0119] In yet another example, the longitudinal coordinate y0 of the optical center or FOE corresponding to the camera module in the first coordinate system can satisfy the following Formula Seven:

[0120] y0 = f y θ s of Formula Seven.

[0121] wherein f y is the focal length of the center of the camera module, θ s is related to θ2, for example, θ s may be a function g(θ2) related to θ2, is the VFOV of the camera module.

[0122] Optionally, when the imaging model of the camera module is a fisheye imaging model, the method of Formula Seven above can be adopted. At this time, θ s = θ2(1+k1θ2 2 +k2θ2 4 +k3θ12 6 +k4θ2 8), k1, k2, k3, k4 are the four coefficients in the fisheye imaging model. For example, the values ​​of k1, k2, k3, k4 can be: -1.2101823606265119, 2.348159905176264, -2.8413822488946474, 1.3818466241138192; for another example, the values ​​of k1, k2, k3, k4 can be: -1.1529851704803267, 2.1144 43595798193, -2.458009210238794, 1.1606670303240054; for another example, the values ​​of k1, k2, k3, and k4 can be: -1.1741024894366126, 2.1870282871688733, -2.5272904743180695, 1.170976436497773. Of course, the values ​​of k1, k2, k3, and k4 can also be other values, which are not listed here in this application.

[0123] It should be noted that the above θ s The calculation method is only an example, and there may be many other methods, which are not limited in this application.

[0124] In specific implementation, when the camera module is Figure 7 (b) and Figure 9 When the camera module shown in is used, the methods in Formula 6 and Formula 7 above can be used.

[0125] Exemplarily, the mobile platform may be a vehicle or the like.

[0126] For example, when the camera module is installed on a vehicle, Figure 7 (a) and Figure 8 The camera module shown can be applied to the vehicle-mounted surround view perception system. Figure 7 (b) and Figure 9 The camera module shown can be applied to a vehicle-mounted forward-view perception system. In one embodiment, the camera module installation method provided in the embodiments of the present application can be used to install the camera module on a vehicle without having to tilt the camera module during installation to meet detection requirements.

[0127] In an optional embodiment, when the mobile platform is a vehicle, and the camera module installed on the vehicle is used in the vehicle-mounted surround view perception system, it is not necessary to tilt the camera module downward to point it at the ground during installation, thereby ensuring that the blind spot around the vehicle body is sufficiently small. For example, when the camera module is used as a front-view camera in the vehicle-mounted surround view perception system, assuming that the VFOV of the camera module is 120 degrees, the installation diagram of the camera module can be as follows: Figure 10 As shown. Figure 10 It can be seen that the optical axis of the camera module is parallel to the ground where the vehicle is located, which ensures that the direction of movement is perpendicular to the image sensor plane, thereby improving the accuracy and robustness of the camera motion estimation and reducing the target positioning error.

[0128] In another optional embodiment, when the mobile platform is a vehicle, and the camera module installed on the vehicle is used in the vehicle-mounted forward-view perception system, it is not necessary to install the camera module tilted upward during installation, thereby increasing the perception range of targets such as traffic signs and traffic lights. For example, when the camera module is used in the vehicle-mounted forward-view perception system, assuming that the VFOV of the camera module is 40 degrees, the installation diagram of the camera module can be as follows: Figure 11 As shown. Figure 11 It can be seen that the optical axis of the camera module is parallel to the ground where the vehicle is located, which ensures that the direction of movement is perpendicular to the image sensor plane, thereby improving the accuracy and robustness of the camera motion estimation and reducing the target positioning error.

[0129] By adopting the installation method of the camera module provided in the embodiment of the present application, since the camera module can be parallel to the ground where the mobile platform is located, the movement direction can be ensured to be perpendicular to the image sensor plane. This can improve the accuracy and robustness of the camera motion estimation, thereby reducing the target positioning error, and there is no need to install the camera module at an angle to meet the required detection range.

[0130] Based on the above description, an embodiment of the present application further provides a mobile platform, comprising a camera module, wherein the camera module comprises a lens group and an image sensor, wherein the lens group comprises at least one lens; wherein: the camera module is parallel to the ground on which the mobile platform is located; the projection of the center of the lens group on the plane of the image sensor is a first position; the distance between the first position and the center of the image sensor is greater than a first threshold, and the first threshold is greater than 0. Specifically, a detailed introduction to the camera module can be found in the relevant description involved in the above embodiments, which will not be repeated here. In an optional embodiment, the mobile platform can be, but is not limited to, a vehicle, etc.

[0131] Based on the above embodiments, the camera module is installed on the mobile platform by using the installation method of the camera module provided in the embodiments of the present application to perform target positioning, and the matching feature points in the two frames of images captured by the camera module are used to estimate the camera motion, which is an important step in the spatial three-dimensional reconstruction algorithm. Figure 12 For example, it is assumed that the motion between two frames of images I1 and I2 is calculated, that is, the rotation R and the translation t of the camera from the first frame of image I1 to the second frame of image I2 are calculated. The centers of the lens groups corresponding to the two frames of images are O1 and O2, respectively. It is considered that a feature point p1 in I1 corresponds to a feature point p2 in I2. Wherein, the matching or corresponding feature points in the two frames of images represent the same spatial three-dimensional point P in the two frames of images.

[0132] From the algebraic point of view, the geometric relationship is analyzed, and in the coordinate system of the first frame of image, the spatial position of P is: P = [X, Y, Z] T .

[0133] When the imaging model of the camera module is a pinhole imaging model, according to the pinhole imaging model, the pixel positions of the pixel points p1 and p2 of the point P in the two frames of images I1 and I2 can conform to the following formula eight:

[0134] s1p1 = KP, s2p2 = K(RP + t) Formula eight;

[0135] Wherein, K is the camera intrinsic matrix.

[0136] In the homogeneous coordinate system, a vector will be equal to itself multiplied by any non-zero constant. This is usually used to express a projection relationship. For example, p1 and s1p1 form a projection relationship, p1 and s1p1 are equal in the sense of homogeneous coordinates, or in the sense of scale, which can be written as: s1p1 ~ p1.

[0137] Then, the two projections in formula eight can be written as: p1 ~ KP, p2 ~ K(RP + t);

[0138] Let x1 = K -1 p1, x2 = K -1 p2, substituting the above formula has: x2 ~ Rx1 + t;

[0139] Both sides are simultaneously left multiplied by t^, and then simultaneously left multiplied by Then, Wherein, t = [t1, t2, t3] T ,

[0140] Since the vector t^x2 on the left side of the equation and the vector x2 are perpendicular, the inner product of the two is 0, and therefore the following formula nine is obtained:

[0141]

[0142] The above formula 9 can be called the epipolar constraint. Define the essential matrix E = t^R, which is a 3×3 matrix.

[0143] Assume that the normalized coordinates of a pair of matching points are x1=[u1, v1, 1] T , x2=[u2,v2,1] T According to the above epipolar constraints, we have:

[0144] The matrix E is expanded into a vector form: e = [e1, e2, e3, e4, e5, e6, e7, e8, e9] T ;

[0145] Then the epipolar constraint can be written in the following linear form: [u2u1, u2v1, u2, v2u1, v2v1, v2, u1, v1, 1]·e=0.

[0146] Typically, n ≥ 8 pairs of feature points can be used to estimate E. Putting all the points into one equation can be transformed into a linear equation system as shown in Formula 10:

[0147] A e =0 Formula 10

[0148] in,

[0149] It is easy to prove that the solution of the linear equations shown in the above formula 10 is the matrix A T The eigenvector corresponding to the smallest eigenvalue of A.

[0150] After solving E, the camera rotation R and translation t can be recovered through singular value decomposition (SVD). Let the SVD of E be: E = U∑V T , where U and V are orthogonal matrices and ∑ is a singular value matrix. For any E, there are two possible R and t corresponding to it as follows:

[0151]

[0152] Among them, R z (π / 2) represents the rotation matrix obtained by rotating 90° along the Z axis. Therefore, when decomposing E into R and t, there are a total of 4 possible solutions, which can be as follows Figure 13 shown.

[0153] from Figure 13 It can be seen that only the first solution ( Figure 13In (a), P has a positive depth in both cameras. Therefore, by substituting any point into the four solutions and checking the depth of the point under the two cameras, we can determine which solution is correct.

[0154] Let B = A T A, diagonalized by the orthogonal matrix H, that is, H -1 BH=diag{λ1, λ2,...,λ n}, where λ i , i=1, 2, ..., n is the eigenvalue of matrix B.

[0155] Without loss of generality, let λ1 be a simple eigenvalue, and let λ1<λ2≤λ3≤…≤λ n , and then let the corresponding eigenvector be H = [h1, h2, ..., h n ], where e is the eigenvector corresponding to the eigenvalue λ1, that is, e is in the space spanned by h1.

[0156] Let B(∈)=B+Δ B , indicating that noise is superimposed on B, where ∈ is a matrix The maximum value of , then: in

[0157] Therefore, there is |b ij |≤1. Let matrix Δ B The minimum eigenvalue is λ1(∈), and the corresponding eigenvector is: e(∈)=e+δ e Among them, δ e In {h2, h3, ..., h n}In the space formed by Zhang Cheng.

[0158] When ∈ is small enough, the error λ1(∈)-λ1 can be expanded into the series λ1(∈)-λ1=a1∈+a2∈ 2 +a3∈ 3 +…, where the linear part can be expressed as:

[0159] Let H2 = [h2, h3, ..., h n ]. It is easy to prove that there exists an (n-1)-dimensional vector g1, g2, g3, ..., such that δ e =∈H2g1+∈ 2 H2g2+∈ 3 H2g3+…, where the linear part can be expressed as: ∈H2g1=HΔH T Δ B e. Wherein, Δ can be in accordance with the following formula 11:

[0160] Δ=diag{0,(λ1-λ2) -1 ,...,(λ1-λ n ) -1} Formula 11.

[0161] Discarding the quadratic and higher-order terms in the error series, the error of the eigenvalue λ1 can be The error of the eigenvector e can be

[0162] When matrix A is a non-degenerate matrix, that is, when the rank of matrix A is 8, λ1 = 0. However, when the rank of matrix A is less than 8, the solution of matrix E is noise-sensitive. The noise mainly comes from feature point detection error, feature point matching error, quantization error, and camera intrinsic parameter calibration error. Specifically, when the rank of matrix A is less than 8, λ1 ≈ λ2. As can be seen from the above formula 11, the second term of Δ becomes infinite, which makes the estimation error of matrix E also become infinite.

[0163] The above noise impact is reflected as follows when the camera module is actually working:

[0164] From the above formula 9 and the above R and t calculation process, it can be seen that: t·(x2×Rx1)=0. That is, t and x2×Rx1 are perpendicular to each other.

[0165] like Figure 14 As shown in (a), when the translation vector t is perpendicular to the image sensor plane XY, x2×Rx1 covers a larger area (see the shaded area); Figure 14 As shown in (b), when the translation vector t is parallel to the image sensor plane XY, the x2×Rx1 coverage area is smaller (see the shaded area). When the matrix A is affected by noise, the shaded area will deviate from its original position, thereby introducing errors in the estimation of the translation vector t. Obviously, at this time Figure 14 Compared with the scene shown in (a) Figure 14 The scenario shown in (b) has higher robustness.

[0166] Based on the above analysis, it can be seen that in order to ensure the accuracy and robustness of camera motion estimation, the camera translation direction (motion direction) should be perpendicular to the image sensor plane.

[0167] Based on the above analysis, it is obvious that due to the installation method of the camera module provided by the embodiment of the present application, when the camera module is installed on a mobile platform, the camera module can be made parallel to the ground on which the mobile platform is located. Specifically, the optical axis of the lens group of the camera module and the normal of the image sensor plane can be made parallel to the ground on which the mobile platform is located, for example Figure 10 and Figure 11As shown, the camera movement direction is perpendicular to the image sensor plane, thereby improving the accuracy and robustness of camera motion estimation. That is, compared with the prior art, when the camera module is installed on a mobile platform, the camera module is installed at an angle, for example Figure 4 and Figure 5 As shown, the camera movement direction cannot be perpendicular to the image sensor plane, affecting the accuracy and robustness of the camera motion estimation. After adopting the installation method of the camera module provided in the embodiment of the present application, the accuracy and robustness of the camera motion estimation can be significantly improved.

[0168] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0169] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0170] Obviously, those skilled in the art may make various modifications and variations to this application without departing from the scope of protection of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A method for installing a camera module, characterized in that: Mounting the camera module on a mobile platform, wherein the camera module is parallel to the ground where the mobile platform is located; The camera module includes a lens group and an image sensor, and the lens group includes at least one lens; The projection of the center of the lens group on the image sensor plane is a first position; The distance between the first position and the center of the image sensor is greater than a first threshold, and the first threshold is greater than 0; In which, the distance between the first position and the center of the image sensor is related to a first angle, the first angle being the angle between the angular bisector of the vertical field of view VFOV of the camera module and the optical axis of the lens group, and the first angle being greater than 0 degrees and less than or equal to half of the supplementary angle of the VFOV of the camera module.

2. The installation method according to claim 1, wherein: The optical axis of the lens group and the normal of the image sensor plane are both parallel to the ground where the mobile platform is located.

3. The installation method according to claim 1 or 2, characterized in that: The line connecting the first position and the center of the image sensor is perpendicular to the horizontal axis in the first coordinate system. The first coordinate system is a rectangular coordinate system established with the center of the image sensor as the origin, the horizontal rightward direction as the positive direction of the horizontal axis, and the vertical downward direction as the positive direction of the vertical axis.

4. The installation method according to any one of claims 1 to 3, characterized in that: The absolute value of the vertical coordinate y0 of the optical center or expanded focus FOE corresponding to the camera module in the first coordinate system conforms to the following formula: |y0|=|f y h(θ)| Among them, f y is the focal length of the center of the camera module, θ is the first angle, h(θ) = tan(θ), or h(θ) is a univariate Nth-order function of θ, N is an integer greater than 0, and the first coordinate system is a rectangular coordinate system established with the center of the image sensor as the origin, the horizontal right direction as the positive direction of the horizontal axis, and the vertical downward direction as the positive direction of the vertical axis.

5. The installation method according to any one of claims 1 to 3, characterized in that: The vertical coordinate y0 of the optical center or expanded focus FOE corresponding to the camera module in the first coordinate system conforms to the following formula: Among them, f y is the focal length of the center of the camera module, is the VFOV of the camera module, and the first coordinate system is a rectangular coordinate system established with the center of the image sensor as the origin, the horizontal rightward direction as the positive direction of the horizontal axis, and the vertical downward direction as the positive direction of the vertical axis.

6. The installation method according to any one of claims 1 to 3, characterized in that: The vertical coordinate y0 of the optical center or expanded focus FOE corresponding to the camera module in the first coordinate system conforms to the following formula: y0=-f y i d Among them, f y is the focal length of the camera module center, θ d Related to θ1, is the VFOV of the camera module, and the first coordinate system is a rectangular coordinate system established with the center of the image sensor as the origin, the horizontal rightward direction as the positive direction of the horizontal axis, and the vertical downward direction as the positive direction of the vertical axis.

7. The installation method according to any one of claims 1 to 3, characterized in that: The vertical coordinate y0 of the optical center or expanded focus FOE corresponding to the camera module in the first coordinate system conforms to the following formula: Among them, f y is the focal length of the center of the camera module, θ0 is the preset degree, is the VFOV of the camera module, and the first coordinate system is a rectangular coordinate system established with the center of the image sensor as the origin, the horizontal rightward direction as the positive direction of the horizontal axis, and the vertical downward direction as the positive direction of the vertical axis.

8. The installation method according to any one of claims 1 to 3, characterized in that: The vertical coordinate y0 of the optical center or expanded focus FOE corresponding to the camera module in the first coordinate system conforms to the following formula: y0=f y i s Among them, f y is the focal length of the camera module center, θ s Related to θ2, is the VFOV of the camera module, θ0 is a preset degree, and the first coordinate system is a rectangular coordinate system established with the center of the image sensor as the origin, the horizontal right direction as the positive direction of the horizontal axis, and the vertical downward direction as the positive direction of the vertical axis.

9. A mobile platform, characterized in that: The camera module includes a lens group and an image sensor, and the lens group includes at least one lens; wherein: The camera module is parallel to the ground where the mobile platform is located; The projection of the center of the lens group on the image sensor plane is a first position; The distance between the first position and the center of the image sensor is greater than a first threshold, and the first threshold is greater than 0; The distance between the first position and the center of the image sensor is related to a first angle, which is the angle between the angular bisector of the vertical field of view VFOV of the camera module and the optical axis of the lens group. The first angle is greater than 0 degrees and less than or equal to half of the supplementary angle of the VFOV of the camera module.

10. The mobile platform according to claim 9, wherein: The optical axis of the lens group and the normal of the image sensor plane are both parallel to the ground where the mobile platform is located.

11. The mobile platform according to claim 9 or 10, characterized in that: The line connecting the center of the lens group and the center of the image sensor is perpendicular to the horizontal axis in the first coordinate system. The first coordinate system is a rectangular coordinate system established with the center of the image sensor as the origin, the horizontal right direction as the positive direction of the horizontal axis, and the vertical downward direction as the positive direction of the vertical axis.

12. The mobile platform according to any one of claims 9 to 11, wherein: The absolute value of the vertical coordinate y0 of the optical center or expanded focus FOE corresponding to the camera module in the first coordinate system conforms to the following formula: |y0|=|f y h(θ)| Among them, f y is the focal length of the center of the camera module, θ is the first angle, h(θ) = tan(θ), or h(θ) is a univariate Nth-order function of θ, N is an integer greater than 0, and the first coordinate system is a rectangular coordinate system established with the center of the image sensor as the origin, the horizontal right direction as the positive direction of the horizontal axis, and the vertical downward direction as the positive direction of the vertical axis.

13. The mobile platform according to any one of claims 9 to 11, wherein: The vertical coordinate y0 of the optical center or expanded focus FOE corresponding to the camera module in the first coordinate system conforms to the following formula: Among them, f y is the focal length of the center of the camera module, is the VFOV of the camera module, and the first coordinate system is a rectangular coordinate system established with the center of the image sensor as the origin, the horizontal rightward direction as the positive direction of the horizontal axis, and the vertical downward direction as the positive direction of the vertical axis.

14. The mobile platform according to any one of claims 9 to 11, wherein: The vertical coordinate y0 of the optical center or expanded focus FOE corresponding to the camera module in the first coordinate system conforms to the following formula: y0=-f y i d Among them, f y is the focal length of the camera module center, θ d Related to θ1, is the VFOV of the camera module, and the first coordinate system is a rectangular coordinate system established with the center of the image sensor as the origin, the horizontal rightward direction as the positive direction of the horizontal axis, and the vertical downward direction as the positive direction of the vertical axis.

15. The mobile platform according to any one of claims 9 to 11, wherein: The vertical coordinate y0 of the optical center or expanded focus FOE corresponding to the camera module in the first coordinate system conforms to the following formula: Among them, f y is the focal length of the center of the camera module, θ0 is the preset degree, is the VFOV of the camera module, and the first coordinate system is a rectangular coordinate system established with the center of the image sensor as the origin, the horizontal rightward direction as the positive direction of the horizontal axis, and the vertical downward direction as the positive direction of the vertical axis.

16. The mobile platform according to any one of claims 9 to 11, wherein: The vertical coordinate y0 of the optical center or expanded focus FOE corresponding to the camera module in the first coordinate system conforms to the following formula: y0=f y i s Among them, f y is the focal length of the camera module center, θ s Related to θ2, is the VFOV of the camera module, θ0 is a preset degree, and the first coordinate system is a rectangular coordinate system established with the center of the image sensor as the origin, the horizontal right direction as the positive direction of the horizontal axis, and the vertical downward direction as the positive direction of the vertical axis.

Images