RVM implementation method and system based on high-pass 8155 platform and medium

Abstract

The invention discloses an RVM implementation method and system based on a high-pass 8155 platform and a medium, and the method comprises the steps: dividing a reverse image RVM into an APK layer and a server layer, deploying the APK layer at an Android side, and deploying the server layer at a QNX side; an algorithm layer is integrated in the server layer; the method comprises the following steps: distributing an RVM layer to a QNX side by modifying display configuration of a high-pass 8155 chip, and configuring the RVM layer to be fused with a layer of an Android side; the server layer collects an original image through a fisheye camera, and superimposes a vehicle auxiliary line on the original image based on an RVM vehicle auxiliary line drawing method to obtain a first image; and a fisheye camera rapid dedistorting algorithm based on a correction coefficient is adopted to carry out dedistorting correction on the first image to obtain a corrected second image, and the corrected second image is displayed on the Android side through display configuration. According to the method, the RVM starting speed is remarkably increased, the image quality is improved, and the vehicle auxiliary line is accurately drawn.

Description

Technical Field

[0001] This invention relates to the field of RVM implementation method technology, and specifically to an RVM implementation method, system and medium based on the Qualcomm 8155 platform. Background Technology

[0002] The existing methods for implementing reversing camera (RVM) are usually deployed on the vehicle's infotainment system (i.e., the Android side), which have the following drawbacks: (1) The Android operating system is used, which has the disadvantage of slow startup time. The Android operating system takes about 12 seconds to start, and then the RVM starts. Therefore, it takes about 15 seconds for the RVM to be ready for use. If the user quickly enters the reversing position after getting into the car, the RVM will not be usable. (2) In order to save costs, traditional RVM uses ordinary cameras to collect images, which has the limitation of narrow field of view, low display resolution, unclear image, and poor image quality. (3) The traditional vehicle auxiliary line drawing has the problem of inaccuracy, which affects the user's judgment and is not very reliable.

[0003] In view of the above, this application is hereby submitted. Summary of the Invention

[0004] The technical problem this invention aims to solve is that existing methods for implementing Reversing Camera Visualization (RVM) suffer from slow startup speed, poor image quality, and inaccurate vehicle guide lines. This invention aims to provide an RVM implementation method, system, and medium based on the Qualcomm 8155 platform. This invention deploys the main RVM functions on the QNX side, and the RVM content is displayed on the Android side screen through layer configuration, significantly improving RVM startup speed. Furthermore, on the QNX side, the server layer acquires original images using a fisheye camera, draws vehicle guide lines, and performs distortion correction, improving image quality and achieving accurate drawing of vehicle guide lines.

[0005] This invention is achieved through the following technical solution:

[0006] In a first aspect, the present invention provides an RVM implementation method based on the Qualcomm 8155 platform, the method comprising:

[0007] The reversing camera RVM is divided into an APK layer and a server layer. The APK layer is deployed on the Android side, and the server layer is deployed on the QNX side. The server layer integrates an algorithm layer, which is used to provide algorithm logic for drawing vehicle auxiliary lines and performing distortion correction.

[0008] By modifying the display configuration of the Qualcomm 8155 chip, an RVM layer is assigned to the QNX side, and the RVM layer is configured to blend with the layer on the Android side.

[0009] The server layer acquires the original image using a fisheye camera, and overlays vehicle auxiliary lines onto the original image based on the RVM vehicle auxiliary line drawing method to obtain the first image. Then, it uses a fast distortion correction algorithm based on the fisheye camera with correction coefficients to correct the distortion of the first image, and obtains the corrected second image, which is then displayed on the Android side through display configuration.

[0010] Furthermore, the APK layer is used to transmit messages to the QNX-side server layer through inter-system communication when a user clicks a button, thus implementing the interaction logic;

[0011] The server layer is used to receive messages from the APK layer, messages from the CAN bus, or vehicle reverse gear information, and then call the algorithm layer for display.

[0012] Furthermore, by modifying the display configuration of the Qualcomm 8155 chip, including:

[0013] On the QNX side, a first display interface and a second display interface are configured. The first display interface is used for normal instrument content display; the second display interface is used for projection onto the Android side.

[0014] Configure the server layer to use the second display interface on the QNX side as its own display interface;

[0015] On the Android side, a third, fourth, and fifth display interface are configured. The third display interface is used for basic content display on the Android side; the fourth display interface is used for the APK layer; and the fifth display interface is used for navigation map projection, the content of which will be displayed on the instrument panel.

[0016] Furthermore, based on the RVM vehicle auxiliary line drawing method, vehicle auxiliary lines are superimposed on the original image to obtain a first image, including:

[0017] Obtain the world coordinates of the vehicle guide lines. The vehicle guide lines are points that mark a fixed distance behind the vehicle to help the driver judge distance; the world coordinates are a coordinate system with the center of the vehicle as the origin.

[0018] Transform world coordinates into camera coordinates; camera coordinates are a coordinate system with the optical center of the fisheye camera as the origin.

[0019] Based on camera coordinates, the imaging process of a fisheye camera is simulated to obtain the distorted camera coordinates;

[0020] Calculate pixel coordinates based on the distorted camera coordinates and the fisheye camera intrinsic parameter matrix;

[0021] Based on the pixel coordinates, vehicle auxiliary lines are superimposed on the original image to obtain the first image.

[0022] Furthermore, the fast distortion correction algorithm for fisheye cameras based on correction coefficients is a process of adding correction coefficients to existing fast distortion correction algorithms for fisheye cameras to obtain new texture coordinates and then rendering them.

[0023] Furthermore, a fast distortion correction algorithm based on correction coefficients for fisheye cameras is used to correct the distortion of the first image, resulting in a corrected second image, including:

[0024] Divide the distortion coefficients of the first image from the center point to the farthest point into a preset number of levels;

[0025] The base distance of the first image is divided into a preset number of parts; the base distance refers to the distance from the farthest point of the first image to the center point.

[0026] Calculate the distortion coefficients of a preset number of pixels from the center point to the farthest point to obtain the preset number of distortion coefficients;

[0027] During rendering, the distance from the current point to the center point is calculated based on the original texture coordinates of the current point and recorded as the first distance;

[0028] Divide the first distance by the base distance to obtain the ratio; take the corresponding distortion coefficient based on the ratio; and multiply the distortion coefficient by the original texture coordinates to obtain the new texture coordinates;

[0029] The new texture coordinates are then used to retrieve the pixel values ​​from the original image to obtain the corrected second image.

[0030] Furthermore, the method also includes: fine-tuning the first image and comparing whether the vehicle auxiliary lines of the RVM match the actual calibration lines.

[0031] Secondly, the present invention provides an RVM implementation system based on the Qualcomm 8155 platform, the system comprising:

[0032] The deployment unit is used to divide the reversing camera RVM into an APK layer and a server layer. The APK layer is deployed on the Android side, and the server layer is deployed on the QNX side. The server layer integrates an algorithm layer, which is used to provide algorithm logic for drawing vehicle auxiliary lines and performing distortion correction.

[0033] The configuration unit is used to allocate an RVM layer to the QNX side by modifying the display configuration of the Qualcomm 8155 chip, and to configure the RVM layer to blend with the layer on the Android side.

[0034] The rendering and distortion correction unit is used by the server layer to acquire the original image through a fisheye camera, and to overlay vehicle auxiliary lines on the original image based on the RVM vehicle auxiliary line drawing method to obtain the first image; and to perform distortion correction on the first image using a fisheye camera fast distortion correction algorithm based on the correction coefficient to obtain the corrected second image and display it on the Android side through the display configuration.

[0035] Furthermore, based on the RVM vehicle auxiliary line drawing method, vehicle auxiliary lines are superimposed on the original image to obtain a first image, including:

[0036] Obtain the world coordinates of the vehicle guide lines. The vehicle guide lines are points that mark a fixed distance behind the vehicle to help the driver judge distance; the world coordinates are a coordinate system with the center of the vehicle as the origin.

[0037] Transform world coordinates into camera coordinates; camera coordinates are a coordinate system with the optical center of the fisheye camera as the origin.

[0038] Based on camera coordinates, the imaging process of a fisheye camera is simulated to obtain the distorted camera coordinates;

[0039] Calculate pixel coordinates based on the distorted camera coordinates and the fisheye camera intrinsic parameter matrix;

[0040] Based on the pixel coordinates, vehicle auxiliary lines are superimposed on the original image to obtain the first image.

[0041] Thirdly, the present invention provides a computer-readable storage medium storing a computer program, characterized in that the computer program, when executed by a processor, implements the above-described RVM implementation method based on the Qualcomm 8155 platform.

[0042] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0043] 1. This invention relates to an RVM implementation method, system, and medium based on the Qualcomm 8155 platform. This invention deploys the main functions of RVM on the QNX side, and displays the RVM content on the Android side screen through layer configuration, which significantly improves the RVM startup speed. Furthermore, on the QNX side, the server layer acquires the original image based on a fisheye camera, draws vehicle auxiliary lines, and performs distortion correction, thereby improving the image quality and achieving accurate drawing of vehicle auxiliary lines.

[0044] 2. The RVM of this invention can provide drivers with images of the rear of the vehicle, reducing the occurrence of accidents. Therefore, speeding up the start-up is very meaningful and can avoid safety accidents caused by the RVM being unusable when the user quickly starts the vehicle and reverses.

[0045] 3. This invention uses a fisheye camera, which can display a wider field of view, avoid the danger of reversing due to insufficient field of view, and improve the product experience. At the same time, it utilizes the characteristics of fisheye cameras to design a fast distortion correction algorithm, which can meet the display requirements of RVM and save CPU (central computing unit) computing power.

[0046] 4. The vehicle auxiliary line drawing method based on physical location of the present invention accurately draws vehicle auxiliary lines and provides users with accurate distance reference. Attached Figure Description

[0047] The accompanying drawings, which are included to provide a further understanding of embodiments of the invention and form part of this application, do not constitute a limitation thereof. In the drawings:

[0048] Figure 1 This is a flowchart of the RVM implementation method based on the Qualcomm 8155 platform of the present invention;

[0049] Figure 2 This is a diagram of the RVM design framework of the present invention;

[0050] Figure 3 This is a layer distribution diagram of the overall RVM system of this invention;

[0051] Figure 4 This is a flowchart of the invention based on the RVM vehicle auxiliary line drawing method, which involves superimposing vehicle auxiliary lines on the original image to obtain a first image.

[0052] Figure 5 This is a geometric model diagram of a fisheye camera;

[0053] Figure 6 Display flowcharts for traditional cameras;

[0054] Figure 7 This is a flowchart of the fast distortion correction algorithm for fisheye cameras based on correction coefficients according to the present invention.

[0055] Figure 8 This is a schematic diagram illustrating the calculation of the correction coefficient in this invention;

[0056] Figure 9 This is a block diagram of the RVM implementation system based on the Qualcomm 8155 platform of this invention. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the embodiments and accompanying drawings. The illustrative embodiments and descriptions of this invention are only for explaining this invention and are not intended to limit this invention.

[0058] 1. Traditional Reversing Camera (RVM) deployments are on the Android platform. The average startup speed of Android devices exceeds 12 seconds, and the RVM takes an additional 1-2 seconds to prepare, which cannot meet the needs of users requiring quick vehicle startup. This invention aims to solve the problem of slow RVM startup speed. Using the method described in this invention, the RVM startup speed can be increased to 8 seconds (taking the Qualcomm 8155 platform as an example).

[0059] 2. This invention utilizes a fisheye camera as an RVM camera to overcome the limitations of the field of view. At the same time, it employs a fast distortion correction algorithm for fisheye cameras based on correction coefficients to resolve the distortion caused by fisheye cameras and improve image quality.

[0060] 3. Determine the position of the vehicle auxiliary line using world coordinates, and then calculate the position of the vehicle auxiliary line in the original image using camera intrinsic and extrinsic calibration parameters to achieve accurate drawing of the vehicle auxiliary line.

[0061] Example 1

[0062] like Figure 1 As shown, this invention provides an RVM implementation method based on the Qualcomm 8155 platform, which includes:

[0063] Step 1: Divide the reversing camera RVM into an APK layer and a server layer. Deploy the APK layer on the Android side and the server layer on the QNX side. The server layer integrates an algorithm layer, which provides algorithm logic to draw vehicle auxiliary lines and perform distortion correction.

[0064] Step 2: By modifying the display configuration of the Qualcomm 8155 chip, an RVM layer is assigned to the QNX side, and the RVM layer is configured to blend with the layer on the Android side.

[0065] Step 3: The server layer acquires the original image through a fisheye camera, and overlays vehicle auxiliary lines on the original image based on the RVM vehicle auxiliary line drawing method to obtain the first image; and uses a fisheye camera fast distortion correction algorithm based on correction coefficient to correct the distortion of the first image to obtain the corrected second image and display it on the Android side through display configuration.

[0066] In this embodiment, the RVM design framework diagram of the present invention is as follows: Figure 2 As shown, Figure 2 middle:

[0067] Hypiox is a protocol for communication between Android and QNX.

[0068] SurfaceFlinger is the graphics display interface for the Android operating system;

[0069] libDRM is the graphics driver for the Android operating system.

[0070] The layer mixer is the layer blending unit for the Qualcomm 8155 chip.

[0071] Display refers to the physical screen;

[0072] AIS Server is a middleware for camera streaming on the Qualcomm 8155 platform.

[0073] Screen:QNX is the operating system's graphical interface;

[0074] The Hypervisor is a virtual machine for the Qualcomm 8155 chip.

[0075] Specifically, the APK layer, deployed on the Android side, mainly handles user click events. When a user clicks a button, it transmits the message to the QNX-side server layer (RVM Service) via inter-system communication to implement interactive logic, such as adjusting display brightness or switching viewing angles.

[0076] The server layer (RVM Service), deployed on the QNX side, handles various interaction logics, including messages transmitted from the Android APK layer and messages on the CAN bus. The vehicle's reverse gear information is sent via CAN messages. When receiving messages from the APK layer, messages on the CAN bus, or vehicle reverse gear information, the algorithm layer is called for display.

[0077] The algorithm layer, integrated within the server layer (RVM Service), primarily provides the algorithm logic. It acquires data from the fisheye camera and performs tasks such as drawing vehicle auxiliary lines and distortion correction. The final image is displayed on the Android screen.

[0078] In this embodiment, current in-vehicle entertainment systems all have a need for content projection between different systems, such as projecting navigation information from the Android side onto the instrument panel. To facilitate content projection from different domains, the Qualcomm 8155 chip can support more flexible layer fusion by modifying its configuration. This invention utilizes this feature to enable the QNX-side RVM application to project display content onto the Android screen. The specific configuration implementation is as follows:

[0079] Assign an RVM layer to the QNX side, and configure this RVM layer to eventually merge with the layer on the Android side. This way, the RVM application content displayed on the QNX side will ultimately be shown on the Android screen, even if the Android system is not running. A diagram of the final layer is shown below. Figure 3 As shown.

[0080] Figure 3 The layer distribution of the overall RVM system is described below:

[0081] Two display interfaces are configured on the QNX side: display0 and display1. Display0 is used for normal instrument content display; display1 is used for projection onto the Android side.

[0082] Configure the server layer to use the second display interface display1 on the QNX side as its own display interface;

[0083] Three display interfaces are configured on the Android side: display2 (third display interface), display3 (fourth display interface), and display4 (fifth display interface). The third display interface is used for basic content display on the Android side; the fourth display interface is used for the APK layer; and the fifth display interface is used for navigation map projection, the content of which will be displayed on the instrument panel.

[0084] In this embodiment, vehicle auxiliary lines are points marked at fixed distances behind the vehicle to assist the driver in judging distances. A method based on physical coordinates is used to draw these auxiliary lines. Since line segments originate from points, the following description will use the drawing of a point in the physical world as an example.

[0085] The central idea is to find the position of a point in the physical world on the original image, and then overlay vehicle auxiliary lines on the original image, such as... Figure 4 As shown, step 3, based on the RVM vehicle auxiliary line drawing method, overlays vehicle auxiliary lines onto the original image to obtain the first image, including the following sub-steps:

[0086] Step A: Obtain the world coordinates of the vehicle's auxiliary lines. The world coordinates are a coordinate system with the vehicle's center as the origin.

[0087] Step B involves transforming world coordinates into camera coordinates; camera coordinates are a coordinate system with the optical center of the fisheye camera as the origin.

[0088] Let the physical coordinates (i.e., world coordinates) of the point be... The transformation between the two coordinate systems is achieved through rotation and translation matrices. The formula for calculating the camera coordinates of this point is as follows:

[0089]

[0090] Where R is the rotation matrix of the fisheye camera and T is the translation matrix of the camera, these matrices can be obtained through calibration.

[0091] Step C: Based on the camera coordinates, simulate the fisheye camera imaging process to obtain the distorted camera coordinates;

[0092] Camera coordinates were obtained Since light rays have directionality, and we only consider the angle of incidence, we can imagine the point as a plane with Z=1, i.e. ( ).

[0093] Considering that actual fisheye cameras will produce distortion during imaging, therefore ( The coordinates need to be distorted, and the distorted camera coordinates are ( , This is a two-dimensional coordinate system.

[0094] Step D: Calculate pixel coordinates based on the distorted camera coordinates and the fisheye camera intrinsic parameter matrix;

[0095]

[0096]

[0097] in, , All of these are internal references from fisheye cameras.

[0098] Step E: Based on the pixel coordinates, vehicle auxiliary lines are superimposed on the original image to obtain the first image.

[0099] The above provides the world coordinates of the point's location on the original image. Therefore, we only need to overlay the car's auxiliary lines onto the original image. This adds the car's auxiliary lines to the original image, and finally, we can uniformly remove distortion and output the result to the screen.

[0100] In this embodiment, the fast distortion correction algorithm for fisheye cameras based on correction coefficients is a process of adding correction coefficients to obtain new texture coordinates and then rendering, based on the existing fast distortion correction algorithm for fisheye cameras.

[0101] Images captured by a fisheye camera are distorted. Fisheye cameras utilize a convex lens to refract incident light, allowing a wider range of light to enter the imaging plane. The geometric model of a fisheye camera is as follows: Figure 5 As shown:

[0102] Figure 5 The red light in the image represents the incident ray with an angle of incidence of θ. Without deflection, the ray would fall on point P1 on the imaging plane. After being deflected by the lens, the ray is deflected at an angle of θd. It is this deflection that allows more image information to be displayed on the imaging plane. The display process is the distortion correction process, and the steps are as follows:

[0103] Step a: Assume the point to be displayed is P1, and the texture coordinates are ( , The texture coordinates are points in the imaging plane coordinate system. According to the pinhole imaging principle, their relationship with points in the camera coordinate system is as follows:

[0104]

[0105]

[0106] Where (u,v) are known pixel coordinates. , Since this is an intrinsic parameter of the fisheye camera, the coordinates of the incident ray in the camera coordinate system can be calculated using the formula above. , ).

[0107]

[0108]

[0109] Step b: The distortion model of a fisheye camera is usually simulated using the following formula:

[0110]

[0111] in , The distortion coefficient parameters of a fisheye camera can be read from the camera's registers, therefore only the following calculation is needed. You can get .

[0112] Step c, based on the coordinate values ​​in the camera coordinate system calculated in step a ( , To facilitate the calculation of the incident angle, Z=1 is taken, and the distance from this point to the incident angle is calculated. Distance between axes:

[0113]

[0114] Since Z=1 has already been assumed at this point, therefore .

[0115] Step d, will Substituting into step b, we get .

[0116] Step e, according to Given the focal length f, the distance O-P2 in the imaging plane coordinate system can be obtained:

[0117]

[0118] Step f: Based on geometric relationships, the u and v coordinates of point P2 are obtained as follows:

[0119]

[0120]

[0121] Step g: When drawing point P1, we use the texture coordinates of point P1 ( , ), calculate the texture coordinates of point P2 ( , (and when displaying, retrieve the color value corresponding to point P2 to achieve distortion correction).

[0122] Specifically, the traditional camera display process is as follows: Figure 6 As shown, it consists of four steps:

[0123] Step 1: Obtain the raw image from the camera.

[0124] Step 2: Load the camera image into the GPU (Graphics Acceleration Unit) through the OpenGL (General 3D rendering interface) to create a texture.

[0125] Step 3: Run the GPU's Shader (a GPU-recognizable programming language) program to process the image.

[0126] Step 4: Output the processed results to the screen.

[0127] After adding the distortion correction process, each pixel needs distortion correction during shader rendering. Since the distortion coefficients of each pixel are different, this invention proposes to first calculate the correction coefficients corresponding to all pixels, and then upload these correction coefficients to the GPU in a texture-like manner, creating a correction coefficient lookup table. When the shader is working, it looks up the correction coefficient corresponding to the current pixel in the table based on the current pixel coordinates. The entire process is as follows: Figure 7 As shown.

[0128] The advantage of this approach is that it trades space for time. Calculating the correction coefficients in advance and then looking them up in a table when needed is much more efficient than calculating them manually during use. The specific steps are as follows:

[0129] (1) Based on the distortion removal algorithm from step a to step g, it was found that the process of calculating the distortion coefficient is related to the distance of the pixel from the center of the image.

[0130] (2) Divide the distortion coefficients of the first image from the center point to the farthest point into 1000 levels; the distortion of the center point (0,0) is 0, the distortion coefficient of the farthest point (640,480) is the largest, and the distance of the farthest point from the center point is 800.

[0131] (3) Divide the base distance od (800) of the first image into 1000 equal parts; the base distance refers to the distance from the farthest point of the first image to the center point;

[0132] (4) Calculate the distortion coefficients of a preset number of pixels from the center point to the farthest point to obtain 1000 distortion coefficients; upload the 1000 distortion coefficients as textures to the GPU.

[0133] (5) During rendering, with Figure 8 Taking point m as an example, the distance from the current point to the center point is calculated based on the original texture coordinates (u,v) of the current point and denoted as the first distance om; om is the same as the on distance;

[0134] (6) Divide the first distance om by the base distance od to obtain the ratio R; take the corresponding distortion coefficient according to the ratio R, that is, the distortion coefficient of point m is equal to the distortion coefficient of point n; and multiply the distortion coefficient by the original texture coordinates to obtain the new texture coordinates. );

[0135] (7) and utilize new texture coordinates ( To obtain the pixel values ​​from the original image, we can get the corrected color values ​​and the corrected second image.

[0136] In this embodiment, the method further includes: fine-tuning the first image and comparing whether the vehicle auxiliary lines of the RVM match the actual calibration lines.

[0137] Specifically, drawing vehicle auxiliary lines using physical world coordinates introduces some calculation errors, leading to inaccuracies compared to reality. Therefore, fine-tuning is necessary. The fine-tuning method is as follows:

[0138] 1) Lay the actual calibration lines behind the vehicle. The location of the calibration lines is determined according to actual needs, such as 50cm, 2m, 3m, and ensures accuracy with a measuring tape.

[0139] 2) Turn on the RVM inside the vehicle and observe whether the RVM vehicle guide lines match the actual lines;

[0140] 3) Set four arrows (up, down, left, right) at each key point of the vehicle guide line on the interface. When the user clicks, the key point can be moved, and the distance moved by each point is recorded.

[0141] 4) After the adjustment is complete, save the distance moved by each key point and take it into account in future drawing.

[0142] Example 2

[0143] like Figure 9 As shown, the difference between this embodiment and Embodiment 1 is that this embodiment provides an RVM implementation system based on the Qualcomm 8155 platform, which corresponds one-to-one with the RVM implementation method based on the Qualcomm 8155 platform in Embodiment 1; the system includes:

[0144] The deployment unit is used to divide the reversing camera RVM into an APK layer and a server layer. The APK layer is deployed on the Android side, and the server layer is deployed on the QNX side. The server layer integrates an algorithm layer, which is used to provide algorithm logic for drawing vehicle auxiliary lines and performing distortion correction.

[0145] The configuration unit is used to allocate an RVM layer to the QNX side by modifying the display configuration of the Qualcomm 8155 chip, and to configure the RVM layer to blend with the layer on the Android side.

[0146] The rendering and distortion correction unit is used by the server layer to acquire the original image through a fisheye camera, and to overlay vehicle auxiliary lines on the original image based on the RVM vehicle auxiliary line drawing method to obtain the first image; and to perform distortion correction on the first image using a fisheye camera fast distortion correction algorithm based on the correction coefficient to obtain the corrected second image and display it on the Android side through the display configuration.

[0147] As a further implementation, based on the RVM vehicle auxiliary line drawing method, vehicle auxiliary lines are superimposed on the original image to obtain a first image, including:

[0148] Obtain the world coordinates of the vehicle guide lines. The vehicle guide lines are points that mark a fixed distance behind the vehicle to help the driver judge distance; the world coordinates are a coordinate system with the center of the vehicle as the origin.

[0149] Transform world coordinates into camera coordinates; camera coordinates are a coordinate system with the optical center of the fisheye camera as the origin.

[0150] Based on camera coordinates, the imaging process of a fisheye camera is simulated to obtain the distorted camera coordinates;

[0151] Calculate pixel coordinates based on the distorted camera coordinates and the fisheye camera intrinsic parameter matrix;

[0152] Based on the pixel coordinates, vehicle auxiliary lines are superimposed on the original image to obtain the first image.

[0153] The execution process of each unit can be carried out according to the RVM implementation method flow steps based on the Qualcomm 8155 platform in Embodiment 1, and will not be described in detail in this embodiment.

[0154] Meanwhile, the present invention also provides a computer-readable storage medium storing a computer program, characterized in that the computer program, when executed by a processor, implements the above-described RVM implementation method based on the Qualcomm 8155 platform.

[0155] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0156] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0157] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0158] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0159] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An RVM implementation method based on the Qualcomm 8155 platform, characterized in that, The method includes: The reversing camera (RVM) is divided into an APK layer and a server layer. The APK layer is deployed on the Android side, and the server layer is deployed on the QNX side. The server layer integrates an algorithm layer, which provides algorithm logic for drawing vehicle auxiliary lines and performing distortion correction. By modifying the display configuration of the Qualcomm 8155 chip, an RVM layer is assigned to the QNX side, and the RVM layer is configured to be merged with the layer on the Android side. The server layer acquires the original image using a fisheye camera, overlays vehicle auxiliary lines onto the original image based on the RVM vehicle auxiliary line drawing method to obtain a first image; and then uses a fisheye camera fast distortion correction algorithm based on correction coefficients to correct the distortion of the first image to obtain a corrected second image, which is then displayed on the Android side through display configuration.

2. The RVM implementation method based on the Qualcomm 8155 platform according to claim 1, characterized in that, The APK layer is used to transmit messages to the QNX-side server layer through inter-system communication when a user clicks a button, thereby implementing the interaction logic. The server layer is used to receive messages from the APK layer, messages from the CAN bus, or vehicle reverse gear information, and call the algorithm layer for display.

3. The RVM implementation method based on the Qualcomm 8155 platform according to claim 1, characterized in that, By modifying the display configuration of the Qualcomm 8155 chip, including: On the QNX side, a first display interface and a second display interface are configured. The first display interface is used for normal instrument content display; the second display interface is used for projection onto the Android side. Configure the server layer to use the second display interface on the QNX side as its own display interface; On the Android side, a third, fourth, and fifth display interface are configured. The third display interface is used for basic content display on the Android side; the fourth display interface is used for the APK layer; and the fifth display interface is used for navigation map projection, the content of which will be displayed on the instrument panel.

4. The RVM implementation method based on the Qualcomm 8155 platform according to claim 1, characterized in that, Based on the RVM vehicle auxiliary line drawing method, vehicle auxiliary lines are superimposed on the original image to obtain a first image, including: Obtain the world coordinates of the vehicle auxiliary lines, which are points marking a fixed distance behind the vehicle to assist the driver in judging distance; the world coordinates are a coordinate system with the vehicle center as the origin; The world coordinates are transformed into camera coordinates; the camera coordinates are a coordinate system with the optical center of the fisheye camera as the origin. Based on the camera coordinates, the fisheye camera imaging process is simulated to obtain the distorted camera coordinates; Calculate the pixel coordinates based on the distorted camera coordinates and the fisheye camera intrinsic parameter matrix; Based on the pixel coordinates, vehicle auxiliary lines are superimposed on the original image to obtain a first image.

5. The RVM implementation method based on the Qualcomm 8155 platform according to claim 1, characterized in that, The fast distortion correction algorithm for fisheye cameras based on correction coefficients is a process of adding correction coefficients to existing fast distortion correction algorithms for fisheye cameras to obtain new texture coordinates and then rendering them.

6. The RVM implementation method based on the Qualcomm 8155 platform according to claim 5, characterized in that, A fast distortion correction algorithm based on a fisheye camera, using correction coefficients, is applied to the first image to correct distortion, resulting in a corrected second image, including: The distortion coefficients of the first image from the center point to the farthest point are divided into a preset number of levels; The first image is divided into a preset number of basic distances; the basic distance refers to the distance from the farthest point of the first image to the center point. Calculate the distortion coefficients of a preset number of pixels from the center point to the farthest point to obtain the preset number of distortion coefficients; During rendering, the distance from the current point to the center point is calculated based on the original texture coordinates of the current point and recorded as the first distance; Divide the first distance by the base distance to obtain the ratio; take the corresponding distortion coefficient based on the ratio; and multiply the distortion coefficient by the original texture coordinates to obtain the new texture coordinates; The new texture coordinates are then used to retrieve the pixel values ​​from the original image to obtain the corrected second image.

7. The RVM implementation method based on the Qualcomm 8155 platform according to claim 1, characterized in that, The method further includes: fine-tuning the first image and comparing whether the vehicle auxiliary lines of the RVM match the actual calibration lines.

8. A system implementing RVM based on the Qualcomm 8155 platform, characterized in that, The system includes: The deployment unit is used to divide the reversing image RVM into an APK layer and a server layer, deploying the APK layer on the Android side and the server layer on the QNX side; and the server layer integrates an algorithm layer, which is used to provide algorithm logic to draw vehicle auxiliary lines and perform distortion correction. The configuration unit is used to allocate an RVM layer to the QNX side by modifying the display configuration of the Qualcomm 8155 chip, and configure the RVM layer to merge with the layer on the Android side; The rendering and distortion correction unit is used by the server layer to acquire the original image through a fisheye camera, and to overlay vehicle auxiliary lines on the original image based on the RVM vehicle auxiliary line drawing method to obtain a first image; and to perform distortion correction on the first image using a fisheye camera fast distortion correction algorithm based on correction coefficients to obtain a corrected second image and display it on the Android side through display configuration.

9. The RVM implementation system based on the Qualcomm 8155 platform according to claim 1, characterized in that, Based on the RVM vehicle auxiliary line drawing method, vehicle auxiliary lines are superimposed on the original image to obtain a first image, including: Obtain the world coordinates of the vehicle auxiliary lines, which are points marking a fixed distance behind the vehicle to assist the driver in judging distance; the world coordinates are a coordinate system with the vehicle center as the origin; The world coordinates are transformed into camera coordinates; the camera coordinates are a coordinate system with the optical center of the fisheye camera as the origin. Based on the camera coordinates, the fisheye camera imaging process is simulated to obtain the distorted camera coordinates; Calculate the pixel coordinates based on the distorted camera coordinates and the fisheye camera intrinsic parameter matrix; Based on the pixel coordinates, vehicle auxiliary lines are superimposed on the original image to obtain a first image.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the RVM implementation method based on the Qualcomm 8155 platform as described in any one of claims 1 to 7.

Images