System and method for deploying AVM on QNX side based on high-pass 8155 platform

Abstract

The invention relates to a system and method for deploying an AVM on a QNX side based on a high-pass 8155 platform, a panoramic monitoring image system AVM is divided into an APK layer, a logic layer and an algorithm layer, the APK layer is located at the uppermost layer, the logic layer is located at the middle layer, and the algorithm layer is located at the lowermost layer; the logic layer and the algorithm layer are deployed on a QNX operating system side to realize output of content pictures of AVM, and the APK layer is deployed on an Android side to realize interaction of user click events; a display layer is distributed to the QNX side by modifying display configuration of the 8155 chip, and hardware fusion is finally achieved between the display layer and an Android layer; and the AVM content of the QNX side and the content of the Android side are fused in a layer synthesizer to form a final picture. According to the method, the AVM image output by the QNX side is ensured to be displayed on the Android screen through the configuration of the image layer, the QNX side is always responsible for image output, the problem of image jitter during switching of the QNX side and the Android side does not exist, and the AVM is quickly started due to the fact that the starting speed of the QNX side has the advantage.

Description

Technical Field

[0001] The present invention belongs to the technical field of AVM deployment, and in particular to a system and method for deploying AVM on a QNX side based on a Qualcomm 8155 platform. Background Art

[0002] The AVM system uses multiple (usually four) ultra-wide-angle fisheye lenses to capture images, and then uses a special algorithm to correct distortion and stitch the captured images to form a panoramic image around the object. AVM is currently mostly used for on-board 360-degree parking surround view, that is, using a wide-angle camera to obtain real-time images around the vehicle, and then synthesizing them into a 360-degree surround view through software and displaying them on the vehicle's multimedia screen to assist the driver in safely and easily parking the vehicle in complex situations.

[0003] Qualcomm 8155 cockpit integration platform adopts virtual machine architecture, which is divided into QNX side and Android side. The QNX side mainly deploys instrument-related services, and the Android side mainly deploys car entertainment system-related applications. There are currently two mainstream implementation methods for AVM: one is to deploy it all on the Android side. The Android startup speed is slow. The average startup speed of Android on the market exceeds 12s. It takes 1 to 2s for AVM to be ready. When the user starts the vehicle quickly, AVM cannot be displayed, affecting the user experience. The other is to deploy it on both the QNX side and the Android side at the same time. After the system starts, QNX is responsible for displaying the AVM screen, which is called the AVM "quick start" function. When the Android startup is completed, the Android app notifies the QNX side program. After receiving the notification, the QNX side exits the AVM display and the Android side continues to display the AVM screen, which is called the AVM "normal" function. The defect of this solution is that it cannot achieve seamless switching. Due to the limitations of the Qualcomm 8155 chip, the QNX side and the Android side cannot obtain camera data at the same time. When switching, the QNX side is required to turn off the AVM first, and then the Android application is responsible for display. Users can clearly perceive the screen jitter during switching.

[0004] Therefore, how to improve the existing AVM deployment method to solve the slow startup speed of AVM and the screen jitter caused by AVM switching between the QNX side and the Android side is a technical problem that needs to be solved urgently. Summary of the invention

[0005] The purpose of the present invention is to provide a system and method for deploying AVM on the QNX side based on the Qualcomm 8155 platform, to improve the existing AVM deployment method, to solve the problem of slow startup speed of AVM and screen jitter when AVM switches between the QNX side and the Android side.

[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is as follows:

[0007] The method for deploying AVM on the QNX side based on the Qualcomm 8155 platform includes the following steps:

[0008] S1: Divide the panoramic monitoring image system AVM into three parts: APK layer, logic layer and algorithm layer, wherein the APK layer is located at the top layer, the logic layer is located at the middle layer, and the algorithm layer is located at the bottom layer;

[0009] S2: The APK layer is deployed on the Android side to display various buttons and implement interaction with user click events. The logic layer and algorithm layer are deployed on the QNX side, collectively referred to as the AVM service, which outputs the AVM screen content. The APK layer and the AVM service communicate through the inter-core communication component.

[0010] S3: Configure the chip display layer: By modifying the display configuration of the 8155 chip, add a display layer to the QNX side, and the AVM service uses this layer to output images; add a layer to the Android side, and the APK application uses this layer, and make the layer on the Android side higher than the layer on the QNX side, to ensure that the various buttons displayed by the APK application are above the AVM screen;

[0011] S4: The AVM content on the QNX side and the APK content on the Android side are merged in the chip's internal layer compositor to form the final image.

[0012] Preferably, in step S3, the display configuration of the 8155 chip is implemented by modifying the qcdisplaycfg.xml file.

[0013] Preferably, the specific process of modifying qcdisplaycfg.xml is as follows:

[0014] S31: Modify the display configuration on the QNX side, add a display port display, and specify its physical screen output as the Android screen;

[0015] S32: After the QNX system is started, the AVM service program is started. The program uses the display added in S31 to display the AVM screen.

[0016] S33: Modify the display configuration on the Android side, add a display port, and specify its physical screen output as the Android screen, but the layer height order z order is higher than the display port added on the QNX side in S31.

[0017] S34: The AVM APK on the Android side uses the layer newly added in S33, and the various buttons of the APK will be displayed on the AVM screen.

[0018] Preferably, the specific configuration of adding wfd port 2 in the client in step S31 is as follows:

[0019]

[0020]

[0021] That is, the port number is set to 2, indicating that it is the second display port on the QNX side;

[0022] The display number is set to 2, indicating that the physical screen is an Android screen;

[0023] The display pipe number is set to 15, indicating that the 15th display pipe of the 8155 chip is used;

[0024] The layer order is set to 2, indicating that this layer is the second layer in the vertical relationship of the layers.

[0025] Preferably, in step S32, eQDIDisplayID is set to 2, that is, all content displayed on the display is projected onto the Android screen.

[0026] Preferably, in step S33, the result of adding a client on the basis of the original client is as follows:

[0027]

[0028]

[0029] That is, set the port number to 2, indicating that it is the second display port on the Android side;

[0030] The display number is set to 2, indicating that the physical screen is an Android screen;

[0031] The display pipe number is set to 17, indicating that the 15th display pipe of the 8155 chip is used;

[0032] The layer order is set to 5, indicating that in terms of the vertical relationship of the layers, this layer is in the second layer, which is higher than the second layer of the QNX side port, ensuring that the APK display content on the Android side covers the AVM screen content during synthesis.

[0033] In a second aspect, a system for deploying AVM on the QNX side based on the Qualcomm 8155 platform is provided, which is used to implement any one of the methods for deploying AVM on the QNX side based on the Qualcomm 8155 platform, including a QNX operating system side, an Android side and a management program Hypervisor, the QNX operating system side is provided with a Screen interface, a CAN module and an AVM Service module, the Android side is provided with AVM Apk, SurfaceFlinger and LibDRM, the management program Hypervisor is provided with a CAN module, a Camera module and a Display module, and the Camera module and the AVM Service module communicate through the Camera driver and the Qualcomm camera component AIS Server.

[0034] Preferably, the Fast AVM Service module on the QNX operating system side obtains signals through the CAN module, obtains data from the Camera module through the Camera Engine module and the AIS Server module, completes image stitching, and then displays it through the Screen interface on the QNX side.

[0035] Preferably, after the layer configuration, the AVM content on the QNX side and the content on the Android side are merged through the Layer mixer layer synthesizer to form the final picture.

[0036] Preferably, two displays are configured on the QNX operating system side, display0 is used for normal instrument display, and display1 is newly added and used to display AVM content on the Android screen; three displays are configured on the Android side, display0 is used for normal Android content display, display1 display1 is newly added and used to display AVM APK, and other content that needs to be covered on the AVM screen all the time, including Bluetooth phone calls, and display2 is used for navigation projection, and the content on display2 will be projected onto the instrument screen; the layer mixer is a hardware module on the Qualcomm 8155 chip, which fuses multiple layers according to the hierarchy.

[0037] The beneficial effects of the present invention include:

[0038] The method for deploying AVM on the QNX side based on the Qualcomm 8155 platform provided by the present invention comprises the following steps: dividing the panoramic monitoring image system AVM into three parts: an APK layer, a logic layer and an algorithm layer, wherein the APK layer is located at the top layer, the logic layer is located at the middle layer, and the algorithm layer is located at the bottom layer; the logic layer and the algorithm layer are deployed on the QNX operating system side to output the content screen of the AVM, and the APK layer is deployed on the Android side to interact with the user's click event; configuring the layer: by modifying the display configuration of the 8155 chip, a display layer is allocated to the QNX side, and the layer is finally hardware-fused with the Android layer; the AVM content on the QNX side and the content on the Android side are fused in a layer synthesizer to form a final screen.

[0039] First, the entire AVM function is divided into three parts from top to bottom: APK layer, logic layer, and algorithm layer. The logic and algorithm are deployed on the QNX side, responsible for outputting the AVM content screen. The APK layer is deployed on the Android side, mainly responsible for the interaction of user click events. There is no difference between "fast start" and "normal" AVM in the whole process. Since there is no switching logic between the QNX side and the Android side, it not only reduces the coupling between the systems and the probability of problems, but also solves the problem of screen jitter during switching.

[0040] Secondly, through the configuration of layers, it can be ensured that the AVM image output by the QNX side is displayed on the Android screen. The QNX side is always responsible for image output, so there is no screen jitter problem when switching between the QNX side and the Android side. Since the QNX side has an advantage in startup speed, AVM can be started quickly. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 The figure is a flow chart of the method for deploying AVM on the QNX side based on the Qualcomm 8155 platform according to the present invention.

[0042] Figure 2 The schematic diagram of the system architecture of the present invention is to deploy AVM on the QNX side based on the Qualcomm 8155 platform.

[0043] Figure 3 The diagram of the system layer distribution of deploying AVM on the QNX side based on the Qualcomm 8155 platform of the present invention. DETAILED DESCRIPTION

[0044] The following is combined with Figure 1-Figure 3 The present invention is further described in detail:

[0045] Example 1

[0046] See also Figure 1 , a method for deploying AVM on the QNX side based on the Qualcomm 8155 platform includes the following steps:

[0047] S1: Divide the panoramic monitoring image system AVM into three parts: APK layer, logic layer and algorithm layer, wherein the APK layer is located at the top layer, the logic layer is located at the middle layer, and the algorithm layer is located at the bottom layer;

[0048] S2: The APK layer is deployed on the Android side to display various buttons and implement interaction with user click events; the logic layer and algorithm layer are deployed on the QNX side, collectively referred to as AVM services, to output AVM screen content, and the APK layer and AVM services communicate through inter-core communication components. The entire AVM function is divided into three parts from top to bottom: APK layer, logic layer, and algorithm layer. The logic and algorithm are deployed on the QNX side, responsible for outputting the AVM content screen. The APK layer is deployed on the Android side, mainly responsible for the interaction with user click events. There is no difference between "quick start" and "normal" AVM in the whole process, and there is no problem of switching between the two.

[0049] S3: Configure the chip display layer: By modifying the display configuration of the 8155 chip, add a display layer to the QNX side, and the AVM service uses this layer to output images; add a layer to the Android side, and the APK application uses this layer, and make the Android side layer higher than the QNX side layer to ensure that the various buttons displayed by the APK application are above the AVM screen. Through the configuration of the layer, it can be ensured that the AVM image output by the QNX side is displayed on the Android screen. The QNX side is always responsible for image output, so there is no screen jitter problem when switching between the QNX side and the Android side. Because the QNX side has an advantage in startup speed, AVM can be started quickly.

[0050] S4: The AVM content on the QNX side and the APK content on the Android side are merged in the chip's internal layer compositor to form the final image.

[0051] In this embodiment, the entire AVM function is divided into three parts from top to bottom: APK layer, logic layer, and algorithm layer, where the logic and algorithm are deployed on the QNX side, responsible for outputting the AVM content screen. The APK layer is deployed on the Android side, and is mainly responsible for the interaction of user click events. There is no difference between "quick start" and "normal" AVM in the whole process, and there is no problem of switching between the two. Through the configuration of the layer, it can be ensured that the AVM image output by the QNX side is displayed on the Android screen. The QNX side is always responsible for image output, so there is no screen jitter problem when switching between the QNX side and the Android side. Since the QNX side has an advantage in startup speed, AVM can be started quickly.

[0052] The APK layer of AVM is deployed on the Android side, mainly to implement the response to user click events. When receiving a user click on a button, it is transmitted to the AVM logic layer on the QNX side through inter-system communication to implement the interactive logic. On the display, the AVM APK is displayed on the Android side display1, which is physically higher than the AVM content layer. The AVM logic layer is deployed on the QNX side to process various interactive logics, including messages transmitted by the Android APK and messages on the CAN bus. The AVM algorithm layer mainly provides algorithm logic, including splicing algorithms, calibration algorithms, etc. After obtaining the camera data, splicing is implemented according to the algorithm. The final picture is displayed on the QNX side display1 and presented on the Android side through hardware synthesis.

[0053] The above solution solves the problem that the APK layer, logic layer, and algorithm layer of the entire AVM function are all deployed on the Android side in the prior art. Android startup speed is slow. The average startup speed of Android on the market exceeds 12s. It takes 1 to 2s for AVM to be ready. When the user starts the vehicle quickly, AVM cannot be displayed, affecting the user experience. And it is deployed on the QNX side and the Android side at the same time. After the system starts, QNX is responsible for displaying the AVM screen, which is called the AVM "quick start" function. When Android starts, the Android app notifies the QNX end program. After receiving the notification, the QNX end exits the AVM display and continues to display the AVM screen on the Android side, which is called the AVM "normal" function. The defect of this solution is that it cannot achieve seamless switching. Due to the limitation of Qualcomm 8155 chip, the QNX side and the Android side cannot obtain camera data at the same time. When switching, the QNX side is required to turn off AVM first, and then the Android application is responsible for display. Users can clearly perceive the screen jitter during switching.

[0054] Example 2

[0055] On the basis of Example 1, in step S3, the display configuration of the 8155 chip is realized by modifying the qcdisplaycfg.xml file. The specific process of modifying the qcdisplaycfg.xml file is as follows:

[0056] S31: Modify the display configuration on the QNX side, add a display port display, and specify its physical screen output as the Android screen;

[0057] S32: After the QNX system is started, the AVM service program is started. The program uses the display added in S31 to display the AVM screen.

[0058] S33: Modify the display configuration on the Android side, add a display port, and specify its physical screen output as the Android screen, but the layer height order z order is higher than the display port added on the QNX side in S31.

[0059] S34: The AVM APK on the Android side uses the layer newly added in S33, and the various buttons of the APK will be displayed on the AVM screen.

[0060] In this embodiment, see Figure 2 By modifying the display configuration of the 8155 chip, a display layer is assigned to the QNX side, which will eventually be hardware-fused with the Android layer. Finally, the AVM image content will be integrated with other content on the Android side. The key to this process is to modify qcdisplaycfg_ADP_STAR_LA_MULTI_DISP.xml.

[0061] The specific configuration of adding wfd port 2 in the client in step S31 is as follows:

[0062]

[0063] That is, the port number is set to 2, indicating that it is the second display port on the QNX side;

[0064] The display number is set to 2, indicating that the physical screen is an Android screen;

[0065] The display pipe number is set to 15, indicating that the 15th display pipe of the 8155 chip is used;

[0066] The layer order is set to 2, indicating that this layer is the second layer in the vertical relationship of the layers.

[0067] In step S32, eQDIDisplayID is set to 2, that is, all content displayed on the display is projected onto the Android screen.

[0068] In step S33, the result of adding a client to the original client is as follows:

[0069]

[0070]

[0071] That is, set the port number to 2, indicating that it is the second display port on the Android side;

[0072] The display number is set to 2, indicating that the physical screen is an Android screen;

[0073] The display pipe number is set to 17, indicating that the 15th display pipe of the 8155 chip is used;

[0074] The layer order is set to 5, indicating that in terms of the vertical relationship of the layers, this layer is in the second layer, which is higher than the second layer of the QNX side port, ensuring that the APK display content on the Android side covers the AVM screen content during synthesis.

[0075] Example 3

[0076] On the basis of Example 1 or Example 2, a system for deploying AVM on the QNX side based on the Qualcomm 8155 platform is used to implement any of the methods for deploying AVM on the QNX side based on the Qualcomm 8155 platform, including a QNX operating system side, an Android side, and a hypervisor, wherein the QNX operating system side is provided with a Screen interface, a CAN module, and an AVMService module, the Android side is provided with an AVM Apk, SurfaceFlinger, and LibDRM, the hypervisor is provided with a CAN module, a Camera module, and a Display module, and the Camera module and the AVM Service module communicate with the Qualcomm camera component AIS Server through the Camera driver. The Fast AVMService module on the QNX operating system side obtains signals through the CAN module, obtains data from the Camera module through the Camera Engine module and the AIS Server module, completes image splicing, and then displays it through the Screen interface on the QNX side.

[0077] After the layer configuration, the AVM content on the QNX side and the content on the Android side are merged through the Layer mixer to form the final image. Figure 3 Two displays are configured on the QNX operating system side. Display0 is used for normal instrument display, and display1 is newly added to display AVM content on the Android screen. Three displays are configured on the Android side. Display0 is used for normal Android content display, and display1 is newly added to display AVM APK and other content that needs to be covered on the AVM screen, including Bluetooth calls. Display2 is used for navigation projection, and the content on display2 will be projected onto the instrument screen. The layer mixer is a hardware module on the Qualcomm 8155 chip that merges multiple layers according to their hierarchical levels.

[0078] In summary, the method for deploying AVM on the QNX side based on the Qualcomm 8155 platform provided by the present invention divides the panoramic monitoring image system AVM into three parts: an APK layer, a logic layer, and an algorithm layer, wherein the APK layer is located at the top layer, the logic layer is located at the middle layer, and the algorithm layer is located at the bottom layer; the logic layer and the algorithm layer are deployed on the QNX operating system side to output the content screen of the AVM, and the APK layer is deployed on the Android side to interact with user click events; the layers are configured: by modifying the display configuration of the 8155 chip, a display layer is allocated to the QNX side, and the layer is finally hardware-fused with the Android layer; the AVM content on the QNX side and the content on the Android side are fused in a layer synthesizer to form a final screen.

[0079] The entire AVM function is divided into three parts from top to bottom: APK layer, logic layer, and algorithm layer. The logic and algorithm are deployed on the QNX side and are responsible for outputting the AVM content screen. The APK layer is deployed on the Android side and is mainly responsible for the interaction of user click events. There is no difference between "fast start" and "normal" AVM in the whole process. Since there is no switching logic between the QNX side and the Android side, it not only reduces the coupling between the systems and the probability of problems, but also solves the problem of screen jitter during switching. Through the configuration of the layer, it can be ensured that the AVM image output by the QNX side is displayed on the Android screen. The QNX side is always responsible for image output, so there is no screen jitter problem when switching between the QNX side and the Android side. Since the QNX side has an advantage in startup speed, AVM can be started quickly.

[0080] The above-mentioned embodiments only express the specific implementation method of the present application, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the protection scope of the present application. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the technical solution concept of the present application, and these all belong to the protection scope of the present application.

Claims

1. A method for deploying AVM on the QNX side based on the Qualcomm 8155 platform, characterized in that: The following steps are involved: S1: Divide the panoramic monitoring image system AVM into three parts: APK layer, logic layer and algorithm layer, wherein the APK layer is located at the top layer, the logic layer is located at the middle layer, and the algorithm layer is located at the bottom layer; S2: The APK layer is deployed on the Android side to display various buttons and implement interaction with user click events. The logic layer and algorithm layer are deployed on the QNX side, collectively referred to as the AVM service, which outputs the AVM screen content. The APK layer and the AVM service communicate through the inter-core communication component. S3: Configure the chip display layer: By modifying the display configuration of the 8155 chip, add a display layer to the QNX side, and the AVM service uses this layer to output images; add a layer to the Android side, and the APK application uses this layer, and make the layer on the Android side higher than the layer on the QNX side, to ensure that the various buttons displayed by the APK application are above the AVM screen; S4: The AVM content on the QNX side and the APK content on the Android side are merged in the chip's internal layer compositor to form the final image.

2. The method for deploying AVM on the QNX side based on the Qualcomm 8155 platform according to claim 1, characterized in that: In step S3, the display configuration of the 8155 chip is implemented by modifying the qcdisplaycfg.xml file.

3. The method for deploying AVM on the QNX side based on the Qualcomm 8155 platform according to claim 2, characterized in that: The specific process of modifying qcdisplaycfg.xml is as follows: S31: Modify the display configuration on the QNX side, add a display port display, and specify its physical screen output as the Android screen; S32: After the QNX system is started, the AVM service program is started. The program uses the display added in S31 to display the AVM screen. S33: Modify the display configuration on the Android side, add a display port, and specify its physical screen output as the Android screen, but the layer height order z order is higher than the display port added on the QNX side in S31. S34: The AVM APK on the Android side uses the layer newly added in S33, and the various buttons of the APK will be displayed on the AVM screen.

4. The method for deploying AVM on the QNX side based on the Qualcomm 8155 platform according to claim 3, characterized in that: The key configuration for adding the QNX side display port in step S31 is as follows: The port number is set to 2, indicating that it is the second display port on the QNX side; The display number is set to 2, indicating that the physical screen is an Android screen; The display pipe number is set to 15, indicating that the 15th display pipe of the 8155 chip is used; The layer order is set to 2, indicating that this layer is the second layer in the vertical relationship of the layers.

5. The method for deploying AVM on the QNX side based on the Qualcomm 8155 platform according to claim 3, characterized in that: In step S32, eQDIDisplayID is set to 2, that is, all content displayed on the display is projected onto the Android screen.

6. The method for deploying AVM on the QNX side based on the Qualcomm 8155 platform according to claim 3, characterized in that: In step S33, based on the original display port of Android, the key configuration of adding a display port is as follows: The port number is set to 2, indicating that it is the second display port on the Android side; The display number is set to 2, indicating that the physical screen is an Android screen; The display pipe number is set to 17, indicating that the 15th display pipe of the 8155 chip is used; The layer order is set to 5, indicating that in terms of the vertical relationship of the layers, this layer is in the second layer, which is higher than the second layer of the QNX side port, ensuring that the APK display content on the Android side covers the AVM screen content during synthesis.

7. A system for deploying AVM on the QNX side based on the Qualcomm 8155 platform, used to implement the method for deploying AVM on the QNX side based on the Qualcomm 8155 platform as described in any one of claims 1 to 6, characterized in that: The system includes a QNX operating system side, an Android side and a management program Hypervisor. The QNX operating system side is provided with a Screen interface, a CAN module and an AVM Service module. The Android side is provided with AVM Apk, SurfaceFlinger and LibDRM. The management program Hypervisor is provided with a CAN module, a Camera module and a Display module. The Camera module and the AVM Service module communicate with each other through a Camera driver and a Qualcomm camera component AIS Server.

8. The system for deploying AVM on the QNX side based on the Qualcomm 8155 platform according to claim 7, characterized in that: The AVM Service module on the QNX operating system side obtains signals through the CAN module, obtains data from the Camera through the Camera driver module and the AISServer module, completes image stitching, and then displays it through the Screen interface on the QNX side.

9. The system for deploying AVM on the QNX side based on the Qualcomm 8155 platform according to claim 7, characterized in that: After the layer configuration, the AVM content on the QNX side and the content on the Android side are merged through the layer mixer to form the final picture.

10. The system for deploying AVM on the QNX side based on the Qualcomm 8155 platform according to claim 9, characterized in that: Two displays are configured on the QNX operating system side, display0 is used for normal instrument display, and display1 is newly added and used to display AVM content on the Android screen; three displays are configured on the Android side, display0 is used for normal Android content display, display1 display1 is newly added and used to display AVM APK, and other content that needs to be covered on the AVM screen, including Bluetooth calls, and display2 is used for navigation projection. Display2 is used for navigation projection, and the content on display2 will be projected onto the instrument screen; the layer mixer is a hardware module on the Qualcomm 8155 chip, which merges multiple layers according to the hierarchy.

Images