Reversing image display method and apparatus, device, and computer storage medium

Abstract

Disclosed are a reversing image display method and apparatus, a device, and a computer storage medium, which are applied to an in-vehicle terminal for installing an Android system. The reversing image display method comprises: mounting a target partition in a kernel_init thread startup phase of an Android system; running a preset program in the target partition to create a file system directory of a reversing application, and mounting a file system in the target partition to the file system directory, the file system comprising files required for running of the reversing application; re-running the preset program to set a user role and a read-write permission of the reversing application in the file system directory; and starting the reversing application to control a camera connected to the in-vehicle terminal to acquire reversing image data, and displaying the reversing image data.

Description

Reversing image display method, device, equipment, and computer storage medium

[0001] Citation of Related Applications

[0002] This disclosure claims all rights and interests in the invention patent application with application number 202311016273.0, filed with the State Intellectual Property Office of the People's Republic of China on August 11, 2023, and entitled "A reversing image display method, device, equipment, and computer storage medium," and incorporates its entire contents into this disclosure by reference.

[0003] field

[0004] The present disclosure generally relates to the field of reverse image processing technology, and more specifically, to a reverse image display method, device, equipment, and computer storage medium.

[0005] background

[0006] Currently, electronic devices installed with the Android system must wait for the Android system to start before they can perform corresponding operations. For example, a vehicle-mounted terminal can only operate its own camera to collect and display reversing image data after the Android system is started. As a result, the camera startup time is too long, affecting the display speed of the reversing image data, and then affecting the speed of the reversing operation.

[0007] Overview

[0008] In a first aspect, the present disclosure provides a method for displaying a reverse image, which is applied to a vehicle-mounted terminal installed with an Android system, and includes:

[0009] During the kernel_init thread startup phase of the Android system, mounting the target partition;

[0010] Running a preset program in the target partition to create a file system directory of a reversing application, and mounting the file system in the target partition to the file system directory, wherein the file system includes files required for running the reversing application;

[0011] Re-running the preset program to set the user role and read / write permissions of the reversing application program in the file system directory; and

[0012] The reversing application is started to control the camera connected to the vehicle-mounted terminal to collect reversing image data and display the reversing image data.

[0013] In some embodiments, the step of starting the reversing application to control the camera connected to the vehicle-mounted terminal to collect reversing image data includes:

[0014] The reversing application is started to control the camera to collect the reversing image data through the V4L2 framework.

[0015] In some embodiments, during the kernel_init thread startup phase of the Android system, after mounting the target partition, the method further includes:

[0016] After the vehicle-mounted terminal is started, a target plug-in is used to interact with the driver of the camera through the V4L2 framework to light up the camera.

[0017] In some embodiments, after mounting the file system in the target partition to the file system directory and before re-running the preset program to set the user role and read / write permissions of the reversing application in the file system directory, the method further includes:

[0018] Load the SE security policy and set the SE security mode.

[0019] In some embodiments, the processing of the reversing image data includes:

[0020] Displaying the reversing image data via a first graphics card;

[0021] After processing the reversing image, the method further includes:

[0022] The second graphics card displays other image data besides the reversing image data.

[0023] In a second aspect, the present disclosure provides a reversing image display device, which is applied to a vehicle-mounted terminal installed with an Android system, and includes:

[0024] A preset program is used to create a file system directory for the reversing application during the kernel_init thread startup phase of the Android system, and mount the file system in the target partition of the vehicle terminal to the file system directory, wherein the file system includes files required for the operation of the reversing application; and set the user role and read and write permissions of the reversing application in the file system directory;

[0025] The reversing application is used to control the camera connected to the vehicle-mounted terminal to collect reversing image data and process the reversing image data.

[0026] In some embodiments, the reversing image display device further includes:

[0027] The V4L2 framework is used to interact with the camera driver under the control of the target plug-in to light up the camera after the vehicle terminal is started; and is used to control the camera to collect the reversing image data under the instruction of the reversing application.

[0028] In some embodiments, the reversing image display device further includes:

[0029] a first graphics card configured to display the reversing image data;

[0030] The second graphics card is configured to display other image data besides the reversing image data.

[0031] In a third aspect, the present disclosure provides a computer device comprising:

[0032] a memory configured to store a computer program;

[0033] The processor is configured to implement the reversing image display method described in the present disclosure when executing the computer program.

[0034] In a fourth aspect, the present disclosure provides a computer storage medium, in which a computer program is stored. When the computer program is executed by a processor, the reversing image display method described in the present disclosure is implemented.

[0035] In certain implementation schemes, a vehicle-mounted terminal installed with an Android system can mount and run a preset program in a target partition to create a file system directory for a reversing application during the kernel_init thread startup phase of the Android system, mount the file system in the target partition to the file system directory, and rerun the preset program to set the user role and read-write permissions of the reversing application in the file system directory, thereby setting up a running environment for the reversing application in advance, and starting the reversing application to control the camera to collect reversing image data and process the reversing image data. This enables the reversing application to start the camera for reversing display during the kernel_init thread startup phase of the Android system, which speeds up the startup speed of the camera compared to a solution that waits for the Android system to start before starting the camera for reversing display. This speeds up the display speed of the reversing image data, and ultimately speeds up the reversing display speed of the vehicle-mounted terminal.

[0036] BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are merely embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without any creative work.

[0038] FIG1 is a first flow chart of a method for displaying a reverse image according to an embodiment of the present disclosure;

[0039] FIG2 is a second flow chart of a method for displaying a reverse image according to an embodiment of the present disclosure;

[0040] FIG3 is a third flow chart of a method for displaying a reverse image according to an embodiment of the present disclosure;

[0041] Figure 4 is a normal boot flow chart of the Android system;

[0042] Figure 5 is a flowchart of reverse start-up of the reverse imaging device;

[0043] FIG6 is a schematic flow chart of the structure of a reversing image display device provided by an embodiment of the present disclosure; and

[0044] FIG7 is a structural diagram of a computer device 20 according to an embodiment of the present disclosure.

[0045] Details

[0046] The following will be combined with the accompanying drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the embodiments described are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of the present disclosure.

[0047] Please refer to FIG. 1 , which is a first flow chart of a method for displaying a reverse image according to an embodiment of the present disclosure.

[0048] The method for displaying a reverse image provided by the embodiments of the present disclosure is applied to a vehicle-mounted terminal installed with an Android system and may include the following steps:

[0049] Step S101: During the kernel_init thread startup phase of the Android system, read the mount target partition;

[0050] Step S102: running a preset program in the target partition to create a file system directory for the reversing application, and mounting the file system in the target partition to the file system directory, where the file system includes files required for running the reversing application;

[0051] Step S103: re-running the preset program to set the user role and read / write permissions of the reverse application program in the file system directory; and

[0052] Step S104: starting a reversing application to control the camera connected to the vehicle-mounted terminal to collect reversing image data and display the reversing image data.

[0053] In certain implementation schemes, a vehicle-mounted terminal installed with an Android system can read the mounted target partition during the startup phase of the kernel_init thread of the Android system, so as to subsequently build the operating environment of the reversing application for controlling the camera in advance based on the mounted target partition, so as to start the camera connected to the vehicle-mounted terminal during the startup phase of the kernel_init thread of the Android system.

[0054] In certain implementation schemes, when a vehicle-mounted terminal installed with an Android system builds an operating environment for a reversing application based on a mounted target partition, it can first run a preset program in the target partition to create a file system directory for the reversing application, and mount the file system in the target partition to the file system directory. The file system includes files required for the operation of the reversing application. It should be noted that the file system may include dynamic libraries, configuration files, security policies, executable files, etc., and may include sysfs, selinuxfs, devtmpfs, etc.; then rerun the preset program to set the user role and read and write permissions of the reversing application in the file system directory.

[0055] In certain implementation schemes, after the vehicle terminal installed with the Android system mounts the file system in the target partition to the file system directory, before re-running the preset program to set the user role and read and write permissions of the reversing application in the file system directory, in order to reduce the dangers brought by the reversing application, the SE security policy in the target partition can also be loaded, and the SE security mode can be set to provide security protection for the reversing application, etc.

[0056] In certain implementation schemes, after the vehicle terminal installed with the Android system builds the operating environment of the reversing application based on the target partition, the reversing application can be started to control the camera connected to the vehicle terminal to collect reversing image data and display the reversing image data, so that the camera can be started for reversing display during the kernel_init thread startup phase of the Android system, thereby quickly providing reversing services to users.

[0057] In certain implementation schemes, a vehicle-mounted terminal installed with an Android system can mount and run a preset program in a target partition to create a file system directory for a reversing application during the kernel_init thread startup phase of the Android system, mount the file system in the target partition to the file system directory, and rerun the preset program to set the user role and read-write permissions of the reversing application in the file system directory, thereby setting up a running environment for the reversing application in advance, and starting the reversing application to control the camera to collect reversing image data and process the reversing image data. This enables the reversing application to start the camera for reversing display during the kernel_init thread startup phase of the Android system, which speeds up the startup speed of the camera compared to a solution that waits for the Android system to start before starting the camera for reversing display. This speeds up the display speed of the reversing image data, and ultimately speeds up the reversing display speed of the vehicle-mounted terminal.

[0058] Please refer to FIG. 2 , which is a second flow chart of a method for displaying a reverse image according to an embodiment of the present disclosure.

[0059] The method for displaying a reverse image provided by the embodiments of the present disclosure is applied to a vehicle-mounted terminal installed with an Android system and may include the following steps:

[0060] Step S201: During the kernel_init thread startup phase of the Android system, the target partition is mounted;

[0061] Step S202: After the vehicle terminal is started, the target plug-in is used to interact with the camera driver through the V4L2 framework to light up the camera;

[0062] Step S203: running a preset program in the target partition to create a file system directory for the reversing application, and mounting the file system in the target partition to the file system directory, where the file system includes files required for running the reversing application;

[0063] Step S204: re-running the preset program to set the user role and read / write permissions of the reverse application program in the file system directory; and

[0064] Step S205: Start the reversing application to control the camera to collect reversing image data through the V4L2 framework.

[0065] In certain embodiments, after mounting the target partition during the kernel_init thread startup phase of the Android system, the vehicle terminal installed with the Android system can also activate the camera based on the target plug-in after the vehicle terminal is started. That is, the target plug-in can be used to quickly activate the camera after the vehicle terminal is started. The target plug-in can be written in C or C++, etc., which is not specifically limited in this disclosure.

[0066] In certain embodiments, when a vehicle terminal installed with an Android system lights up a camera based on a target plug-in, the target plug-in can be used to interact with the camera driver through the V4L2 (Video for Linux2) framework, without having to wait for the Android system to be fully started, thereby achieving the purpose of quickly lighting up the camera.

[0067] In certain embodiments, the process of mounting the target partition and quickly starting the user space program in the kernel_init thread may be as follows: in kernel_init() of kernel / msm-4.14 / init / main.c, the kernel_init_freeable() function is first called. In the kernel_init_freeable() function, the newly added launch_early_services() function is called to mount the target partition and start the user space early_init program (preset program) through the call_usermodehelper() function. Then, the get_early_services_status() function is called to query the execution result of the early_init program and wait for the security policy to be loaded. The " / dev / sedone" identifier is created by the early_init program. The code may be as follows:

[0068] In certain implementation schemes, when a vehicle terminal installed with an Android system starts a reversing application to control a camera connected to the vehicle terminal to collect reversing image data, the reversing application can be started to control the camera to collect reversing image data through the V4L2 framework, that is, the V4L2 framework can be used to quickly interact with the camera, and then the reversing image data can be quickly obtained for display.

[0069] Please refer to FIG. 3 , which is a third flow chart of the reversing image display method provided in one embodiment of the present disclosure.

[0070] The method for displaying a reverse image provided by the embodiments of the present disclosure is applied to a vehicle-mounted terminal installed with an Android system and may include the following steps:

[0071] Step S301: During the kernel_init thread startup phase of the Android system, the target partition is mounted;

[0072] Step S302: running a preset program in the target partition to create a file system directory for the reversing application, and mounting the file system in the target partition to the file system directory, where the file system includes files required for running the reversing application;

[0073] Step S303: re-running the preset program to set the user role and read and write permissions of the reverse application program in the file system directory;

[0074] Step S304: starting a reversing application to control a camera connected to the vehicle-mounted terminal to collect reversing image data, and displaying the reversing image data via the first graphics card; and

[0075] Step S305: Displaying other image data except the reversing image data via the second graphics card.

[0076] In certain embodiments, considering that displaying the reversing image data captured by the camera will occupy the display interface of the vehicle terminal, and in the early stages of booting, the Android surfacefinger service is not yet running and cannot be used for display, in order to avoid this situation and flexibly display the image information captured by the camera and other images during the kernel_init thread startup phase, the vehicle terminal can display the reversing image data through the first graphics card during the image information processing process; accordingly, after displaying the reversing image data, other image data other than the reversing image data can also be displayed based on the second graphics card of the vehicle terminal, such as displaying the Android interface, displaying the road image of the car moving forward, etc. In other words, the vehicle terminal can use a dual graphics card mechanism to display the reversing image data captured by the camera and the Android interface. In this way, the reversing image data captured by the camera and the Android interface can be displayed in two different layers, so that the display of the reversing image data captured by the camera does not affect the display process of the Android interface. For example, during the boot animation playback phase, even if the camera is frequently used to capture reversing image data, the two graphics cards can be used to seamlessly switch between displaying the reversing image data captured by the camera and displaying the boot animation.

[0077] In order to facilitate understanding of the reversing image display solution provided by the present disclosure, the startup process of the camera connected to the vehicle terminal is now described in conjunction with the vehicle terminal. The vehicle terminal adopts a dual graphics card mechanism, that is, there are two layers, one layer is used to display the original Android interface, using the / dev / dri / card0 node, and the other layer is specifically used for displaying the reversing screen, using the / dev / dri / card2 node. Assuming that the target partition is the early_services partition, it can include the following steps:

[0078] Step 1: Write the camera's reversing application, early_camera, and its required .so dynamic library, configuration files, security policies, and executable files required for system startup into the early_services partition. Mount the early_services partition through the kernel_init thread of the vehicle terminal. Set the early_init process to start the camera.

[0079] Step 2: After the vehicle terminal is started, use the target plug-in to interact with the camera driver through the V4L2 framework to quickly light up the camera;

[0080] Step 3: During the kernel_init thread startup phase of the Android system, the vehicle terminal runs the early_init process based on the early_services partition corresponding to the kernel_init thread. Specifically, in the first phase of the early_init process: create a file system directory and mount related file systems, including sysfs, selinuxfs, devtmpfs, etc.; load the SE security policy and set the SE security mode, and restart to enter the second phase through the execv() function; the second phase: set the user role and read and write permissions of the files in the early_services directory, and parse the early_init.conf configuration file to start the application defined in the early_init.conf, and the early_camera application is defined in the configuration file;

[0081] Step 4: The vehicle terminal sends the reversing command to the early_camera application, which then activates the camera through the V4L2 framework to capture the reversing image. The early_camera application then writes data to the / dev / dri / card2 node, and the vehicle terminal displays the reversing image on its own display based on the first layer. At this point, the reversing image overlays the original Android interface. During this process, the early_camera application can use the DRM (DirectRenderingManager) graphics display framework for rendering; and

[0082] Step 5: After reversing is completed, the early_camera application stops writing data to the / dev / dri / card2 node, and the vehicle terminal displays the Android interface based on the second layer on its own display.

[0083] It should be noted that the early_init program is started instead of the early_camera reversing application during the kernel phase because the environment required for the reversing application to run must be configured first. In layman's terms, the steps required to ensure the normal operation of the reversing application in the init process are streamlined and implemented in early_init. Furthermore, the use of devtmpfs is to establish a preliminary / dev file early in the Linux kernel boot process, allowing general applications to begin execution as soon as possible without waiting for udev to complete. To support devtmpfs, the following configuration must be added to the kernel's xxx_defconfig and xxx-perf_defconfig configuration files:

[0084] CONFIG_DEVTMPFS=y

[0085] CONFIG_DEVTMPFS_MOUNT=y.

[0086] It should also be noted that the implementation of the launch_early_services() and get_early_services_status() functions can be as follows:

[0087] As can be seen from the code, saved_early_userspace contains the device name of the early_services partition, such as / dev / block / sda14, which is passed from the Bootloader (EDK2). In EDK2, the part_name of the early_services partition is obtained through the GetSystemPath() function and passed to the Kernel through the command line. Therefore, corresponding modifications must be made in EDK2. The specific changes will not be repeated here. Among them, EDK2 is also written as EDKII, which is a set of open source codes implemented by Intel for the UEFI specification. UEFI (Unified Extensible Firmware) is the unified extensible firmware interface, which defines the interface specification between the computer and platform firmware.

[0088] rc to trigger the init.rc script.

[0089] Please refer to FIG6 , which is a schematic structural flowchart of a reversing image display device provided by an embodiment of the present disclosure.

[0090] The reversing image display device provided by the embodiments of the present disclosure is applied to a vehicle-mounted terminal installed with an Android system and may include:

[0091] The preset program 101 is used to create a file system directory for the reversing application during the kernel_init thread startup phase of the Android system, and mount the file system in the target partition of the vehicle terminal to the file system directory, where the file system includes files required for the reversing application to run; set the user role and read and write permissions of the reversing application in the file system directory; and

[0092] The reversing application 102 is used to control the camera connected to the vehicle-mounted terminal to collect reversing image data and process the reversing image data.

[0093] The present disclosure provides a reversing image display device, which is applied to a vehicle-mounted terminal installed with an Android system and may further include:

[0094] The V4L2 framework 103 is used to interact with the camera driver under the control of the target plug-in 104 to light up the camera after the vehicle terminal is started; and is used to control the camera to collect reversing image data under the instruction of the reversing application.

[0095] The present disclosure provides a reversing image display device, which is applied to a vehicle-mounted terminal installed with an Android system and may further include:

[0096] A first graphics card 105 is configured to display reverse image data;

[0097] The second graphics card 106 is configured to display image data other than the reversing image data.

[0098] The description of the corresponding components in the reversing image display device provided by the present disclosure can be found in the above embodiments and will not be repeated here.

[0099] The present disclosure also provides a computer device. FIG7 is a structural diagram of a computer device 20 according to an exemplary embodiment, and the content in the diagram should not be considered as any limitation on the scope of application of the present disclosure.

[0100] Figure 7 is a schematic diagram of the structure of a computer device 20 provided in an embodiment of the present disclosure. The computer device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is configured to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps of the aforementioned reverse image display method of the present disclosure. In certain embodiments, the computer device 20 in this embodiment may be a server.

[0101] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the computer device 20; the communication interface 24 can create a data transmission channel between the computer device 20 and external devices. The communication protocol it follows is any communication protocol that can be applied to the technical solution of the present disclosure and is not specifically limited here; the input and output interface 25 is configured to obtain external input data or output data to the outside world. Its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0102] In some embodiments, the memory 22 serves as a carrier for resource storage, which may be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon may include an operating system 221, a computer program 222 and video data 223, etc. The storage method may be temporary storage or permanent storage.

[0103] In certain embodiments, the operating system 221 is used to manage and control the hardware devices and computer programs 222 on the computer device 20, enabling the processor 21 to operate and process the massive amount of video data 223 in the memory 22. The operating system 221 may be Windows Server, NetWare, Unix, Linux, etc. In addition to including computer programs capable of implementing the disclosed method for displaying a reverse image by the computer device 20, the computer programs 222 may further include computer programs capable of performing other specific tasks. In certain embodiments, the data 223 may include various data collected by the computer device 20.

[0104] In certain embodiments, the computer device may be a module capable of realizing a positioning communication function or a terminal device comprising a module, etc. The terminal device may be a mobile terminal and / or a smart device, etc. The mobile terminal may be at least one of a mobile phone, a tablet computer, a laptop computer, etc. The smart device may be at least one of a smart watch, a smart refrigerator, a smart speaker, a smart washing machine, a smart TV, etc. The module may be any one of a 2G communication module, a 3G communication module, a 4G communication module, a 5G communication module, a NB-IOT communication module, etc.

[0105] The embodiment of the present disclosure further discloses a computer storage medium, in which a computer program is stored. When the computer program is loaded and executed by a processor, the reversing image display method of the present disclosure is implemented.

[0106] It should also be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0107] The above description of the disclosed embodiments will enable those skilled in the art to implement or use the present disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure is not limited to the embodiments shown herein, but is to be construed in the widest manner consistent with the principles and novel features disclosed herein.

Claims

1. A method for displaying a reversing image, applied to a vehicle terminal installed with an Android system, comprising: During the kernel_init thread startup phase of the Android system, mounting the target partition; Running a preset program in the target partition to create a file system directory of the reversing application, and mounting the file system in the target partition to the file system directory, wherein the file system includes files required for running the reversing application; Re-running the preset program to set the user role and read-write permissions of the reversing application program in the file system directory; as well as The reversing application is started to control the camera connected to the vehicle-mounted terminal to collect reversing image data and display the reversing image data.

2. The method of claim 1, wherein: The starting of the reversing application to control the camera connected to the vehicle terminal to collect reversing image data includes: The reversing application is started to control the camera to collect the reversing image data through the V4L2 framework.

3. The method according to claim 1 or 2, wherein: In the kernel_init thread startup phase of the Android system, after mounting the target partition, the method further includes: After the vehicle-mounted terminal is started, the target plug-in is used to interact with the driver of the camera through the V4L2 framework to light up the camera.

4. The method according to any one of claims 1 to 3, wherein: After mounting the file system in the target partition to the file system directory, and before re-running the preset program to set the user role and read / write permissions of the reversing application program in the file system directory, the method further includes: Load the SE security policy and set the SE security mode.

5. The method according to any one of claims 1 to 4, wherein: The displaying of the reversing image data includes: Displaying the reversing image data via a first graphics card; After displaying the reversing image, the method further includes: The other image data besides the reversing image data are displayed by the second graphics card.

6. A reversing image display device, applied to a vehicle terminal installed with an Android system, comprising: A preset program is used to create a file system directory of the reversing application program during the kernel_init thread startup phase of the Android system, and mount the file system in the target partition of the vehicle terminal to the file system directory, wherein the file system includes files required for the operation of the reversing application program; Setting the user role and read / write permissions of the reversing application program in the file system directory; The reversing application is used to control the camera connected to the vehicle-mounted terminal to collect reversing image data so as to display the reversing image data.

7. The apparatus of claim 6, further comprising: The V4L2 framework is used to interact with the driver of the camera under the control of the target plug-in after the vehicle-mounted terminal is started to light up the camera; Used to control the camera to collect the reversing image data under the instruction of the reversing application.

8. The device according to claim 6 or 7, further comprising: A first graphics card configured to display the reversing image data; as well as The second graphics card is configured to display other image data besides the reversing image data.

9. Computer equipment, comprising: a memory configured to store a computer program; A processor is configured to implement the reversing image display method described in any one of claims 1 to 5 when executing the computer program.

10. A computer storage medium, wherein: The computer storage medium stores a computer program, and when the computer program is executed by a processor, the reversing image display method described in any one of claims 1 to 5 is implemented.

Images