Reversing image control method and system and vehicle

Abstract

The invention provides a reversing image control method and system and a vehicle, and the method comprises the steps: obtaining reversing image data through a first terminal, and transmitting the reversing image data to a virtual screen of the first terminal, the first terminal being an instrument end of the vehicle; the reversing image data are obtained from the virtual screen through a second terminal and displayed on the second terminal in a picture-in-picture mode, and the second terminal is a central large screen end of the vehicle. According to the method, the reversing image data are obtained through the first terminal and sent to the virtual screen, after the second terminal is started, the reversing image data can be directly obtained from the virtual screen, and the data are processed and can be directly displayed, so that the response speed is much higher; meanwhile, display is carried out on the second terminal in a picture-in-picture mode, the situation that an application mode occupies a memory is avoided, the time consumed for starting the application is also avoided, the central control large screen cannot be completely shielded in the picture-in-picture mode, and touch operation of other functions can also be carried out when the reversing image is started.

Description

Technical Field

[0001] The present invention relates to the field of vehicle control technology, and in particular to a reversing image control method and system, and a vehicle. Background Art

[0002] The panoramic surveillance camera system (AVM, Around View Monitor) is an advanced technology that can achieve 360-degree monitoring without blind spots. It integrates high-resolution cameras, image processing algorithms, network transmission technology and intelligent analysis systems to capture, transmit and analyze all dynamic information in the monitoring area in real time.

[0003] Existing central control car machines (i.e., car central control) are mostly developed based on Android when realizing the functions of panoramic monitoring camera systems. Most of these functions are realized through the application layer (the application layer is also called the application entity, which is composed of several specific application service elements and one or more common application service elements). Since starting an application in the Android system requires going through multiple process services of the system, and each time the application is started, the camera needs to be reopened, parameters need to be set, images need to be acquired, and algorithms need to be used for processing, resulting in a certain delay in each response. The delayed display brings a very bad experience to the user, and the image driver reverses normally. At the same time, when realizing the functions of the panoramic monitoring camera system based on the application layer, a certain amount of system memory needs to be occupied.

[0004] Especially when the vehicle is just started, in many scenarios, the vehicle needs to be reversed immediately after it is started. However, the central control car computer starts slowly, and it is very likely that it has not been completely started. Wait until the central control car computer starts before starting the corresponding application layer, and the delay display will be more obvious at this time. Summary of the invention

[0005] In view of the above defects of the prior art, the present invention provides a reversing image control method and system, and a vehicle to solve the technical problems of the reversing image activation delay and memory usage.

[0006] To achieve the above-mentioned purpose and other related purposes, the present invention provides a reversing image control method, including: obtaining reversing image data through a first terminal, and sending the reversing image data to a virtual screen of the first terminal, the first terminal being the instrument end of the vehicle; obtaining the reversing image data from the virtual screen through a second terminal, and displaying it in the form of a picture-in-picture on the second terminal, the second terminal being the central large screen end of the vehicle.

[0007] In one embodiment of the present invention, reversing image data is obtained through a first terminal, and the reversing image data is sent to a virtual screen of the first terminal, including: when the vehicle is engaged in a reverse gear, the reversing image data is obtained through the first terminal; the reversing image data is sent to the virtual screen of the first terminal; and a flag is modified to indicate that the reversing image function has been turned on.

[0008] In one embodiment of the present invention, the reversing image data is obtained from the virtual screen through the second terminal and displayed on the second terminal in the form of picture-in-picture, including: obtaining the flag from the first terminal through the second terminal, and judging whether the reversing image function is turned on according to the flag: if yes, obtaining the reversing image data from the virtual screen and displaying it on the second terminal in the form of picture-in-picture; if not, not obtaining the reversing image data from the virtual screen.

[0009] In one embodiment of the present invention, the reversing image data is obtained from the virtual screen through the second terminal and displayed on the second terminal in the form of picture-in-picture, including: obtaining window coordinate information from the first terminal through the second terminal; creating the window on the second terminal according to the window coordinate information; obtaining the reversing image data from the virtual screen and displaying it in the window.

[0010] In an embodiment of the present invention, acquiring window coordinate information from the first terminal through the second terminal includes: acquiring the X-axis coordinate and Y-axis coordinate, width and height of the upper left corner of the window from the first terminal through the second terminal.

[0011] In an embodiment of the present invention, after the step of creating the window on the second terminal according to the window coordinate information, the method further includes: shielding touch events in the window area; and adjusting the level of the window to the highest level.

[0012] In one embodiment of the present invention, adjusting the level of the window to the highest level includes: setting the level of the window to the highest level; reorganizing the level order of all windows on the second terminal; and requesting the next screen refresh signal to synthesize the window content after adjusting the level order.

[0013] To achieve the above-mentioned purpose and other related purposes, the present invention also provides a reversing image control system, including: a first terminal, used to obtain reversing image data and send the reversing image data to a virtual screen of the first terminal, the first terminal is the instrument terminal of the vehicle; a second terminal, used to obtain the reversing image data from the virtual screen and display it in the form of picture-in-picture on the second terminal, the second terminal is the central large screen terminal of the vehicle.

[0014] In one embodiment of the present invention, the operating system of the first terminal is a QNX operating system, and the operating system of the second terminal is an Android, Hongmeng, or iOS operating system.

[0015] To achieve the above-mentioned object and other related objects, the present invention also provides a vehicle, including the above-mentioned reversing image control system.

[0016] Beneficial effects of the present invention: The present invention proposes a reversing image control method, system, and vehicle. The method first obtains reversing image data on a first terminal and sends it to a virtual screen. Once a second terminal is turned on, the reversing image data can be directly obtained from the virtual screen. This data has been processed and can be directly displayed, so the response speed is much faster. At the same time, it is displayed in the form of picture-in-picture on the second terminal, which avoids occupying memory in the form of applications and the time consumed in starting applications. The picture-in-picture format will not completely cover the large central control screen, so that touch operations of other functions can be performed when the reversing image is turned on. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 A flow chart of a control method provided by an embodiment of the present invention;

[0019] Figure 2 A detailed flow chart of step S100 provided in one embodiment of the present invention;

[0020] Figure 3 A first display flow chart of a reversing image provided by an embodiment of the present invention;

[0021] Figure 4 A second display flow chart of a reversing image provided by an embodiment of the present invention;

[0022] Figure 5 A detailed flow chart of step S240 provided for one embodiment of the present invention;

[0023] Figure 6 A schematic diagram of a control system provided in accordance with an embodiment of the present invention.

[0024] Explanation of the reference numerals: 10, first terminal; 20, second terminal. DETAILED DESCRIPTION

[0025] The following describes the embodiments of the present invention through specific specific embodiments, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. It should be noted that the following embodiments and the features in the embodiments can be combined with each other without conflict. In addition to the specific methods, equipment, and materials used in the embodiments, according to the technical personnel in this technical field's grasp of the prior art and the records of the present invention, any methods, equipment, and materials of the prior art that are similar or equivalent to the methods, equipment, and materials described in the embodiments of the present invention can also be used to implement the present invention.

[0026] It should be understood that the terms used in the embodiments of the present invention are for describing specific embodiments rather than for limiting the scope of protection of the present invention. Unless otherwise defined, all technical and scientific terms used in the present invention have the same meaning as those generally understood by those skilled in the art.

[0027] In the following description, numerous details are discussed to provide a more thorough explanation of the embodiments of the present invention. However, it is obvious to those skilled in the art that the embodiments of the present invention can be implemented without these specific details. In some of the embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present invention difficult to understand.

[0028] The flowchart and block diagram in the accompanying drawings illustrate the system architecture, functions and operations that may be implemented according to the methods and computer program products of various embodiments disclosed in the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment, or a part of a code, and the module, the program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of the boxes in the block diagram and / or the flowchart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0029] See also Figure 1 , Figure 1 A reversing image control method provided by an embodiment of the present invention includes step S100 and step S200.

[0030] Step S100: acquiring reversing image data through the first terminal 10, and sending the reversing image data to the virtual screen of the first terminal 10, where the first terminal 10 is the instrument terminal of the vehicle.

[0031] The first terminal 10 is the instrument terminal of the vehicle, which mostly adopts the QNX (QNX Operating System) operating system. The QNX operating system is a real-time operating system, which is famous for its high reliability, high performance and modular design. It adopts a microkernel design, and the core functions are very streamlined (usually only tens of KB), and other functions (such as file system, network protocol stack, etc.) run in the user space in a modular manner. This design improves the stability and security of the system. Even if a module crashes, it will not affect the entire system. QNX is a hard real-time operating system that can ensure that tasks are completed within strict time limits and is suitable for time-sensitive application scenarios. Therefore, the instrument terminal of the vehicle is mostly developed using the QNX operating system.

[0032] It is understandable that an application that can run on the QNX operating system can be developed in the first terminal 10 to execute the function mentioned in step S100. For example, the application name is AVM, and the AVM application on the first terminal 10 obtains the reversing image data and sends it to the virtual screen.

[0033] In this step, the first terminal 10 implements these steps because after the vehicle is powered on, the startup speed of the instrument terminal is much faster than that of the second terminal 20. Therefore, the acquisition of reversing image data and the processing and display of these data can be quickly started on the instrument terminal. Since the instrument terminal does not have a large screen, in this step, the reversing image data is given to the virtual screen through a virtualization drive. The virtual screen is a virtual display device implemented by software technology. It is not a physical screen, but a display environment simulated by software.

[0034] The reversing image data may be, for example, an image taken by a camera at the rear of the vehicle, or a panoramic image taken by multiple cameras around the vehicle and synthesized. In some traditional vehicles, the reversing image data generally only includes images taken by the camera at the rear of the vehicle. With the update of technology, panoramic monitoring camera systems are becoming more and more popular. Whether it is reversing or moving the car in a small space, the panoramic monitoring camera system will be directly activated. It uses multiple cameras to shoot 360° of the vehicle without blind spots, and finally displays a panoramic view of the vehicle. The concept of reversing image data is used in each embodiment of the present invention, which includes both a single camera solution at the rear of the vehicle and a panoramic image data solution.

[0035] See also Figure 2 In a specific embodiment of the present invention, step S100 includes steps S101 to S103.

[0036] Step S101, when the vehicle is in reverse gear, reversing image data is obtained through the first terminal 10. It can be understood that the reversing image data is only needed when the vehicle is in reverse gear, so in this step, the reversing image data is only obtained when the vehicle has a reversing demand.

[0037] Step S102, sending the reversing image data to the virtual screen of the first terminal 10. The reversing image data from the camera (whether a single camera or multiple cameras) generally needs to undergo some processing, such as distortion correction, image enhancement, multi-camera image stitching, etc. In this step, before being sent to the virtual screen, corresponding processing should be performed, and the reversing image data sent to the virtual screen should be image data that can be directly displayed on the screen.

[0038] Step S103, modifying the flag to indicate that the reversing image function is turned on. In this step, the flag is set so that the second terminal 20 can quickly determine whether the reversing image function of the vehicle is turned on according to the flag.

[0039] Step S200: Obtain the reversing image data from the virtual screen through the second terminal 20, and display it in the form of picture-in-picture on the second terminal 20, where the second terminal 20 is the central large screen of the vehicle. Displaying the reversing image data in the form of picture-in-picture is a display method in which the reversing image is superimposed on the main screen of the central large screen in the form of a small window. This method allows the user to monitor the reversing image in real time while viewing the main screen, thereby improving driving safety and convenience, and avoiding memory usage and startup delay caused by starting the application.

[0040] In a specific embodiment of the present invention, step S200 includes: obtaining a flag from the first terminal 10 through the second terminal 20, and judging whether the reversing image function is turned on according to the flag: if yes, obtaining the reversing image data from the virtual screen and displaying it on the second terminal 20 in the form of picture-in-picture; if no, not obtaining the reversing image data from the virtual screen. In this embodiment, corresponding to the above step S103, after the flag is set in step S103, the second terminal 20 can judge whether to display the reversing image data in the form of picture-in-picture according to the flag. For example, the flag is stored in the attribute persist.sys.pason. When the value of the attribute is 1, it indicates that the reversing image function is turned on. Then, when the second terminal 20 obtains the attribute value of persist.sys.pason as 1, the picture-in-picture mode of the second terminal 20 will be started, and the corresponding data will be obtained and displayed in the virtual screen. When the second terminal 20 obtains the attribute value of persist.sys.pason as 0, it indicates that the reversing image function is not turned on, and there is no need to obtain the reversing image data at this time.

[0041] See also Figure 3 In a specific embodiment of the present invention, the steps of acquiring the reversing image data from the virtual screen through the second terminal 20 and displaying it on the second terminal 20 in the form of picture-in-picture include step S210, step S220 and step S250.

[0042] Step S210: Obtain window coordinate information from the first terminal 10 through the second terminal 20. The window coordinate information is obtained from the first terminal 10, and the window coordinate information may be preset.

[0043] In a specific embodiment of the present invention, the X-axis coordinate and Y-axis coordinate, width and height of the upper left corner of the window are obtained from the first terminal 10 through the second terminal 20. That is, the window coordinate information includes the X-axis coordinate pas.touchblock.x, the Y-axis coordinate pas.touchblock.y, the width pas.touchblock.w and the height pas.touchblock.h of the upper left corner of the window. With these four data, the size of the window is determined.

[0044] In addition, the X-axis coordinates and Y-axis coordinates of other vertices of the window may also be used as the window coordinate information, such as the lower left corner coordinates.

[0045] Step S220: Create a window on the second terminal 20 according to the window coordinate information. Taking the Android system as an example, a picture-in-picture of a specified size can be created through AMS (Activity Manager Service), which is a core service in the Android system framework layer and is responsible for managing the life cycle of the application, the task stack, and the startup and scheduling of components.

[0046] Step S250, obtaining the reversing image data from the virtual screen and displaying it in the window. Taking the Android system as an example, the data on the virtual screen can be obtained by means of Display, which is a module in the Android system for managing display content and is responsible for rendering the application interface on the screen.

[0047] Picture-in-picture mode is a native feature of the Android system. The AMS and Display modules already provide complete support without the need for additional APK development. Through system-level implementation, the memory usage and performance overhead of a single APK can be avoided, improving the overall efficiency of the system. In addition, the system-level picture-in-picture mode can ensure seamless integration with other applications and provide a consistent user experience. In addition to the Android system, for other operating systems, such as Hongmeng and iOS (Apple), the native features of these systems can also be used to implement the picture-in-picture mode.

[0048] See also Figure 4In a specific embodiment of the present invention, the second terminal 20 obtains the reversing image data from the virtual screen and displays it on the second terminal 20 in the form of a picture-in-picture, which specifically includes steps S210 to S250. Among them, step S210, step S220 and step S250 have been described in detail in the previous embodiment, and step S230 and step S240 are described in detail below.

[0049] Step S230, shielding touch events in the window area. To ensure that the reversing image data displayed in the picture-in-picture will not be affected by other touch events on the central large screen, taking the Android system as an example, for example, the event of the area size of the window position can be shielded through the cookAndDispatch method of InputReader.

[0050] Step S240: adjust the window level to the highest level.

[0051] See also Figure 5 In a specific embodiment of the present invention, step S240 includes: S241, setting the level of the window to the highest; S242, re-arranging the level order of all windows on the second terminal 20; S243, requesting the next screen refresh signal for synthesizing the window content after the level order is adjusted. The following takes the Android system as an example to explain each step in detail.

[0052] For example, you can call the setLayerOrder interface through SurfaceFlinger to set the window's layer to the top layer, where SurfaceFlinger is the core component of the Display module and setLayerOrder is a method in SurfaceFlinger. Then trigger rebuildLayerState to make adjustments, which is also a method in SurfaceFlinger. After the adjustment is completed, requestNextVsync is used to request the next VSync for synthesis. RequestNextVsync is a method in DisplayEventReceiver or Choreographer. RequestNextVsync is used to actively request the next VSync when needed, so as to render and synthesize frames. VSync (Vertical Synchronization) is a synchronization signal of the Android display system. It is a technology used to synchronize the refresh rate of the display device and the frame rate of graphics rendering. Its main function is to avoid screen tearing and ensure the smoothness of the picture display.

[0053] After the reversing image data is displayed in picture-in-picture format, you can drag the window to change its position. You can also operate other areas outside the window, open other applications or perform other operations. Since the display priority of the above windows is set, other applications or operations will not be displayed above the window and will not affect the display of the reversing image data.

[0054] It should be noted that the step division of the above methods is only for clear description. When implemented, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they contain the same logical relationship, they are all within the scope of protection of this patent; adding insignificant modifications to the algorithm or process or introducing insignificant designs without changing the core design of the algorithm and process are all within the scope of protection of this patent.

[0055] See also Figure 6 , Figure 6 A reversing image control system provided in one embodiment of the present invention includes a first terminal 10 and a second terminal 20. The first terminal 10 is used to obtain reversing image data and send the reversing image data to a virtual screen of the first terminal 10, which is the instrument terminal of the vehicle; the second terminal 20 is used to obtain the reversing image data from the virtual screen and display it in the form of picture-in-picture on the second terminal 20, which is the central large screen terminal of the vehicle.

[0056] It should be noted that the reversing image control system of this embodiment is a system corresponding to the reversing image control method described above, and the functional modules in the reversing image control system may correspond to the corresponding steps in the reversing image control method. The reversing image control system of this embodiment can be implemented in conjunction with the reversing image control method, that is, in the absence of conflict, the relevant technical details mentioned in the reversing image control method of the above embodiment can also be applied to the reversing image control system of this embodiment.

[0057] In a specific embodiment of the present invention, the operating system of the first terminal 10 is a QNX operating system, and the operating system of the second terminal 20 is an Android or Hongmeng or iOS operating system. In the above embodiment, these operating systems are also mentioned when introducing the first terminal 10 and the second terminal 20, so they will not be repeated here.

[0058] In a specific embodiment of the present invention, the first terminal 10 and the second terminal 20 exchange data through the SomeIP protocol. The SomeIP (Scalable service-oriented MiddlewarE over IP) protocol is a scalable, service-oriented middleware communication protocol based on an IP network, mainly used in automotive electronics, the Internet of Things (IoT) and embedded systems. It aims to achieve efficient, reliable and flexible communication, and supports data exchange and service calls between different nodes in a distributed system. For example, the coordinate information of the window mentioned above, the reversing image data, etc. can all be transmitted through the SomeIP protocol.

[0059] An embodiment of the present invention further provides a vehicle, which includes the above-mentioned reversing image control system.

[0060] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by a person of ordinary skill in the art without departing from the spirit and technical concept disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. A reversing image control method, characterized in that: include: Acquiring reversing image data through a first terminal, and sending the reversing image data to a virtual screen of the first terminal, the first terminal being an instrument terminal of the vehicle; The reversing image data is obtained from the virtual screen through a second terminal and displayed on the second terminal in a picture-in-picture format. The second terminal is a central large screen terminal of the vehicle.

2. The reversing image control method according to claim 1, characterized in that: Acquiring reversing image data through a first terminal and sending the reversing image data to a virtual screen of the first terminal includes: When the vehicle is in reverse gear, acquiring the reversing image data through the first terminal; Sending the reversing image data to the virtual screen of the first terminal; Modify the flag to indicate that the reversing image function is turned on.

3. The reversing image control method according to claim 2, characterized in that: Acquiring the reversing image data from the virtual screen through the second terminal and displaying it on the second terminal in a picture-in-picture format includes: The flag bit is obtained from the first terminal through the second terminal, and whether the reversing image function is turned on is determined according to the flag bit: If yes, acquiring the reversing image data from the virtual screen and displaying it on the second terminal in a picture-in-picture format; If not, the reversing image data is not obtained from the virtual screen.

4. The reversing image control method according to claim 1, characterized in that: Acquiring the reversing image data from the virtual screen through the second terminal and displaying it on the second terminal in a picture-in-picture format includes: acquiring window coordinate information from the first terminal through the second terminal; Creating the window on the second terminal according to the window coordinate information; The reversing image data is acquired from the virtual screen and displayed in the window.

5. The reversing image control method according to claim 4, characterized in that: Acquiring window coordinate information from the first terminal through the second terminal includes: The X-axis coordinate and Y-axis coordinate, width and height of the upper left corner of the window are acquired from the first terminal through the second terminal.

6. The reversing image control method according to claim 4, characterized in that: After the step of creating the window on the second terminal according to the window coordinate information, the method further includes: Shielding touch events within the window area; Set the window's level to the highest level.

7. The reversing image control method according to claim 6, characterized in that: Adjust the level of the above windows to the highest level, including: Setting the level of the window to the highest level; Rearranging the hierarchical order of all windows on the second terminal; Request the next screen refresh signal to synthesize the window contents after adjusting the hierarchy order.

8. A reversing image control system, characterized in that: include: A first terminal is used to obtain reversing image data and send the reversing image data to a virtual screen of the first terminal, where the first terminal is a vehicle instrument terminal; The second terminal is used to obtain the reversing image data from the virtual screen and display it in the form of picture-in-picture on the second terminal, and the second terminal is the central large screen terminal of the vehicle.

9. The reversing image control system according to claim 8, characterized in that: The operating system of the first terminal is the QNX operating system, and the operating system of the second terminal is the Android, Hongmeng or iOS operating system.

10. A vehicle, characterized in that: It includes the reversing image control system as claimed in claim 8.

Images