Camera simulation data debugging system, method and device based on vehicle-mounted application

Abstract

The application relates to the field of vehicle-mounted operation system control, and particularly discloses a camera simulation data debugging system, method and equipment based on a vehicle-mounted application; the system comprises an operation system applied to a test platform; the system further comprises an application layer, a driving layer and a system framework layer arranged in the operation subsystem; and a camera simulation service subsystem connected with the system framework layer; the test platform acquires a camera application camera function, constructs a camera data simulation service corresponding to the camera application camera function in the operation system, and reconstructs the operation system; the operation subsystem after reconstruction is configured to call the camera simulation service, configure video simulation interfaces and simulation service parameters of the interfaces, acquire camera simulation data, and execute debugging on the camera simulation data. The above scheme can be independent of the development progress and driving progress of each level in the operation subsystem, thereby simplifying an application debugging process, improving development efficiency, and improving the authenticity of data.

Description

Technical Field

[0001] This application relates to the field of vehicle operating system control, and in particular to a camera simulation data debugging system, method and device based on vehicle applications. Background Technology

[0002] Currently, after the camera applications in in-vehicle infotainment systems (such as AVM, RVC, DMS, OMS, DVR, etc.) are developed, debugging requires the completion of the development of all relevant functions at each level of the system, and the connection of hardware cameras or the use of simulation equipment to simulate cameras before debugging can be performed.

[0003] Under this architecture, all modules in the chain need to be developed before they can be integrated and tested in a real hardware environment. This will have a great impact on the development cycle of rapidly iterating applications and increase the overall development and debugging cycle.

[0004] However, the existing development method involves developing each module in the development chain, such as application software, operating system, and hardware, separately, and then integrating and debugging them after development. This development model has relatively low development efficiency and increases the development cycle and development cost significantly. Summary of the Invention

[0005] Therefore, it is necessary to provide a camera simulation data debugging system, method, and device based on vehicle applications to address the above-mentioned technical problems. This system can reduce development cycle and development cost without relying on hardware facilities, and improve development efficiency.

[0006] In a first aspect, this application provides a camera simulation data debugging system based on vehicle applications. The system includes: an operation subsystem applied to a test platform; an application layer, a driver layer, and a system framework layer deployed in the operation subsystem; and a camera simulation service subsystem connected to the system framework layer; the camera simulation service subsystem is used to provide camera simulation services.

[0007] The application layer includes: multiple camera applications with camera functions;

[0008] The testing platform includes a platform server;

[0009] The platform server is equipped with a video simulation interface; the video simulation interface is used to reconstruct the system framework layer and the camera simulation service subsystem.

[0010] The testing platform is based on the reconstructed system framework layer and camera simulation service subsystem to enable debugging of camera simulation data;

[0011] The video simulation interface includes: a first reconstruction module, used to obtain the camera function of the camera application, and to construct a camera abstract module corresponding to the camera function of the camera application in the system framework layer, so as to meet the driving requirements of the driver layer;

[0012] The second reconstruction module is used to add a video stream adaptation server corresponding to the camera abstraction module to the camera simulation service subsystem; the video stream adaptation server is used to decode the video signal to obtain the original camera data.

[0013] The driver layer is provided with a camera driver module; the camera driver module has an input end connected to the camera application, used to receive camera data obtained by the remote video server from the target object in the virtual environment and convert it into a video signal; the output end is connected to the camera abstraction module, used to drive the camera abstraction module;

[0014] The camera abstraction module is used to receive video signals and call the camera simulation service to perform camera data simulation.

[0015] Optionally, the application layer further includes: a camera application interface module; the camera application interface module is used to provide at least one camera application interface corresponding to the camera application, so as to realize the calling of the camera simulation service provided by the camera simulation service subsystem;

[0016] The camera applications include any one of the following: panoramic image processor (AVM), circumferential vision processor (RVC), driver monitoring module (DMS), passenger monitoring module (OMS), and forward / backward vision processor (DVR).

[0017] Optionally, the camera simulation service subsystem further includes: a simulation image building module connected to the video stream adaptation server;

[0018] The simulation image building module is used to build a simulation image of the camera abstract module after the camera application development is completed, compile it into an image file, and deploy it to run in a virtual environment.

[0019] During operation, the camera abstraction module configures the video simulation interface and the simulation service parameters of the video simulation interface through the camera application interface to obtain camera simulation data.

[0020] Optionally, the video simulation interface includes: a physical interface and a software protocol layer interface;

[0021] The physical interface is used to configure service information; the configuration service information includes service IP and service port information.

[0022] The software protocol layer interface is used to configure the number of video streams, resolution, and frame rate in the simulation.

[0023] Optionally, the platform server includes:

[0024] Define the module, which is used to specify the camera kernel driver adapter file for the camera driver module;

[0025] The configuration module is used to configure the camera kernel driver adapter file into the operating subsystem.

[0026] Optionally, the camera driving module includes:

[0027] A control unit is used to control the camera of the remote video server by acquiring the camera kernel driver adapter file;

[0028] The receiving unit is used to receive the content obtained by the camera from capturing images of target objects in the virtual environment and transmit it back to the camera application.

[0029] Optionally, the camera abstraction module receives a call instruction sent by the platform server based on the physical interface, and responds to the configuration information of the physical interface contained in the call instruction; so that after the camera application development is completed, the camera abstraction module calls the camera simulation service through the camera application interface, and constructs a virtual environment through the video stream adaptation server and the simulation image building module connected to the camera abstraction module; so that the camera abstraction module application runs in the virtual environment.

[0030] Optionally, the simulation image building module receives a call instruction sent by the platform server based on the software protocol layer interface, and in response to the simulation service parameters of the software protocol layer interface included in the call instruction, calls the camera abstraction module so that the simulation image building module runs the camera abstraction module in a virtual environment for development, and compiles the developed camera abstraction module into ROM to achieve data integration and debugging in the virtual environment.

[0031] Secondly, this application provides a method for debugging camera simulation data based on vehicle applications, including:

[0032] Obtain the camera function of the camera application, build a camera data simulation service corresponding to the camera function of the camera application in the operating system, and reconstruct the operating system;

[0033] The reconstructed operating system receives camera simulation data obtained by calling the camera simulation service, configuring the video simulation interface and the simulation service parameters of the video simulation interface, and then performs debugging on the camera simulation data.

[0034] Thirdly, the present invention provides an electronic device including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the method steps as described in any one of the second aspects.

[0035] The above-mentioned camera simulation data debugging system, method and device based on vehicle application, as a debugging technology solution, adopts the simulation method and realizes camera simulation by connecting to remote service. It can be debugged without kernel and camera, reducing debugging cycle and development cost.

[0036] This application reconstructs the camera abstraction module in the framework layer of the operating system and compiles it into a system image. This image can be deployed to a virtual machine environment or a real hardware platform. During runtime, the camera data simulation information can be obtained by configuring the interface of the abstraction module and configuring the parameters of the simulation service.

[0037] The above-mentioned invention does not depend on the development progress of each level in the operation subsystem or the progress of the driving operation subsystem; the testing platform can be used to directly debug the simulation data of the camera during the development of camera applications; thereby simplifying the debugging process of the application, improving development efficiency, and improving the authenticity of the data. Attached Figure Description

[0038] The accompanying drawings used in the specific embodiments or prior art description will be briefly introduced below. In all the drawings, similar elements or parts are generally identified by similar reference numerals. The elements or parts in the drawings are not necessarily drawn to scale.

[0039] Figure 1 This is a schematic diagram of the camera simulation data debugging system based on vehicle applications in an embodiment of the present invention;

[0040] Figure 2 This is a flowchart of a camera simulation data debugging method based on vehicle applications in an embodiment of the present invention;

[0041] Figure 3 This is a diagram of the internal structure of an electronic device in an embodiment of the present invention. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0043] This application provides a camera simulation data debugging system, method, and device based on in-vehicle applications, primarily used in vehicle-mounted systems. The method can be applied to a terminal, a server, or a system including both a terminal and a server, and is implemented through interaction between the terminal and the server. The terminal can be, but is not limited to, various personal computers, laptops, smartphones, tablets, etc.

[0044] In one embodiment, the present invention provides a camera simulation data debugging system for vehicle applications. The embodiment of the present invention will be described below with reference to the accompanying drawings. (Reference) Figure 1 The system specifically includes: an operation subsystem applied to the test platform; an application layer, a driver layer, and a system framework layer deployed in the operation subsystem; and a camera simulation service subsystem connected to the system framework layer; the camera simulation service subsystem is used to provide camera simulation services.

[0045] The application layer includes: multiple camera applications with camera functions; and also includes: a camera application interface module; the camera application interface module is used to provide at least one camera application interface corresponding to the camera application, so as to realize the call of the camera simulation service provided by the camera simulation service subsystem;

[0046] The camera applications include any one of the following: panoramic image processor (AVM), circumferential vision processor (RVC), driver monitoring module (DMS), passenger monitoring module (OMS), and forward / backward vision processor (DVR).

[0047] The test platform includes a platform server; the platform server is equipped with a video simulation interface; the video simulation interface is used to reconstruct the system framework layer and the camera simulation service subsystem;

[0048] The testing platform is based on the reconstructed system framework layer and camera simulation service subsystem to enable debugging of camera simulation data.

[0049] In the above embodiments, the platform server includes:

[0050] Define the module, which is used to specify the camera kernel driver adapter file for the camera driver module;

[0051] The configuration module is used to configure the camera kernel driver adapter file into the operating subsystem.

[0052] In the above embodiments, the video simulation interface includes: a first reconstruction module, used to obtain the camera function of the camera application, and to construct a camera abstract module corresponding to the camera function of the camera application in the system framework layer to meet the driving requirements of the driver layer;

[0053] The second reconstruction module is used to add a video stream adaptation server corresponding to the camera abstraction module to the camera simulation service subsystem; the video stream adaptation server is used to decode the video signal to obtain the original camera data.

[0054] In addition, the camera simulation service subsystem also includes: a simulation image building module that connects to the video stream adaptation server;

[0055] The simulation image building module is used to build a simulation image of the camera abstract module after the camera application development is completed, compile it into an image file, and deploy it to run in a virtual environment.

[0056] During operation, the camera abstraction module configures the video simulation interface and the simulation service parameters of the video simulation interface through the camera application interface to obtain camera simulation data.

[0057] In the above embodiments, from a categorical perspective, video simulation interfaces include at least physical interfaces and software protocol layer interfaces;

[0058] The physical interface is used to configure service information; the configuration service information includes service IP and service port information.

[0059] The software protocol layer interface is used to configure the number of video streams, resolution, and frame rate in the simulation.

[0060] The driver layer is provided with a camera driver module; the camera driver module has an input end connected to the camera application, used to receive camera data obtained by the remote video server from the target object in the virtual environment and convert it into a video signal; the output end is connected to the camera abstraction module, used to drive the camera abstraction module;

[0061] The camera abstraction module is used to receive video signals and call the camera simulation service to perform camera data simulation.

[0062] In the above embodiments, the camera driver module includes:

[0063] A control unit is used to control the camera of the remote video server by acquiring the camera kernel driver adapter file;

[0064] The receiving unit is used to receive the content obtained by the camera from capturing images of target objects in the virtual environment and transmit it back to the camera application.

[0065] In the above embodiments, the camera abstraction module receives a call instruction sent by the platform server based on the physical interface, and responds to the configuration information of the physical interface contained in the call instruction; so that after the camera application development is completed, the camera abstraction module calls the camera simulation service through the camera application interface, and constructs a virtual environment through the video stream adaptation server and the simulation image building module connected to the camera abstraction module; so that the camera abstraction module application runs in the virtual environment.

[0066] The simulation image construction module receives a call instruction sent by the platform server based on the software protocol layer interface. In response to the simulation service parameters of the software protocol layer interface included in the call instruction, it calls the camera abstraction module so that the simulation image construction module runs the camera abstraction module in a virtual environment for development and compiles the developed camera abstraction module into the ROM to achieve data integration and debugging in the virtual environment.

[0067] In the above solution, the camera abstraction module of the system architecture layer of the operation subsystem is reconstructed, and the module is compiled into the system image to generate an image file. This image file can be deployed to a virtual machine environment or a real hardware platform for operation. During operation, the camera data simulation information can be obtained by configuring the interface of the camera abstraction module and configuring the parameters of the simulation service.

[0068] Specifically, the Camera HAL can be modified at the system architecture layer (framework) to retain the original interface to the APP, but the implementation method can be changed to call video data from a remote video server to simulate camera data. This can break free from hardware and low-level development limitations, enable rapid debugging, reduce application development cycle, and save development costs.

[0069] ●Development Process

[0070] Camera HAL-SIM requires some expanded configuration interfaces, such as:

[0071] ■Service Information Configuration: Service IP, port, etc.

[0072] ■ Video simulation configuration: number of video channels, resolution, frame rate, etc.

[0073] Understandably, the above interfaces can be configured through the simulation called by the application, or they can be configured by default through a configuration file.

[0074] Based on the same inventive concept, this application also provides a method for debugging a camera simulation data system for vehicle applications. The solution provided by this method is similar to the implementation described in the above system. Therefore, the specific limitations in one or more embodiments of the camera simulation data debugging method for vehicle applications provided below can be found in the limitations of the camera simulation data debugging system for vehicle applications described above, and will not be repeated here. It is understood that the above description of the system also applies to the description of the method.

[0075] In one embodiment, such as Figure 2 As shown, a method for debugging camera simulation data based on vehicle applications is provided, the method comprising:

[0076] S101 Obtains the camera function of the camera application, constructs a camera data simulation service corresponding to the camera function of the camera application in the operating system, and reconstructs the operating system;

[0077] S102 receives camera simulation data obtained by the reconstructed operating system through calling the camera simulation service, configuring the video simulation interface and the simulation service parameters of the video simulation interface, and performs debugging on the camera simulation data.

[0078] The simulation debugging method proposed in the above embodiments can also be extended to other data simulations, such as simulating CAN and vehicle Ethernet data. The main purpose of this method is to simplify the application debugging process and improve development efficiency. As for the authenticity of the data, it depends on the capabilities of the video server, which is not within the scope of this method.

[0079] The above method is an extended function of the test platform. The platform server provides a video simulation interface, including a physical interface and a software protocol layer interface; this time, the software protocol interface is used for further extension. By developing a video stream adaptation server and a hardware abstraction layer at the operating system level, the original video simulation interface is directly connected to the operating system, eliminating intermediate processing layers. When developing camera applications, data debugging can be performed directly without waiting for the completion of intermediate layer development, as shown in the architecture diagram. This eliminates dependence on the system development progress and the driver development progress of remote cameras. The completed Camera-HAL-SIM can also be compiled into ROM, enabling real data integration and debugging in a virtual machine environment, greatly improving development and debugging efficiency.

[0080] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0081] Meanwhile, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor executes the computer program to implement the above-mentioned dynamic height and width limit warning method based on navigation information.

[0082] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the above-described dynamic height and width limit warning method based on navigation information.

[0083] In one embodiment, the computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of any one of steps S101 to S102 of the method.

[0084] In one embodiment, the electronic device may be a terminal, and its internal structure diagram may be as follows: Figure 3 As shown. The electronic device includes a processor, memory, communication interface, display screen, and input device connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements the method described in any one of steps S101 to S102. The display screen can be a liquid crystal display (LCD) or an electronic ink display. The input device can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the device's casing, or an external keyboard, touchpad, or mouse.

[0085] Those skilled in the art will understand that Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0086] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0087] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric memory (FRAM), phase-change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in this application can include at least one of relational and non-relational databases. Non-relational databases can include blockchain-based distributed databases, etc., and are not limited thereto. The processors involved in the various embodiments provided in this application may be general-purpose processors, central processing units, graphics processors, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited thereto.

[0088] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0089] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A camera emulation data debugging system based on vehicle-mounted applications, characterized in that, The system comprises an operation subsystem applied to a test platform, further comprises an application layer, a driver layer and a system framework layer arranged in the operation subsystem, and a camera simulation service subsystem connected with the system framework layer, wherein the camera simulation service subsystem is configured to provide a camera simulation service; The application layer comprises a plurality of camera applications with camera functions; The test platform comprises a platform server; The platform server is provided with a video simulation interface, which is configured to reconstruct the system framework layer and the camera simulation service subsystem; The test platform is configured to realize debugging of camera simulation data based on the reconstructed system framework layer and the camera simulation service subsystem; The video simulation interface comprises a first reconstruction module configured to acquire the camera function of the camera application, construct a camera abstraction module corresponding to the camera function of the camera application in the system framework layer, so as to meet the driving requirements of the driver layer; A second reconstruction module is configured to add a video stream adaptation server corresponding to the camera abstraction module in the camera simulation service subsystem, wherein the video stream adaptation server is configured to decode a video stream of a video signal to obtain original camera data; The driver layer is provided with a camera driving module, wherein an input end of the camera driving module is connected with the camera application and configured to receive camera data obtained by remotely shooting a target object in a virtual environment by a remote video server and convert the camera data into a video signal, and an output end of the camera driving module is connected with the camera abstraction module and configured to drive the camera abstraction module; The camera abstraction module is configured to receive the video signal and call the camera simulation service to simulate data of the camera.

2. The system of claim 1, wherein, The application layer further comprises a camera application interface module, wherein the camera application interface module is configured to provide at least one camera application interface corresponding to the camera application, so as to realize calling of the camera simulation service provided by the camera simulation service subsystem; The camera application comprises any one of an all-vision monitor (AVM), a ring-vision camera (RVC), a driver monitoring module (DMS), an occupant monitoring system (OMS) and a front and rear vision camera (DVR).

3. The system of claim 2, wherein, The camera simulation service subsystem further comprises a simulation image construction module connected with the video stream adaptation server; The simulation image construction module is configured to construct a simulation image of the camera abstraction module after the camera application is developed, compile the simulation image into an image file and deploy the image file to a virtual environment for running; During the running, the camera abstraction module configures a video simulation interface and simulation service parameters of the video simulation interface by calling the camera simulation service through the camera application interface, so as to obtain camera simulation data.

4. The system of claim 1, wherein, The video simulation interface comprises a physical interface and a software protocol layer interface; The physical interface is configured to configure service information, wherein the service information comprises service IP and service port information; The software protocol layer interface is configured to configure a number of simulated video streams, resolutions and frame rates.

5. The system of claim 4, wherein, The platform server comprises: a definition module configured to make a camera kernel driving adaptation file of the camera driving module; A configuration module is configured to configure the camera kernel driver adaptation file into the operation subsystem.

6. The system of claim 5, wherein, The camera driver module comprises: A control unit is configured to control the camera of the remote video server by acquiring the camera kernel driver adaptation file; A receiving unit is configured to receive the content obtained by the camera shooting the target object in the virtual environment and return the content to the camera application.

7. The system of claim 4, wherein, The camera abstraction module receives the calling instruction sent by the platform server based on the physical interface, and responds to the configuration information of the physical interface contained in the calling instruction; So that the camera abstraction module calls the camera simulation service through the camera application interface after the camera application is developed, constructs the virtual environment through the video stream adaptation server and the simulation image construction module connected with the camera abstraction module; So that the camera abstraction module application runs in the virtual environment.

8. The system of claim 7, wherein, The simulation image construction module receives the calling instruction sent by the platform server based on the software protocol layer interface, responds to the simulation service parameters of the software protocol layer interface contained in the calling instruction, calls the camera abstraction module, so that the simulation image construction module runs the camera abstraction module in the virtual environment for development, compiles the developed camera abstraction module into ROM, and realizes data joint debugging in the virtual environment.

Images