Multi-camera background recording method and system based on Android13 vehicle system

Abstract

The present invention belongs to the field of vehicle camera recording technology, and specifically relates to a multi-camera background recording method and system based on an Android13 vehicle system, including: initializing camera management; periodically detecting the connection status of the camera, and displaying a real-time video preview of the connected camera on the user interface of the vehicle system when at least one camera is detected to be connected; responding to user operation or vehicle system triggering, starting the background recording function of at least one camera; when the user exits the preview interface or the vehicle system switches to background operation, maintaining the continuous operation of the background recording function; when the user re-enters the preview interface, resuming the display of the camera's real-time video preview and displaying the time information of continuous recording; responding to user operation, stopping the camera's background recording function, and saving the recorded content to an external storage device. The present invention can realize simultaneous preview and recording of multiple cameras.

Description

Technical Field

[0001] The present invention belongs to the field of vehicle-mounted camera recording technology, and specifically relates to a multi-camera background recording method and system based on the Android13 vehicle-mounted system. Background Art

[0002] With the rapid development of the new energy vehicle industry, the functional requirements of vehicle systems are becoming increasingly rich, among which multi-camera simultaneous preview and recording have become important functional requirements of vehicle systems. However, in vehicle systems before Android 13, the implementation of multi-cameras has the following technical problems: (1) System compatibility issues: The native SDK does not support the simultaneous use of multiple cameras and needs to be implemented through custom interfaces; the custom interfaces of different vehicle systems are incompatible with each other, resulting in high development costs; the code portability is poor and difficult to reuse between different projects; there is a potential conflict with the Extended View System (EVS), which affects system performance. (2) Data processing efficiency issues: The YUV data transmission process needs to be manually processed; the explicit transmission of data in the program increases the risk of errors; there is uncertainty in compatibility with hard encoding and decoding; there are multiple redundant copies in the data processing process. (3) Functional flexibility issues: The preview and recording functions are strongly coupled and difficult to control independently; switching the preview state may cause recording interruption; background recording is complex to implement and has poor stability; camera resource management is inefficient. (4) Development and maintenance cost issues: Dedicated interfaces need to be developed for different vehicle models; high code complexity and difficult maintenance; re-adaptation is required when the system is upgraded; and the workload of testing and verification is large. (5) Performance overhead issues: Custom interface solutions lead to low system resource utilization; high CPU usage during data processing; poor memory usage efficiency; and relatively high power consumption.

[0003] These technical problems seriously restrict the development efficiency and user experience of the vehicle-mounted multi-camera system: extend the product development cycle; increase project development costs; affect the system operation stability; and reduce the user experience.

[0004] With the release of Android 13, its native Camera2 API added support for simultaneous access to multiple cameras, providing new possibilities for solving the above technical problems. This technological advancement has created conditions for the development of a more efficient and stable in-vehicle multi-camera system. Summary of the invention

[0005] Purpose of the invention: The purpose of the present invention is to address the deficiencies in the prior art and provide a multi-camera background recording method and system based on the Android 13 vehicle-mounted system, which can realize simultaneous preview and recording of multiple cameras.

[0006] Technical solution: The multi-camera background recording method based on the Android13 vehicle-mounted system of the present invention comprises the following steps:

[0007] S1. Initialize camera management to maintain the connection status of several cameras connected to the vehicle system;

[0008] S2, periodically detecting the connection status of the cameras, and when detecting that at least one camera is connected, displaying a real-time video preview of the connected camera on the user interface of the vehicle system;

[0009] S3, in response to a user operation or a vehicle system trigger, starting a background recording function of at least one camera, and storing the recorded video data in an external storage device;

[0010] S4. When the user exits the preview interface or the vehicle system switches to background operation, the background recording function is kept running continuously;

[0011] S5. When the user re-enters the preview interface, the real-time video preview of the camera is restored and the time information of the continuous recording is displayed;

[0012] S6. In response to the user operation, stop the background recording function of the camera and save the recorded content to an external storage device.

[0013] To further improve the above technical solution, the initialization in S1 includes: creating a camera mapping table, wherein the mapping table stores identification information of several cameras and corresponding preview parameters.

[0014] Furthermore, the periodic detection in S2 includes: periodically detecting the connection status of each camera in the vehicle-mounted system and obtaining the node information of the camera; judging whether the camera is online based on the device node information, and updating the connection status information of the camera based on the status change; when a connection change is detected, updating the camera mapping table, and adjusting the camera preview interface in the vehicle-mounted system according to the new mapping table.

[0015] Furthermore, starting the background recording function includes: detecting the status of the external storage device; configuring encoding parameters; enabling real-time preview and background recording functions; using MediaMuxer to encapsulate the video and complete the processing of the video file; updating the file index after recording is completed and managing the storage space.

[0016] Furthermore, during the execution of S2 to S6: the state of the CameraDevice object is kept unchanged, and by dynamically adjusting the Surface list of CameraCaptureSession, the rendering object is switched between only preview Surface, only recording Surface, and both preview and recording Surface, thereby achieving seamless conversion between the three working modes of preview only, recording only, and preview and recording.

[0017] Furthermore, when recording is not enabled and you enter the preview page for the first time, follow these steps to implement preview-only mode:

[0018] Get all camera IDs supported by the current device through CameraManager;

[0019] Call the CameraManager.openCamera function according to the id to open the corresponding camera and obtain the CameraDevice object of the camera, so that the camera is in the open state;

[0020] Call CameraDevice.createCaptureSession to create a CameraCaptureSession object for each camera to manage the camera preview and recording session; for the camera that needs to be previewed, create a SurfaceView object for display and get its associated Surface to display real-time video stream data;

[0021] Call CameraDevice.createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW) to create a capture request and add the Surface object in SurfaceView to the request;

[0022] For each camera object that needs to be previewed, call CaptureSession.setRepeatingRequest to start the preview function of the corresponding camera. The camera will continue to transmit real-time video stream data to Surface and display it on the preview interface;

[0023] Preview and record: In the preview interface, the user chooses to start recording, and stops the current session that only contains preview by calling CaptureSession.stopRepeating. Then, a new CaptureSession object is created, and the preview and recording Surface objects are added to it. Then, CaptureSession.setRepeatingRequest is called again to start a new session, thus achieving seamless switching from preview only to preview and recording.

[0024] When the user exits the preview interface or the vehicle system switches to background operation, the system determines the recording status of each camera. If it is recording, first stop the current CameraCaptureSession by calling CaptureSession.stopRepeating, then call createCaptureSession to create a new CaptureSession, pass the Surface list parameter to the recording Surface, call CaptureSession.setRepeatingRequest to turn it on, and switch to the recording state without preview;

[0025] When the user re-enters the preview interface, first stop the current CameraCaptureSession that only contains the recording Surface by calling CaptureSession.stopRepeating, call createCaptureSession and pass in the preview and recording Surfaces, create a new CaptureSession that contains preview and recording, call and call CaptureSession.setRepeatingRequest to switch to the preview and recording state;

[0026] Stop preview and continue recording: First stop the current CameraCaptureSession that contains the recording and preview Surfaces by calling CaptureSession.stopRepeating, call createCaptureSession and pass in the recording Surface, create a new CaptureSession that contains the recording, call CaptureSession.setRepeatingRequest to turn it on, and exit the page directly without turning off the camera to ensure that the background recording function continues to run.

[0027] Furthermore, the detection of the status of the external storage device includes: polling the external disk manager at specified intervals to detect available external storage devices; when a change in the status of the external storage device is detected, updating the recording enable status; when the external storage device is removed, stopping all ongoing recording tasks, saving the recorded video files and updating the file index; when it is detected that the external storage device is reinserted, resuming the available recording tasks.

[0028] Furthermore, the method also includes video file management steps: generating a unique file name based on a timestamp; automatically segmenting when the recording duration reaches a preset threshold; using MediaMuxer to encapsulate the video and complete the processing of the video file; updating the file index after the recording is completed and sorting the video files by file name.

[0029] The system for implementing the above-mentioned multi-camera background recording method based on the Android13 vehicle system includes:

[0030] Mount the monitor module to periodically poll the status of the external storage device and dynamically adjust the recording enable;

[0031] The camera scanning module is used to detect changes in camera status and dynamically update the interface layout;

[0032] File storage module, used to manage the creation and writing of video files;

[0033] Recording module, used to handle the encoding and transmission of video data;

[0034] The file management module is used to manage recording files.

[0035] Furthermore, the mount listener module polls the external disk manager at intervals of 5 seconds; updates the recording enable status when the storage device status changes; safely closes the recording thread when the storage device is removed; the camera scanning module polls the camera list at intervals of 5 seconds; dynamically adjusts the layout when cameras are added; and performs exception handling when cameras are removed.

[0036] Beneficial effect: Compared with the prior art, the advantages of the present invention are: the method provided by the present invention is applied to devices equipped with vehicle-mounted Android13 or above systems. Within the range allowed by the interface, users can freely decide the number of cameras to use, and can hot-swap on the preview page. After monitoring the camera changes, the preview page adaptively modifies the layout and previews the images of all available cameras at the same time; after inserting the external storage device, the recording function can be turned on, and multi-channel recording can be supported at the same time. When the recording function is turned on, you can exit to the main page or enter the video list to play historical recorded files, providing users with a more friendly multi-camera recording experience.

[0037] The three core objects, CameraDevice, CameraCaptureSession and MediaCodec, have clear responsibilities. The preview and recording functions are completely decoupled and can be controlled and switched independently. Flexible function combination and expansion are supported, and system maintenance and upgrade costs are significantly reduced.

[0038] Multi-camera synergy effect: supports opening and using multiple cameras at the same time. Different cameras can independently control the preview and recording status. The cameras do not affect each other, the operation is stable, and it is fully compatible with the EVS system without affecting its performance. State switching effect: realize seamless switching between preview and recording states. There is no screen freeze or black screen during the switching process. The recording process will not be interrupted by state switching. It supports adding or removing preview / recording Surfaces at any time. Background running effect: supports continuing recording when the application retreats to the background. The background recording process is stable and reliable, the system resource usage is reasonable, and the preview state can be restored at any time.

[0039] The present invention combines multiple conveniences such as camera hardware, camera management API, hard encoding and decoding, external storage device management, etc., to provide a universal and concise implementation solution for realizing multi-camera background recording. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a flow chart from entering to exiting the preview page;

[0041] Figure 2 Is a flow chart for monitoring available cameras. DETAILED DESCRIPTION

[0042] The technical solution of the present invention is described in detail below with reference to the accompanying drawings, but the protection scope of the present invention is not limited to the embodiments.

[0043] Embodiment 1: A multi-camera background recording method based on the Android 13 vehicle system includes the following steps:

[0044] S1: Detect camera availability

[0045] Check whether the system recognizes an available camera. If no camera is available, prompt the user.

[0046] S2: Determine the camera recording status

[0047] Iterate over each camera and check its current recording status:

[0048] If the camera is recording, execute S5;

[0049] If the camera is not recording, execute S3.

[0050] S3: Initialize the camera

[0051] Initialize the Camera instance, create a CameraCaptureSession, add a preview Surface, and start the camera preview.

[0052] S4: Click the record button

[0053] Check if the USB drive is connected:

[0054] If the USB flash drive exists, execute S5;

[0055] If the USB drive does not exist, a prompt will pop up to inform the user that the recording media is missing.

[0056] S5: Start recording

[0057] Stop the current CameraCaptureSession, create a new CameraCaptureSession and add the recording Surface and preview Surface to start recording the video.

[0058] S6: Monitor USB disk status

[0059] Real-time monitoring of the plug-in and unplug status of the USB flash drive:

[0060] If the USB flash drive is unplugged or there is insufficient free space, stop recording, recreate the CameraCaptureSession, add a preview Surface, and resume preview.

[0061] S7: Handle user switching to the background

[0062] When the user switches the preview page to the background or exits, the recording status of each camera is determined:

[0063] If recording is in progress, execute S8;

[0064] If there is no recording, close the camera resource directly.

[0065] S8: Stop recording

[0066] Stop the current CameraCaptureSession, create a new CameraCaptureSession, add only the recording Surface, and continue recording.

[0067] S9: Processing the user switching back to the foreground

[0068] When the user switches the preview page back to the foreground, the recording status of each camera is determined:

[0069] If recording is in progress, execute S5 (continue recording).

[0070] If there is no recording, execute S3 (restart preview).

[0071] S10: Stop recording

[0072] The user clicks Stop Recording, stops the CameraCaptureSession, closes all camera resources, and exits the page.

[0073] In the above process: keep the state of the CameraDevice object unchanged, and switch the rendering object between only preview Surface, only recording Surface, and both preview and recording Surface by dynamically adjusting the Surface list of CameraCaptureSession, so as to realize seamless conversion of the three working modes of preview only, recording only, and preview and recording, including the following steps.

[0074] When recording is not enabled and you enter the preview page for the first time, follow these steps to implement preview-only mode:

[0075] Get all camera IDs supported by the current device through CameraManager;

[0076] Call the CameraManager.openCamera function according to the id to open the corresponding camera and obtain the CameraDevice object of the camera, so that the camera is in the open state;

[0077] Call CameraDevice.createCaptureSession to create a CameraCaptureSession object for each camera to manage the camera preview and recording session; for the camera that needs to be previewed, create a SurfaceView object for display and get its associated Surface to display real-time video stream data;

[0078] Call CameraDevice.createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW) to create a capture request and add the Surface object in SurfaceView to the request;

[0079] For each camera object that needs to be previewed, call CaptureSession.setRepeatingRequest to start the preview function of the corresponding camera. The camera will continue to transmit real-time video stream data to Surface and display it on the preview interface;

[0080] Preview and record: In the preview interface, the user chooses to start recording, and stops the current session that only contains preview by calling CaptureSession.stopRepeating. Then, a new CaptureSession object is created, and the preview and recording Surface objects are added to it. Then, CaptureSession.setRepeatingRequest is called again to start a new session, thus achieving seamless switching from preview only to preview and recording.

[0081] When the user exits the preview interface or the vehicle system switches to background operation, the system determines the recording status of each camera. If it is recording, first stop the current CameraCaptureSession by calling CaptureSession.stopRepeating, then call createCaptureSession to create a new CaptureSession, pass the Surface list parameter to the recording Surface, call CaptureSession.setRepeatingRequest to turn it on, and switch to the recording state without preview;

[0082] When the user re-enters the preview interface, first call CaptureSession.stopRepeating to stop the current CameraCaptureSession that only contains the recording Surface, call createCaptureSession and pass in the preview and recording Surfaces, create a new CaptureSession that contains preview and recording, call CaptureSession.setRepeatingRequest to turn it on, and switch to the preview and recording state to ensure the continuity and real-time nature of the recording process;

[0083] Stop preview and continue recording: First stop the current CameraCaptureSession that contains recording and preview Surfaces by calling CaptureSession.stopRepeating, call createCaptureSession and pass in the recording Surface, create a new CaptureSession that contains recording, call and call CaptureSession.setRepeatingRequest to turn it on, and exit the page directly without turning off the camera to keep the background recording function running. The system will ensure that the recording task is not affected and the background recording is still in progress.

[0084] The present invention can achieve the following technical effects: (1) Supporting extended cameras. By maintaining a camera list in the system, cameras can be added and removed. Whenever a camera is inserted or unplugged, the list is updated, and a corresponding CameraCaptureSession is created for each camera, forming an independent thread to handle its preview and recording, so as to achieve simultaneous operation. (2) Supporting hot plugging of cameras. When a new camera is inserted, it is initialized and added to the current preview; when a camera is unplugged, it is checked whether it is recording. If not, the preview of the camera is directly removed. During the preview process, by maintaining the state of the current CameraCaptureSession, it is ensured that the newly inserted camera can be initialized and added to the preview without affecting the current preview. (3) During recording, if a new camera is inserted, the camera is added to the recording list and recording starts; if a camera that is not recording is unplugged, the recording of other cameras is not affected. During the recording process, new cameras can be dynamically added for recording, but it should be ensured that their recording states are correctly managed. (4) Implementing a switching mechanism allows switching to a no-preview state during recording. This can be achieved by stopping the preview of the Surface display but continuing to record in the background, ensuring that the camera recording state is maintained when there is no preview, avoiding data loss or recording interruption.

[0085] Example 2: Hardware preparation: 1 device equipped with Android 13 vehicle system, connect at least 1 camera supported by the HAL layer to the Android device, and a USB flash drive.

[0086] The specific steps are as follows:

[0087] Step 1: Insert the USB drive and enter the camera preview page;

[0088] Step 2: Click to start recording one of the cameras;

[0089] Step 3: Exit the camera preview page;

[0090] Step 4: Enter the camera preview page again, you can see that the camera that just triggered the recording is still in the recording state, and the recording time is continuous;

[0091] Step 5: Click the button to stop camera recording;

[0092] Step 6: Exit the camera preview page again;

[0093] Step 7: Enter the video list and view the recorded files. The continuous video content is not affected by the preview action switching.

[0094] Through the above operations, you can cover Figure 1The entire process.

[0095] Example 3: Hardware preparation: 1 device equipped with Android 13 vehicle system, 2 cameras supported by the HAL layer on the Android device, and 1 camera connected to the Android device.

[0096] Figure 2 The main logic of supporting hot swapping of multiple cameras is shown. The specific operation steps are as follows:

[0097] Step 1: Enter the preview page and check that there is a camera preview screen;

[0098] Step 2: Add another camera and observe that the preview page switches from 1-channel preview to 2-channel preview in real time;

[0099] Step 3: Unplug a camera and observe that the preview page switches from 2-way preview to 1-way preview in real time.

[0100] Through the above operations, you can cover Figure 2 The entire process.

[0101] Embodiment 4: A multi-camera background recording system based on an Android 13 vehicle-mounted system for implementing the above embodiment includes:

[0102] The mount listener module polls the external disk manager every 5 seconds to check whether there is an available external storage device. When the available external storage changes, when the external storage is mounted, the recording enable is changed to available. When the external storage is unplugged, the recording enable is changed to unavailable and the currently recording thread is closed.

[0103] The camera scanning module polls all cameras in the camera ID list every 5 seconds, compares the last record, and determines whether the camera is recording the device. If so, the recording thread needs to be removed for exception processing;

[0104] The file storage module names the recording file according to the Cam[id]_yyyyMMdd_HHmmss.mp4 rule. After the recording starts, an mp4 file is created and the mp4 header file is written. When receiving each frame of encoded h264 data, the data and timestamp are written to the mp4 file, and the recording time is sent back to the UI layer for display. When the recording ends, the mp4 file tail is written.

[0105] The recording module turns on the camera, turns on mediacodec encoding, encodes each frame of yuv data output by the camera into h264 data, and outputs it to the file storage device;

[0106] The file management module sorts all recorded files by name, allowing users to delete video files or pass the video address to the player for recorded video preview.

[0107] As described above, although the present invention has been shown and described with reference to specific preferred embodiments, it should not be construed as limiting the present invention itself. Various changes in form and details may be made without departing from the spirit and scope of the present invention as defined in the appended claims.

Claims

1. A multi-camera background recording method based on the Android 13 vehicle system, characterized in that: The following steps are involved: S1, initializing camera management, including: creating a camera mapping table, wherein the mapping table stores identification information of a plurality of cameras and corresponding preview parameters thereof, and is used to maintain connection status of a plurality of cameras connected to the vehicle-mounted system; S2, periodically detecting the connection status of the camera, including: periodically detecting the connection status of each camera in the vehicle-mounted system, obtaining the node information of the camera; judging whether the camera is online according to the node information, and updating the connection status information of the camera according to the change of the online status; when the connection status change is detected, updating the camera mapping table, and adjusting the camera preview interface in the vehicle-mounted system according to the new mapping table; when at least one camera is detected to be connected, displaying the real-time video preview of the connected camera on the user interface of the vehicle-mounted system; S3, in response to a user operation or a vehicle system trigger, starting a background recording function of at least one camera, and storing the recorded video data in an external storage device; S4. When the user exits the preview interface or the vehicle system switches to background operation, the background recording function is kept running continuously; S5. When the user re-enters the preview interface, the real-time video preview of the camera is restored and the time information of the continuous recording is displayed; S6. In response to the user operation, stop the background recording function of the camera and save the recorded content to an external storage device; During the execution of S2 to S6: keep the state of the CameraDevice object unchanged, and dynamically adjust the Surface list of CameraCaptureSession to switch the rendering object between only preview Surface, only recording Surface, and both preview and recording Surface, so as to achieve seamless conversion between the three working modes of preview only, recording only, and preview and recording.

2. According to claim 1, the multi-camera background recording method based on the Android13 vehicle system is characterized in that: The background recording function started in S3 includes: Detect the status of external storage devices; Configure encoding parameters; Enable real-time video preview and background recording functions; Use MediaMuxer to encapsulate the video and complete the processing of the video file; Update file index after recording is completed and manage storage space.

3. According to claim 1, the multi-camera background recording method based on the Android13 vehicle system is characterized in that: When recording is not enabled and you enter the preview page for the first time, follow these steps to implement preview-only mode: Get all camera IDs supported by the current device through CameraManager; Call the CameraManager.openCamera function according to the id to open the corresponding camera and obtain the CameraDevice object of the camera, so that the camera is in the open state; Call CameraDevice.createCaptureSession to create a CameraCaptureSession object for each camera to manage the camera preview and recording session; for the camera that needs to be previewed, create a SurfaceView object for display and get its associated Surface to display real-time video stream data; Call CameraDevice.createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW) to create a capture request and add the Surface object in SurfaceView to the request; For each camera object that needs to be previewed, call CaptureSession.setRepeatingRequest to start the preview function of the corresponding camera. The camera will continue to transmit real-time video stream data to Surface and display it on the preview interface; Preview and record: In the preview interface, the user chooses to start recording, and stops the current session that only contains preview by calling CaptureSession.stopRepeating. Then, a new CaptureSession object is created, and the preview and recording Surface objects are added to it. Then, CaptureSession.setRepeatingRequest is called again to start a new session, thus achieving seamless switching from preview only to preview and recording. When the user exits the preview interface or the vehicle system switches to background operation, the system determines the recording status of each camera. If it is recording, first stop the current CameraCaptureSession by calling CaptureSession.stopRepeating, then call createCaptureSession to create a new CaptureSession, pass the Surface list parameter to the recording Surface, call CaptureSession.setRepeatingRequest to turn it on, and switch to the recording state without preview; When the user re-enters the preview interface, first stop the current CameraCaptureSession that only contains the recording Surface by calling CaptureSession.stopRepeating, call createCaptureSession and pass in the preview and recording Surfaces, create a new CaptureSession that contains preview and recording, call and call CaptureSession.setRepeatingRequest to switch to the preview and recording state; Stop preview and continue recording: First stop the current CameraCaptureSession that contains the recording and preview Surfaces by calling CaptureSession.stopRepeating, call createCaptureSession and pass in the recording Surface, create a new CaptureSession that contains the recording, call CaptureSession.setRepeatingRequest to turn it on, and exit the page directly without turning off the camera to ensure that the background recording function continues to run.

4. According to claim 2, the multi-camera background recording method based on the Android13 vehicle system is characterized in that: The detecting the state of the external storage device comprises: Poll the external disk manager at a specified interval to detect available external storage devices; When the status of the external storage device is detected to change, the recording enable status is updated; When the external storage device is removed, all ongoing recording tasks are stopped, the recorded video files are saved and the file index is updated; When it is detected that the external storage device is reinserted, available recording tasks are resumed.

5. According to claim 2, the multi-camera background recording method based on the Android13 vehicle system is characterized in that: Also includes video file management steps: Generate unique file names based on timestamps; Automatically segment the recording when the recording duration reaches the preset threshold; Use MediaMuxer to encapsulate the video and complete the processing of the video file; Update the file index after recording is completed and sort the video files by file name.

6. A system for implementing the multi-camera background recording method based on the Android 13 vehicle-mounted system of claim 1, characterized in that: include: Mount the monitor module to periodically poll the status of the external storage device and dynamically adjust the recording enable; The camera scanning module is used to detect changes in camera status and dynamically update the interface layout; File storage module, used to manage the creation and writing of video files; Recording module, used to handle the encoding and transmission of video data; The file management module is used to manage recording files.

7. The system according to claim 6, characterized in that The mount listener module polls the external disk manager at intervals of 5 seconds; updates the recording enable status when the storage device status changes; and safely closes the recording thread when the storage device is removed; the camera scanning module polls the camera list at intervals of 5 seconds; dynamically adjusts the layout when cameras are added; and performs exception handling when cameras are removed.

Images