A method and system for video data collection processing

Abstract

The application provides a video data collection processing method and system, and the method comprises the following steps: S1, establishing a two-dimensional space structure on a memory block, and the two-dimensional space structure is used for storing video data of multiple different cameras; S2, setting working parameters of a receiving device according to memory block information and position information of the video data of the multiple different cameras in the two-dimensional space structure; and S3, after a video data receiving operation is started, the receiving device receives the video data according to the set working parameters. The application eliminates the conversion process in camera data processing, reduces system load, improves product performance, and enhances user experience.

Description

Technical Field

[0001] This invention belongs to the field of video data processing, and more specifically, relates to a method and system for video data acquisition and processing. Background Technology

[0002] The adoption rate of 360-degree panoramic systems in automobiles is gradually increasing, greatly improving the user's driving experience. At the same time, users' requirements for the user experience of 360-degree panoramic systems are also getting higher and higher. How to make full use of existing resources to meet user requirements to the greatest extent with the best cost performance is the goal pursued by technicians.

[0003] Most current panoramic systems also integrate dashcam functionality. However, panoramic systems and dashcams use camera data differently, requiring data conversion to meet different usage needs. As camera resolution increases, the system load for processing this data grows, hindering product performance improvements and severely impacting user experience. Summary of the Invention

[0004] This application provides a method and system for video data acquisition and processing, which eliminates the conversion process in camera data processing, reduces system load, improves product performance, and enhances user experience.

[0005] In a first aspect, embodiments of this application provide a method for video data acquisition and processing, including:

[0006] S1, A two-dimensional spatial structure is established on the memory block, the two-dimensional spatial structure being used to store video data from multiple different cameras;

[0007] S2, based on the memory block information and the position information of video data from multiple different cameras in the two-dimensional spatial structure, set the operating parameters of the receiving device;

[0008] S3, After the video data receiving operation is started, the receiving device receives video data according to the set working parameters.

[0009] Step S1 involves establishing a two-dimensional spatial structure on the memory block. This two-dimensional spatial structure is used to store video data from multiple different cameras, including:

[0010] Based on the number of camera inputs, determine the two-dimensional spatial structure and memory space size of the buffer. The two-dimensional spatial structure has n columns and m rows, and the number of cameras is less than or equal to n*m. The memory space size is calculated based on the data format.

[0011] The memory space refers to a physically contiguous block of memory.

[0012] Step S1, which involves establishing a two-dimensional spatial structure on a memory block to store video data from multiple different cameras, further includes:

[0013] Turn on the camera device in sequence, and set the camera's video parameters and two-dimensional position parameters. The video parameters include the image width, height, and data format. The two-dimensional position parameters include the position of the camera input in the two-dimensional spatial structure, and the position includes the number of rows and columns in the two-dimensional spatial structure.

[0014] Step S1, which involves establishing a two-dimensional spatial structure on a memory block to store video data from multiple different cameras, further includes:

[0015] Add the allocated memory space to the camera's input queue.

[0016] S2, based on the memory block information and the position information of video data from multiple different cameras in the two-dimensional spatial structure, sets the operating parameters of the receiving device, including:

[0017] Based on the video format information and location parameters, calculate the receiving address parameters and initialize the receiving device.

[0018] Y component address: Yaddr = Addr + H * (width * height * n) + L * width;

[0019] Cb component:

[0020] Cbaddr=Addr+m*n*width*height+H*width*height*n / 4+L*width;

[0021] Cr content:

[0022] Craddr=Addr+m*n*width*height*5 / 4+H*width*height*n / 4+L*width;

[0023] Buffer width: width*n;

[0024] width is the width of a single camera image, height is the height of a single camera image, Addr is the starting address of the input buffer, and the camera's position in row H and column L of a two-dimensional spatial structure (m rows, n columns).

[0025] In step S3, after the video data receiving operation is initiated, the receiving device receives video data according to the set operating parameters, including:

[0026] After the receiving process is started, the receiving device receives video data according to the operating parameters;

[0027] The method further includes:

[0028] Wait for all camera video data to be received, then read the data from the receive buffer; distribute the video data to the user; return the used buffer to the camera receive queue; repeat this step.

[0029] Distributing video data to users includes:

[0030] The image from camera 1 is obtained from the merged image and fisheye correction is performed. After correction, the image is sent to the stitching and rendering module for processing. The dashcam module obtains the complete merged image, submits it to the encoder for H264 compression encoding, and outputs the H264 bitstream.

[0031] Secondly, this application provides a system for video data acquisition and processing, comprising:

[0032] A building unit is used to build a two-dimensional spatial structure on a memory block, the two-dimensional spatial structure being used to store video data from multiple different cameras;

[0033] The setting unit is used to set the operating parameters of the receiving device according to the memory block information and the position information of video data from multiple different cameras in the two-dimensional spatial structure.

[0034] After the video data receiving operation is initiated, the receiving device receives video data according to the set operating parameters.

[0035] Thirdly, this application provides a vehicle that includes the aforementioned video data acquisition and processing system.

[0036] The video data acquisition and processing method and system of this application have the following beneficial effects:

[0037] The video data acquisition and processing method of this application includes: S1, establishing a two-dimensional spatial structure on a memory block, the two-dimensional spatial structure being used to store video data from multiple different cameras; S2, setting the operating parameters of the receiving device based on the memory block information and the position information of the video data from multiple different cameras in the two-dimensional spatial structure; S3, after the video data reception operation is initiated, the receiving device receives video data according to the set operating parameters. This application eliminates the conversion process in camera data processing, reduces system load, improves product performance, and enhances user experience. Attached Figure Description

[0038] Figure 1 This is a schematic flowchart of a video data acquisition and processing method according to an embodiment of this application;

[0039] Figure 2 This is a schematic diagram of the video data acquisition and processing method according to an embodiment of this application;

[0040] Figure 3 This is a schematic diagram of the functional unit division in the video data acquisition and processing method of this application embodiment;

[0041] Figure 4 This is another flowchart illustrating the video data acquisition and processing method according to an embodiment of this application;

[0042] Figure 5 This is a schematic diagram of the system structure for video data acquisition and processing according to an embodiment of this application. Detailed Implementation

[0043] The present application will be further described below with reference to the accompanying drawings and embodiments.

[0044] In the following description, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance. The following description provides multiple embodiments of the invention, which can be substituted or combined with each other. Therefore, this application can also be considered to include all possible combinations of the same and / or different embodiments described. Thus, if one embodiment includes features A, B, and C, and another embodiment includes features B and D, then this application should also be considered to include embodiments containing one or more other possible combinations of A, B, C, and D, even if such embodiments are not explicitly described in the following text.

[0045] The following description provides examples and does not limit the scope, applicability, or examples set forth in the claims. Changes may be made to the function and arrangement of the described elements without departing from the scope of this application. Various processes or components may be appropriately omitted, substituted, or added to the examples. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Furthermore, features described with respect to some examples may be combined into other examples.

[0046] like Figure 1 As shown, the video data acquisition and processing method of this application includes: S1, establishing a two-dimensional spatial structure on a memory block, the two-dimensional spatial structure being used to store video data from multiple different cameras; S2, setting the operating parameters of the receiving device based on the memory block information and the position information of the video data from multiple different cameras in the two-dimensional spatial structure; S3, after the video data acquisition operation is started, the receiving device receives video data according to the set operating parameters.

[0047] This application eliminates the conversion process in camera data processing, reduces system load, improves product performance, and enhances user experience.

[0048] The adoption rate of 360-degree panoramic systems in automobiles is gradually increasing, greatly improving the user's driving experience. A 360-degree panoramic system includes multiple cameras, such as a front-view camera, a rear-view camera, a left-view camera, and a right-view camera, simultaneously capturing images from all around the vehicle. These images are then processed by a series of intelligent algorithms in the image processing unit to ultimately form a panoramic top-down view of the vehicle's surroundings, displayed on the screen. This intuitively presents the vehicle's position and surrounding environment, allowing drivers to handle situations such as starting the vehicle, turning, parking, passing oncoming traffic in narrow lanes, and avoiding obstacles with ease and confidence.

[0049] A 360-degree surround view system provides drivers with a new, previously unavailable perspective of the vehicle. It typically uses four cameras and an ECU (Electronic Control Unit). These perspectives are extremely useful for drivers in complex situations, such as blind spot detection when changing lanes on highways and safely maneuvering a vehicle out of a parallel parking space. A 360-degree surround view system offers a vehicle-centric view, a rearview camera system, a mirror system, and a next-generation 3D panoramic imaging system. It relies on four fisheye cameras with a horizontal field of view greater than 180° to detect the vehicle's surroundings, and by combining these four cameras, an image of the vehicle's surroundings from any angle can be created.

[0050] The three main functions of a 360-degree panoramic monitoring system are: 1. Four-channel camera monitoring, 2. Dashcam, and 3. Vibration-based anti-theft monitoring when the engine is off. The 360° panoramic monitoring system achieves blind-spot-free operation through four cameras located at the front, sides, and rear of the vehicle. While most current panoramic systems also integrate dashcam functionality, the way panoramic systems and dashcams use camera data differs, necessitating data conversion to meet the needs of different applications.

[0051] A dashcam (DVR) is an instrument that records video and audio information related to a vehicle's movement. After installation, a dashcam can record video and audio of the entire driving process, providing evidence in case of traffic accidents. A dashcam includes: (1) a main unit: including a microprocessor, data storage, real-time clock, display, and lens module; (2) a vehicle speed sensor; and (3) data analysis software.

[0052] like Figure 2-4As shown, this invention uses a contiguous memory block and establishes a two-dimensional spatial structure on this memory block. Each block stores video data from different cameras, effectively completing a merging process. Users can specify the positions of different cameras in the two-dimensional space. During runtime, the application program transmits memory block information (address and two-dimensional space description information) and the camera's position information in the two-dimensional space to the kernel's driver module. The kernel driver module uses the information from the application program to set the operating parameters of the receiving module. After the user initiates the receiving operation, the hardware module responsible for receiving data will place the video data in the designated position according to the set parameters. Once all camera receiving modules have completed receiving one frame of data, the user will obtain a synthesized, complete image from multiple cameras.

[0053] Specifically, the method for acquiring and processing video data in this application includes:

[0054] Step S1: Based on the number of camera inputs in the product, plan the two-dimensional spatial structure and memory space size of the input buffer. The number of cameras <= n*m, where n and m are the number of columns and rows in the two-dimensional structure of the buffer. The memory space is calculated based on the video data format, and physical contiguous memory blocks are allocated. Since the allocated memory blocks need to be provided to the device's DMA (Direct Memory Access), and some devices' DMA does not support non-contiguous address memory operations, the memory blocks in this application are contiguous physical spaces, improving the versatility of this application. In existing technologies, each camera independently allocates image storage space, and the storage spaces of each camera are independent of each other.

[0055] DMA (Direct Memory Access) does not involve saving or restoring the current context during data transfer. Since the CPU does not participate in the transfer operation, it eliminates the need for CPU operations such as fetching instructions, fetching data, and sending data. Memory address modification and word count counting are also implemented directly in hardware, not in software. Therefore, DMA can meet the requirements of high-speed I / O devices and also improves CPU efficiency.

[0056] When a device interface attempts to send data (typically a large volume) directly to another device via the bus, it first sends a DMA request signal to the CPU. The peripheral device, through a dedicated DMA interface circuit—the DMA controller (DMAC)—requests to take over bus control from the CPU. Upon receiving this signal, the CPU, at the end of the current bus cycle, responds to DMA signals according to their priority and the order in which they were requested. When the CPU responds to a DMA request from a device interface, it relinquishes bus control. Thus, under the management of the DMA controller, the peripheral device and memory exchange data directly without CPU intervention. After the data transfer is complete, the device interface sends a DMA end signal to the CPU, returning bus control.

[0057] Step S2: Sequentially open the input camera devices (e.g., / dev / video0... / dev / videoX), and set the camera's video parameters (e.g., data format, video image width, video image height, etc.) and two-dimensional position parameters (the position (row, column) of this camera input in the two-dimensional spatial structure of the input buffer). In existing technologies, it is not necessary to specify the two-dimensional position information of the camera device.

[0058] Step S3: Add the allocated buffer to the camera's input queue. In existing technology, each camera uses a different buffer.

[0059] Step S4: Calculate the receiving address parameters based on the video format information and two-dimensional position parameters, initialize the receiving device, and start the receiving process.

[0060] For example, NV21 format:

[0061] The image width and height of a single camera, the starting address of the input buffer is Addr, and the camera is located at row H and column L of a two-dimensional structure (m rows, n columns);

[0062] Y component address: Yaddr = Addr + H * (width * height * n) + L * width;

[0063] Cb component:

[0064] Cbaddr=Addr+m*n*width*height+H*width*height*n / 4+L*width;

[0065] Cr content:

[0066] Craddr=Addr+m*n*width*height*5 / 4+H*width*height*n / 4+L*width;

[0067] Buffer width: width*n.

[0068] The method of calculating addresses in this application is different from existing technologies; the input of each camera is treated as a sub-block of a large buffer.

[0069] Step S5: Wait for all devices to finish receiving video, then read the data from the receive buffer.

[0070] Step S6: Distribute the data to user 1, for example:

[0071] 1. The AVM (Around View Monitor) acquires images from camera 1 (2,3,4) in the merged image, performs fisheye correction, and then sends the corrected image to the stitching and rendering module for processing.

[0072] 2. The recorder module acquires the complete merged image, submits it to the encoder for H264 compression encoding, and outputs the H264 bitstream.

[0073] Step S7: Return the used buffer to the camera receiving queue.

[0074] Repeat step S5.

[0075] This invention reduces system performance consumption by more than 20%. Specifically, it enables true full HD operation of four 1080P cameras on existing mainstream chip platforms, meeting user needs without requiring higher-performance and more expensive chips, thus improving the product's cost-effectiveness. This application improves product performance and reduces product cost.

[0076] like Figure 5 As shown, this application also provides a video data acquisition and processing system, including: a setup unit 201, used to establish a two-dimensional spatial structure on a memory block, the two-dimensional spatial structure being used to store video data from multiple different cameras; a setting unit 202, used to set the operating parameters of a receiving device according to the memory block information and the position information of the video data from multiple different cameras in the two-dimensional spatial structure; and a receiving unit 203, after the video data receiving operation is started, the receiving device receives video data according to the set operating parameters.

[0077] In this application, the system embodiments for video data acquisition and processing are basically similar to the method embodiments for video data acquisition and processing. For relevant details, please refer to the description of the method embodiments for video data acquisition and processing.

[0078] This application also provides a vehicle that includes the aforementioned video data acquisition and processing system.

[0079] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the above-described video data acquisition and processing method steps. The computer-readable storage medium may include, but is not limited to, any type of disk, including floppy disks, optical disks, DVDs, CD-ROMs, microdrives, as well as magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory devices, magnetic cards or optical cards, nanosystems (including molecular memory ICs), or any type of medium or device suitable for storing instructions and / or data.

[0080] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components may be combined, or integrated into another system, or some features may be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed may be through some interfaces, indirect coupling or communication connection of devices or units, and may be electrical, mechanical, or other forms.

[0081] In the various embodiments of the present invention, all functional units can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0082] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method of video data retrieval processing, the method comprising: Comprise: S1, a two-dimensional space structure is established on the memory block, and the two-dimensional space structure is used for saving video data of a plurality of different cameras; The memory space is a memory block which is continuous in a physical space; S2, according to the information of the memory block, the position information of the video data of a plurality of different cameras in the two-dimensional space structure, the working parameters of the receiving device are set; comprising: According to the video format information and the position parameter, the receiving address parameter is calculated, and the receiving device is initialized, Y component address: Yaddr=Addr+H*(width*height*n)+L*width; Cb component: Cbaddr=Addr+m*n*width*height+H*width*height*n / 4+L*width; Cr component: Craddr=Addr+m*n*width*height*5 / 4+H*width*height*n / 4+L*width; Buffer width: width*n; Width is the image width of a single camera, height is the image height of a single camera, Addr is the first address of the input buffer, the two-dimensional space structure comprises m rows and n columns, and the camera is located at the H row and L column position in the two-dimensional space structure; S3, after the video data receiving operation is started, the receiving device receives the video data according to the set working parameters;Including: after starting the receiving process, the receiving device receives the video data according to the working parameters; Further comprising: waiting for all camera video data to be received, reading the data of the receiving buffer;The video data is distributed to the user;The used buffer is returned to the camera receiving queue;Repeat the step.

2. The method for video data collection processing according to claim 1, wherein, Step S1, a two-dimensional space structure is established on the memory block, and the two-dimensional space structure is used for saving video data of a plurality of different cameras, comprising: According to the number of camera inputs, the two-dimensional space structure of the buffer and the memory space size are determined, the number of columns of the two-dimensional space structure is n, the number of rows is m, and the number of cameras is less than or equal to n*m;The memory space size is calculated according to the data format.

3. The method for video data retrieval processing according to claim 1, wherein, Step S1, a two-dimensional space structure is established on the memory block, and the two-dimensional space structure is used for saving video data of a plurality of different cameras, further comprising: The camera devices are opened in turn, the video parameters and two-dimensional position parameters of the camera are set, the video parameters include image width, height and data format, the two-dimensional position parameters include the position of the camera in the two-dimensional space structure, and the position includes the number of rows and columns in the two-dimensional space structure.

4. The method of video data collection processing according to any one of claims 1-3, wherein, Step S1, a two-dimensional space structure is established on the memory block, and the two-dimensional space structure is used for saving video data of a plurality of different cameras, further comprising: the allocated memory space is added to the input queue of the camera.

5. The method of video data collection processing according to any one of claims 1-3, wherein, The video data is distributed to the user, comprising: The image of the camera is obtained in the merged image to do fish eye correction, and the image is sent to the splicing and rendering module for processing after correction;The complete merged image is obtained by the driving recorder module, submitted to the encoder for H264 compression coding, and the H264 code stream is output.

6. A system for video data harvesting processing, characterized by Comprise: The establishing unit is configured to establish a two-dimensional space structure on the memory block, and the two-dimensional space structure is configured to store video data of a plurality of different cameras; The memory space is a memory block that is continuous in physical space; The setting unit is configured to set working parameters of the receiving device according to information of the memory block and position information of the video data of the plurality of different cameras in the two-dimensional space structure, to calculate receiving address parameters according to video format information and position parameters, and to initialize the receiving device, Y component address: Yaddr = Addr + H * (width * height * n) + L * width; Cb component: Cbaddr = Addr + m * n * width * height + H * width * height * n / 4 + L * width; Cr component: Craddr = Addr + m * n * width * height * 5 / 4 + H * width * height * n / 4 + L * width; Buffer width: width * n; width is the image width of a single camera, height is the image height of a single camera, Addr is the first address of an input buffer, the two-dimensional space structure includes m rows and n columns, and the camera is located at the Hth row and the Lth column of the two-dimensional space structure; The receiving unit is configured to receive video data according to the set working parameters after the video data receiving operation is started. After the receiving process is started, the receiving device receives video data according to the working parameters. After all the video data of the cameras is received, data in the receiving buffer is read. The video data is distributed to users, and the used buffer is returned to the camera receiving queue.

7. An automobile characterized by comprising: A system including the video data receiving process of claim 6.

Images