Video picture separation display method and system and electronic equipment

Abstract

The invention provides a video picture separation display method and system and electronic equipment, and the method comprises the steps: decoding a spliced video, and obtaining each frame of decoding buffer area data from a buffer area of a decoder; performing picture segmentation on each frame of decoding buffer area data according to a preset display area, obtaining a plurality of area video data and storing the area video data in corresponding area buffer areas; and playing at least one of the spliced video and the plurality of regional video data. According to the method provided by the invention, when the recorded video is played back in the central control screen of the vehicle, the user can select the picture of the specific camera for full-screen viewing, so that the convenience of product use is improved, and the use experience of the user is remarkably improved.

Description

Technical Field

[0001] The present application belongs to the field of video processing and display technology, and specifically relates to a video screen separation display method, system and electronic equipment. Background Art

[0002] The dash cam is one of the essential features of modern cars. With the reduction in the cost of on-board cameras and the enhancement of the computing power of smart cockpit chips, vehicle manufacturers are increasingly adopting multi-camera solutions in dash cam products and recording videos through camera stitching technology. However, when it is necessary to play back the recorded video, only the complete stitched picture can be displayed, and users cannot select the picture of a specific camera for full-screen viewing. Since the video display content of a certain camera in the stitched picture only occupies one-quarter or one-half of the area, it cannot be partially enlarged, and the size of the on-board central control screen is generally not large, so it is impossible for the human eye to see the details clearly, which is very inconvenient for users to use and the effect is not ideal, which significantly reduces the user experience. Summary of the invention

[0003] In order to solve the above technical problems, the present application proposes a video screen separation display method, system and electronic device that can clearly see the video details and has high convenience and high user experience.

[0004] Specifically, the present application proposes a video screen separation display method, comprising: S100, decoding the spliced ​​video, and obtaining each frame of decoding buffer data from the buffer of the decoder.

[0005] S200, performing screen segmentation on each frame of the decoding buffer data according to a preset display area, obtaining a plurality of regional video data and storing them in corresponding regional buffers.

[0006] S300, playing at least one of the spliced ​​video and the plurality of regional video data.

[0007] In the above technical solution, the decoded buffer data obtained after decoding is segmented according to the preset display area, and the screen of the segmented regional video data is played, so that when the recorded video is played back on the vehicle's central control screen, the screen of a specific camera can be viewed in full screen according to the user's selection, so that the user can clearly see the details in the played back video, which provides convenience for the user and improves the user experience.

[0008] As an implementation manner, the step S100 may include: The data of each frame of decoded buffer is obtained by creating a surface texture instance through GLES.

[0009] The surface texture can update the texture content of each frame of decoded buffer data in the background thread, reducing rendering delay and blocking. The surface texture provides a more efficient data transmission method for the transmission of each frame of decoded buffer data.

[0010] Furthermore, in step S200, the preset display area is pre-set by the GLES. By setting the preset display area by the GLES to perform screen segmentation on each frame of the decoded buffer data, the final displayed picture can be smoother, and the screen segmentation of the decoded buffer data can be more efficient, thereby ensuring high performance and efficiency of video picture processing.

[0011] Furthermore, the step S200 may further include: The regional video data in the regional buffer is passed to the corresponding surface interface component through the exchange buffer method in GLES, and the surface view component obtains the corresponding regional video data through the surface interface component. Among them, the exchange buffer method is a function in GLES used to exchange the front and back buffers. Through the exchange buffer method, the current drawing content can be exchanged from the background buffer to the front buffer, so that the new image is presented to the corresponding view component. The surface interface component is the native platform interface of OpenGLES. Through the surface interface component, the regional video data can be rendered to a separate surface view component, which can achieve efficient graphics drawing. At the same time, the surface interface component provides the ability to manage graphics resources, and clean up and release these resources after rendering is completed. This management capability helps to ensure the effective use and release of system resources and avoid memory leaks.

[0012] Furthermore, the step S300 may further include: The spliced ​​video is played by controlling the media player component, and the plurality of surface view components are controlled to display at least one of the plurality of regional video data. When the user selects any video display area, the surface view component enlarges and displays the regional video data corresponding to the video display area in full screen.

[0013] By displaying the regional video data, when the user specifies the display content of a specific area, the display content can be displayed in full screen, so that the user can see the details in the playback video, which provides convenience for the user and improves the user experience.

[0014] Based on the same inventive concept, the present application also proposes a system for a video screen separation display method, the system comprising: a video decoding module, a segmentation module and a playback module; The video decoding module is used to decode the spliced ​​video and store it in the buffer of the decoder; The segmentation module is used to obtain each frame of decoding buffer data from the buffer of the decoder, and perform screen segmentation on each frame of decoding buffer data according to a preset display area, obtain multiple area video data and store them in corresponding area buffers; The playing module is used to play the spliced ​​video and at least one of the multiple regional video data.

[0015] The segmentation module includes a surface interface component and a GLES component; the GLES component is used to perform screen segmentation on each frame of decoded buffer data according to the preset display area to obtain the multiple regional video data; the surface interface component is used to transmit the multiple regional video data to the corresponding surface view component.

[0016] The playback module includes a media player component and a plurality of surface view components; the media player component is used to play the spliced ​​video; and the surface view component is used to display the regional video data.

[0017] The decoding buffer data obtained after decoding by the video decoding module is segmented by the segmentation module, and played and displayed by the playback module, so that the video that was originally unable to be partially enlarged can be locally enlarged in high definition, so that the user can clearly view the details of the video content, thereby improving the user experience.

[0018] Based on the same inventive concept, the present application also proposes an electronic device, which includes a memory storing executable program code and a processor coupled to the memory, and the processor calls the executable program code stored in the memory to execute the video screen separation display method.

[0019] Compared with the prior art, this application has at least the following beneficial effects: After decoding the acquired stitched video, the present application divides the screen of each frame of decoding buffer data obtained by decoding according to the preset display area to obtain multiple regional video data, plays the stitched video and displays the multiple regional video data according to the received instructions, so that when the user selects the area that needs to be displayed in full screen as needed, the display content of the area can be displayed in full screen, so that the details in the video can be viewed more clearly, thereby significantly improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a flowchart of a video screen separation display method shown in an embodiment of the present application.

[0021] Figure 2 It is a schematic diagram of a video screen separation display system shown in an embodiment of the present application. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0023] Embodiment 1: Please refer to Figure 1 The video screen separation display method mainly includes steps S100 to S300.

[0024] Wherein, step S100 includes: decoding the spliced ​​video, and obtaining each frame of decoding buffer data from the buffer of the decoder. The spliced ​​video is preferably a video file obtained by splicing video contents recorded by multiple cameras in the vehicle system. After obtaining the spliced ​​video, the spliced ​​video is read through the FUSE (Filesystem in Userspace) file system, and the spliced ​​video stream is extracted through ExtractorService and input into the decoder for decoding.

[0025] The FUSE file system is a framework that allows the creation and operation of a file system in user space. FUSE includes a kernel module and a user space library. The kernel module processes file system operation requests, such as opening, reading, and writing files, and forwards these requests to the FUSE program in the user space for processing. Through the FUSE, the user space program can export the file system to the Linux kernel. The ExtractorService is a file resource extraction component.

[0026] Step S200 includes: performing screen segmentation on the decoded buffer data of each frame according to a preset display area, obtaining multiple regional video data and storing them in the corresponding regional buffer. The preset display area can be a plurality of different display areas pre-set by the coordinate system of OpenGLES (OpenGL for Embedded Systems). The coordinate system of OpenGLES is a two-dimensional coordinate system, consisting of an x-axis and a y-axis, and the coordinate range of the coordinate system of OpenGLES is a range consisting of the x-axis and the y-axis from negative one to positive one, which respectively corresponds to the four corners of the display screen. For example, according to the coordinate system of OpenGLES, the video screen of each frame is divided into four display areas of the same size, namely, the upper left corner display area x (-1, 0), y (0, 1), the upper right corner display area x (0, 1), y (0, 1), the lower left corner display area x (-1, 0), y (-1, 0) and the lower right corner display area x (0, 1), y (0, -1).

[0027] Step S300 includes: playing at least one of the stitched video and the plurality of regional video data. When the user needs to play back the recorded video, the user can control the media player component (MediaPlayer) to play the stitched video, and when the user needs to zoom in on the content of a certain display area, the user can control the surface view component (SurfaceView) to display the content of the display area in full screen.

[0028] In this embodiment, the decoding buffer data of each frame is obtained by using a surface texture (SurfaceTexture) instance created by GLES.

[0029] Among them, the surface texture instance is mainly the texture interface corresponding to GL_TEXTURE_EXTERNAL_OES, which is a texture type extended in OpenGLES and is used to process external texture sources in OpenGLES, especially for processing external image sources, such as the output of a video decoder or image data captured by a camera. In real-time video processing and augmented reality applications, the GL_TEXTURE_EXTERNAL_OES can directly render images captured by a camera onto an OpenGLES texture to achieve real-time image processing effects.

[0030] The surface texture is a class provided by Android, which can be mainly used to provide an interface for outputting to a GLES texture. The surface texture can also provide an image buffer during graphics rendering and video playback. It can allow applications to directly obtain image data from an image source and pass these data as a texture to OpenGLES for rendering. The surface texture mainly acts as a bridge between the image source and the OpenGLES texture. The camera preview or video stream can be directly passed to the OpenGLES texture through the surface texture.

[0031] The GLES, also known as OpenGLES, is a framework for embedded systems. It is a simplified version of OpenGL, optimized for embedded devices, and provides efficient 2D and 3D graphics rendering capabilities. The graphics processing capabilities can be optimized through the OpenGLES, and it can run efficiently on performance-constrained devices. The OpenGLES can provide high-quality graphics image rendering for graphics images and optimize display content details.

[0032] In this embodiment, the preset display area is pre-set by the GLES. The GLES is used to limit the texture area range of the video frame in the display screen, thereby obtaining multiple different or identical display areas. The display area can mainly be a plurality of display ranges in which a display screen is equally divided. For example, the display screen can be set to a display area with equal left and right areas by the GLES.

[0033] In this embodiment, the regional video data in the regional buffer can be transferred to the corresponding surface interface component (EGLSurface) through the swap buffers method in GLES, and the surface view component obtains the corresponding regional video data through the surface interface component. Among them, the swap buffer method is mainly a commonly used function in graphics programming. When rendering using OpenGL or OpenGLES, its main function is to exchange the front and back buffers so that the rendered content can be displayed on the screen. That is, the regional video data in the regional buffer is transferred to the corresponding surface view component. By exchanging and transferring the regional video data through the swap buffer method, double buffering or multi-buffering technology can be used to reduce screen flicker and improve rendering efficiency. Among them, the double buffer mainly includes a foreground buffer and a background buffer; the foreground buffer refers to the image currently displayed on the screen, and the background buffer can mainly be the image being drawn, that is, the regional buffer. During the rendering process, the graphics data is first drawn to the background buffer. When the rendering is completed, the background buffer and the foreground buffer are exchanged by calling the swap buffer method, so that the content of the background buffer is displayed on the screen, and the foreground buffer becomes the background buffer, ready to receive the rendering of the next frame.

[0034] The multi-buffering method can mainly use multiple background buffers to further reduce rendering delays and optimize performance, such as three buffers or four buffers. The core function of the swap buffer method is to ensure that the rendered content is correctly displayed on the screen, and to smoothly switch the displayed image through double buffering technology to reduce visual flicker and incoherence.

[0035] The surface view component is a view component provided by Android, which is used to display drawn content. The surface view component can render content on an independent drawing surface, separated from other view hierarchies, thereby improving rendering performance and smoothness. The surface view component can be mainly used in scenarios that require high-performance drawing or real-time updated content, such as video playback or games. Since the drawing operation of the surface is performed on an independent surface, the surface view component can avoid competition with other UI components, thereby providing higher rendering efficiency.

[0036] During the specific implementation process, the stitched video can be played mainly by controlling the media player component, and the multiple surface view components can be controlled to display at least one of the multiple area video data. When the user needs to view the video, the stitched video can be played by controlling the media player component, and the corresponding video content can be displayed through the multiple surface view components; when the user specifies that a certain display area needs to be enlarged and displayed in full screen, the display content of the display area is displayed in full screen through the surface view component, so that the user can see the details in the video more clearly. For example, when the user is playing back a recorded video, it is found that there is a detail part in the video that needs to be enlarged and displayed in full screen. The user operates the display screen, and the display screen uses the surface view component to enlarge and display the detail in high definition in full screen.

[0037] During the specific implementation process, take the case where a user needs to replay a recorded video and needs to view certain details as an example. When the user issues a video playback command through the vehicle-mounted central control screen, the spliced ​​video file is accessed from the vehicle-mounted system file through the FUSE file system according to the video playback command, and then the spliced ​​video stream is extracted from the spliced ​​video file through the ExtractorService and input into the decoder for decoding. The decoded buffer data of each frame after decoding in the decoder is divided through the display area pre-set by GLES, and the GLES stores the obtained multiple regional video data in the corresponding regional buffer. The corresponding regional video data is obtained through multiple surface view components. When the user finds that there is content of interest in a certain area while viewing the video, the display content of the area is displayed in full screen through the surface view component according to the instruction issued by the user.

[0038] Embodiment 2: Please refer to Figure 2 ,This application also proposes a system for a video screen separation display method, which mainly includes: a video decoding module, a segmentation module and a playback module; The video decoding module is used to decode the spliced ​​video and store it in the buffer of the decoder. The video decoding module may also include a FUSE file system, a Source file system resource reading interface, a Decode decoding component, and an ExtractorService file resource extraction component. The multi-screen spliced ​​video file stored in the vehicle system is accessed through the FUSE file system. After accessing the multi-screen spliced ​​video file, the ExtractorService file resource extraction component obtains the multi-screen spliced ​​video file through the Source file system resource reading interface, and parses the multi-screen spliced ​​video file, extracts the spliced ​​video therefrom and transmits it to the Decode decoding component, and the Decode decoding component decodes the spliced ​​video and stores it in the buffer of the decoder.

[0039] Among them, the Source file system read resource interface is an interface for accessing and managing resources in the file system. It allows programs to read, write and manage files and data from the file system. The ExtractorService file resource extraction component can read data in the file system through the Source file system read resource interface. It can mainly include reading bytes, characters or data streams from files. The Source file system read resource interface can provide the system with efficient file reading, writing and management functions. It enables applications to reliably access and operate data in the file system.

[0040] The ExtractorService file resource extraction component is a tool for extracting and parsing information from files or data streams. The ExtractorService file resource extraction component can be used to extract the spliced ​​video from the multi-screen spliced ​​video file. The ExtractorService file resource extraction component can be used to parse different types of file formats, such as compressed files, database files, media files, etc., to extract useful information or data therein. Specific data fields or content can be extracted from files. For example, event records can be extracted from log files, metadata can be extracted from image files, etc.

[0041] The Decode component can be used to convert the encoded data back to its original format. For example, the spliced ​​video file is decoded and converted into the original video file in mp4 format. The Decode component can efficiently process large-scale data to ensure that the decoding process is fast and reliable. Those skilled in the art can apply the Decode component to image processing, video processing, and audio processing according to actual scenarios. For example, a compressed image file is decoded into an image object for display or editing, and an audio file is decoded into a playable audio stream. The Decode component also includes a decoding buffer to store the data obtained after decoding in the decoding buffer.

[0042] The segmentation module is used to obtain each frame of decoded buffer data from the decoder buffer, and perform screen segmentation on each frame of decoded buffer data according to a preset display area, obtain multiple regional video data and store them in corresponding regional buffers. The segmentation module includes a surface interface component and a GLES component. The GLES component is used to perform screen segmentation on each frame of decoded buffer data according to the preset display area, and obtain the multiple regional video data. The surface interface component is used to transmit the multiple regional video data to the corresponding surface view component.

[0043] The GLES component can mainly draw textures on the set multiple FrameArea display areas through the operation of drawing textures by OpenGLES. At the same time, the GLES component creates a texture object, obtains each frame of decoded buffer data in the surface interface through the surface texture, and divides each frame of decoded buffer data through the multiple FrameArea display areas. After the division is completed, the obtained multiple regional video data are stored in the corresponding regional buffer. Among them, the FrameArea display area is mainly the screen area of ​​the video frame in OpenGLES, such as the screen area of ​​x(0,200),y(100,300). The surface texture enables the application to pass image data from a camera, video stream or other source to the texture object, and render these textures through OpenGLES.

[0044] The playback module is used to play at least one of the stitched video and the multiple regional video data. The playback module may mainly include a media player component and multiple surface view components. The media player component is used to play the stitched video. The surface view component is used to display the regional video data. The media player component and the surface view component run simultaneously. If the playback module only runs the media player component but does not run the surface view component, only the audio of the stitched video will be played on the vehicle display screen but the picture will not be displayed. The video content can only be displayed when the surface view component is running. The GLES component passes the regional video data in the regional buffer to the surface interface component through the exchange buffer method, and the surface view component obtains the corresponding regional video data from the surface interface component.

[0045] The surface interface component is the native platform interface of OpenGLES, and the surface view component can obtain the corresponding regional video data through the surface interface component. The surface view component is a view component in the Android system, which is used to provide a surface view component that can be directly drawn in the application. It is used in scenes that require frequent updates or processing of complex graphics, such as video playback and real-time image processing. The independent drawing surface provided by the surface component can be used to perform drawing operations in the background thread without blocking the main thread.

[0046] In the specific implementation process, when the user uses the vehicle display screen to play back the recorded video, the playback module mainly accesses the multi-screen spliced ​​video file stored in the vehicle system through the FUSE file system, and the ExtractorService file resource extraction component obtains the accessed spliced ​​video file through the Source file system reading resource interface, and parses the multi-screen spliced ​​video file, extracts the spliced ​​video from it and transmits it to the Decode decoding component, and the Decode decoding component decodes the spliced ​​video and stores it in the decoder buffer. The segmentation module mainly obtains the decoding buffer data of each frame through the surface interface component, and through the GLES component, the decoding buffer data of each frame is screen-segmented according to the preset display area to obtain the video data of the multiple regions. The playback module mainly plays the spliced ​​video through the media player component according to the received instructions. The regional video data is displayed through the surface view component. When the user needs to zoom in to view the video during viewing, the surface view component displays the details that the user needs to view in full screen.

[0047] Embodiment three: The present application also proposes an electronic device, which includes a memory storing executable program codes and a processor coupled to the memory, wherein the processor calls the executable program codes stored in the memory to execute the video screen separation display method.

[0048] The electronic device of this embodiment includes: at least one processor, a memory, and an executable program code stored in the memory and executable on the at least one processor, and the processor implements the steps of any of the above method embodiments when executing the executable program code.

[0049] The electronic device may be a vehicle-side electronic device, including but not limited to an in-vehicle infotainment system, a vehicle-side host, etc. It is understandable that the specific structure and function of the vehicle-side mobile device on different vehicle models and vehicle manufacturers are different. The electronic device may include but is not limited to a processor and a memory. The processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The memory may be an internal storage unit of the electronic device in some embodiments, such as a hard disk or memory of the electronic device. The memory may also be an external storage device of the electronic device in other embodiments, such as a plug-in hard disk, a smart memory card (SMC), a secure digital (SD) card, a flash card, etc., equipped on the electronic device. Furthermore, the memory may include both an internal storage unit of the electronic device and an external storage device. The memory is used to store an operating system, an application, a boot loader, data, and other programs, such as program codes of the executable program. The memory may also be used to temporarily store data that has been output or is to be output.

[0050] In summary, the present application obtains multiple regional video data by dividing each frame of decoded buffer data obtained after decoding the obtained spliced ​​video through a preset display area, and plays the spliced ​​video through instructions and displays at least one of the multiple regional video data. When playing back a recorded video, any detail part of the video can be displayed in full screen. Therefore, the present application can better display the local content of the video in full screen, so that the user can better see any detail part of the video, which significantly improves the user experience.

[0051] In several embodiments provided in the present application, it is understood that each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved.

[0052] If the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in an electronic device. Based on this understanding, the technical solution of the present application can essentially or in other words, the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product. The computer software product is stored in an electronic device and includes several instructions for enabling an electronic device to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned electronic devices include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0053] The specific embodiments described above further describe the purpose, technical solutions and beneficial effects of the present application in detail. It should be understood that the above description is only a specific embodiment of the present application and is not intended to limit the scope of protection of the present application. It is particularly pointed out that for those skilled in the art, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of protection of the present application.

Claims

1. A video screen separation display method, characterized in that: include: S100, decoding the spliced ​​video, and obtaining each frame of decoding buffer data from the buffer of the decoder; S200, dividing each frame of the decoded buffer data into segments according to a preset display area to obtain multiple area videos The data is stored in the corresponding regional buffer; S300, playing at least one of the spliced ​​video and the plurality of regional video data.

2. The video screen separation display method according to claim 1, characterized in that: In the step S100, the decoding buffer data of each frame is obtained by obtaining the surface texture instance created by GLES.

3. The video screen separation display method according to claim 1, characterized in that: In the step S200, the preset display area is pre-set through the GLES.

4. The video screen separation display method according to claim 1, characterized in that: The step S200 further includes: The regional video data in the regional buffer is transferred to the corresponding surface interface component through the exchange buffer method in GLES, and the surface view component obtains the corresponding regional video data through the surface interface component.

5. The video screen separation display method according to claim 4, characterized in that: The step S300 includes: The stitched video is played by controlling a media player component, and the plurality of surface view components are controlled to display at least one of the plurality of regional video data.

6. The video screen separation display method according to claim 5, characterized in that: The step S300 further includes: When the user selects any video display area, the surface view component enlarges the regional video data corresponding to the video display area and displays it in full screen.

7. A system based on the video screen separation display method according to any one of claims 1 to 6, characterized in that: The system includes: a video decoding module, a segmentation module and a playback module; The video decoding module is used to decode the spliced ​​video and store it in the buffer of the decoder; The segmentation module is used to obtain each frame of decoding buffer data from the buffer of the decoder, and perform screen segmentation on each frame of decoding buffer data according to a preset display area, obtain multiple area video data and store them in corresponding area buffers; The playback module is used to play at least one of the spliced ​​video and the multiple regional video data.

8. The system of the video screen separation display method according to claim 7, characterized in that: The segmentation module includes a surface interface component and a GLES component; The GLES component is used to perform screen segmentation on the decoded buffer data of each frame according to the preset display area to obtain the video data of the multiple regions; The surface interface component is used to transmit the multiple area video data to the corresponding surface view component.

9. The system of the video screen separation display method according to claim 7, characterized in that: The playback module includes a media player component and a plurality of surface view components; The media player component is used to play the spliced ​​video; The surface view component is used to display the regional video data.

10. An electronic device, characterized in that: include: A memory storing executable program code and a processor coupled to the memory, wherein the processor calls the executable program code stored in the memory to execute the video screen separation display method according to any one of claims 1 to 6.

Images