A video data sharing method and system

Abstract

This application relates to the field of automotive technology and provides a video data sharing method and system. The video data sharing method includes: a main process responding to a connection request, acquiring a target video channel, and at least acquiring a video access handle corresponding to the target video channel, and sending the video access handle to a slave process; the slave process acquiring video frame description information corresponding to video frames under the target video channel from shared memory corresponding to the target video channel; the slave process obtaining at least a virtual access address corresponding to a video frame under the target video channel based on the video access handle and the video frame description information, and accessing the video frame in the cache block corresponding to the target video channel through the virtual access address. This method solves the technical problem of multiple processes being unable to access video data simultaneously and reduces the coupling between functional modules of automotive products.

Description

Technical Field

[0001] This application belongs to the field of vehicle technology, and in particular relates to a video data sharing method and system. Background Technology

[0002] As hardware becomes increasingly complex, traditional automotive devices fall short in code reuse, resulting in bloated hardware drivers. Therefore, the V4L2 framework was proposed. V4L2 provides basic software components and simplifies driver development through shared functions, enhancing code reusability. Based on the V4L2 framework, applications in automotive devices can more transparently utilize hardware to drive audio and video processing. Currently, the V4L2 framework is being adopted by an increasing number of automotive devices.

[0003] Within the V4L2 framework, only one instance of video capture software can run at a time, restricting other processes from accessing video data simultaneously. However, in recent years, automotive devices have been evolving towards diversification and intelligence. Automotive device companies often collaborate technically, integrating their respective strengths into their products to enhance competitiveness and improve user experience. In these collaborations, different companies' technical teams contribute their functional modules, often resulting in multiple processes running within the automotive device, thus complicating video data access. Summary of the Invention

[0004] This application provides a video data sharing method and system that can solve the above-mentioned technical problems.

[0005] In a first aspect, embodiments of this application provide a video data sharing method, comprising: obtaining a target video channel from a process; generating and sending a connection request to a main process based on the target video channel; the main process responding to the connection request by obtaining the target video channel and at least obtaining a video access handle corresponding to the target video channel, and sending the video access handle corresponding to the target video channel to a slave process; the slave process receiving the video access handle corresponding to the target video channel and obtaining video frame description information corresponding to video frames under the target video channel from the shared memory corresponding to the target video channel; the slave process obtaining at least a virtual access address corresponding to the video frames under the target video channel based on the video access handle corresponding to the target video channel and the video frame description information corresponding to the video frames under the target video channel, and accessing video frames in the cache block corresponding to the target video channel through the virtual access address corresponding to the video frames under the target video channel.

[0006] Furthermore, before responding to a connection request, the main process includes: receiving video data from each video channel and writing the video data from each video channel into the corresponding buffer block of each video channel; wherein the video data from each video channel consists of video frames from that video channel; and wherein the video channel includes at least the target video channel.

[0007] Furthermore, after the main process responds to the connection request, it includes: responding to the connection success instruction, allocating shared memory corresponding to each video channel, and writing the video frame description information corresponding to the video frames under each video channel into the corresponding shared memory.

[0008] Furthermore, before responding to a connection request, the main process includes: receiving video data from each video channel; wherein the video data from each video channel consists of video frames from that video channel; the main process adds timestamps to the video frames from each video channel, and constructs video frame packets based on the video frames from each video channel and their corresponding timestamps; wherein a video frame packet includes one video frame from each video channel; for each video frame packet constructed, the main process obtains the stitched video frame based on the video frame packet and a preset surround view stitching order, and writes the stitched video frame into the cache block corresponding to the target video channel; the cache block corresponding to the target video channel is a cache block retrieved from the cache pool.

[0009] Furthermore, in response to the connection request, the main process includes: in response to the connection success instruction, allocating shared memory corresponding to the target video channel, and writing the video frame description information corresponding to the spliced ​​video frames into the shared memory corresponding to the target video channel.

[0010] Furthermore, based on the video frames under each video channel and the timestamps corresponding to the video frames under each video channel, a video frame package is constructed, including: the main process determines the reference video frame and its corresponding timestamp based on the timestamps corresponding to the video frames under each video channel; the main process obtains the time interval between the timestamps corresponding to the video frames under each video channel and the timestamps corresponding to the reference video frame; if the time interval is not less than a preset time interval threshold, the main process discards the corresponding video frame, waits for the next video frame in the video channel where the corresponding video frame is located, and repeats the above steps until all time intervals are less than the preset time interval threshold, thus obtaining the video frame package.

[0011] Furthermore, the method also includes: if the process accesses a video frame in the buffer block corresponding to the target video channel, the process triggers an increment of the reference count value of the video frame description information corresponding to the video frame; if the process stops accessing the video frame in the buffer block corresponding to the target video channel, the process triggers a decrement of the reference count value of the video frame description information corresponding to the video frame.

[0012] Furthermore, the video frame description information corresponding to the video frame is stored in the shared memory corresponding to the target video channel in the form of a queue. The method also includes: if the number of video frame description information stored in the shared memory corresponding to the target video channel exceeds a preset number, the main process releases the video frame description information in the shared memory corresponding to the target video channel according to a preset release priority rule; wherein, the preset release priority rule from high to low is the video frame description information with a reference count of zero at the head of the queue, the video frame description information with a reference count of zero at the tail of the queue, and the video frame description information waiting to enter the queue.

[0013] Furthermore, the method also includes: in response to a release request sent by a slave process, the main process obtains a reference count value for the target video frame and the video frame description information corresponding to the target video frame; if the reference count value of the video frame description information corresponding to the target video frame is zero, the main process releases the cache corresponding to the target video frame.

[0014] Secondly, embodiments of this application provide a video data sharing device, including:

[0015] The acquisition module is used to acquire the target video channel, generate and send a connection request to the main process based on the target video channel;

[0016] The response module is used to respond to the connection request, obtain the target video channel, and at least obtain the video access handle corresponding to the target video channel, and send the video access handle corresponding to the target video channel to the slave process.

[0017] The shared module is used to receive the video access handle corresponding to the target video channel and obtain the video frame description information corresponding to the video frame under the target video channel from the shared memory corresponding to the target video channel.

[0018] The access module is used to obtain, from the process, at least the virtual access address of the video frame corresponding to the target video channel based on the video access handle corresponding to the target video channel and the video frame description information corresponding to the video frame under the target video channel, and then access the video frame in the buffer block corresponding to the target video channel through the virtual access address of the video frame under the target video channel.

[0019] Furthermore, the device also includes: a first writing module, configured to receive video data from each video channel and write the video data from each video channel into the corresponding buffer block of each video channel; wherein the video data from each video channel consists of video frames from the video channel; wherein the video channel includes at least a target video channel.

[0020] Furthermore, the device also includes: a first connection module, used to, in response to a connection success command, allocate shared memory corresponding to each video channel, and write the video frame description information corresponding to the video frames under each video channel into the corresponding shared memory respectively.

[0021] Furthermore, the device also includes: a receiving module for receiving video data from each video channel; wherein the video data from each video channel consists of video frames from that video channel;

[0022] The construction module is used to add timestamps to the video frames under each video channel and construct video frame packages based on the video frames under each video channel and the timestamps corresponding to the video frames under each video channel; wherein, a video frame package includes one video frame from each video channel.

[0023] The second writing module is used to construct a video frame package, obtain the spliced ​​video frame according to the video frame package and the preset panoramic view splicing order, and write the spliced ​​video frame to the cache block corresponding to the target video channel; the cache block corresponding to the target video channel is a cache block retrieved from the cache pool.

[0024] Furthermore, the device also includes a second connection module, which, in response to a connection success command, allocates shared memory corresponding to the target video channel and writes the video frame description information corresponding to the spliced ​​video frames into the shared memory corresponding to the target video channel.

[0025] Furthermore, the construction module is specifically used to: determine the reference video frame and its corresponding timestamp based on the timestamps of the video frames in each video channel; obtain the time interval between the timestamps of the video frames in each video channel and the timestamps of the reference video frame; if the time interval is not less than a preset time interval threshold, discard the corresponding video frame, wait for the next video frame in the video channel where the corresponding video frame is located, and repeat the above steps until all time intervals are less than the preset time interval threshold, thus obtaining a video frame package.

[0026] Furthermore, the device also includes: a first counting module, configured to trigger an increment of the reference count value of the video frame description information corresponding to the video frame if the process accesses a video frame in the buffer block corresponding to the target video channel; and a second counting module, configured to trigger a decrement of the reference count value of the video frame description information corresponding to the video frame if the process stops accessing the video frame in the buffer block corresponding to the target video channel.

[0027] Furthermore, the device also includes: a first release module, used to release the video frame description information in the shared memory corresponding to the target video channel according to a preset release priority rule when the number of video frame description information stored in the shared memory corresponding to the target video channel exceeds a preset number; wherein, the preset release priority rule from high to low is video frame description information with a reference count of zero at the head of the queue, video frame description information with a reference count of zero at the tail of the queue, and video frame description information waiting to enter the queue.

[0028] Furthermore, the device also includes: a release response module, used to obtain a reference count value of the target video frame and the video frame description information corresponding to the target video frame in response to a release request sent from the process;

[0029] The second release module is used to release the cache corresponding to the target video frame if the reference count value of the video frame description information corresponding to the target video frame is zero.

[0030] Thirdly, embodiments of this application provide a video data sharing system, including: a video acquisition device and a vehicle-mounted device;

[0031] The video capture equipment is used to collect video data from each video channel and send the video data from each video channel to the vehicle-mounted equipment;

[0032] The vehicle-mounted device is equipped with whole-machine software. The functions of the whole-machine software are implemented by a main process and a slave process. The vehicle-mounted device implements the video data sharing method in the first aspect through the main process and the slave process.

[0033] Fourthly, embodiments of this application provide an in-vehicle device, including a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the video data sharing method as described in the first aspect above.

[0034] Fifthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the video data sharing method as described in the first aspect above.

[0035] In this embodiment, each slave process can connect to the master process and access video frames in the cache block based on the video access handle shared by the master process and the video frame description information stored in the shared memory. Specifically, the slave process first generates and sends a connection request to the slave process based on the target video channel. The master process responds to the connection request, obtains the target video channel, and at least obtains the video access handle corresponding to the target video channel. It then sends the video access handle corresponding to the target video channel to the slave process. It can be understood that for any number of slave processes requesting a connection, the master process will send the corresponding video access handle to the slave process. After obtaining the video access handle corresponding to the target video channel, the slave process also needs to obtain the video frame description information corresponding to the video frames under the target video channel from the shared memory corresponding to the target video channel. Then, based on the video access handle corresponding to the target video channel and the video frame description information corresponding to the video frames under the target video channel, it obtains at least the virtual access address corresponding to the video frames under the target video channel. Thus, it can access the video frames in the cache block corresponding to the target video channel through the virtual access address corresponding to the video frames under the target video channel. In the above method, each slave process can access the same video frame through different virtual access addresses, thereby realizing the sharing of video data among multiple slave processes. This solves the technical problem that multiple processes cannot access video data at the same time. Compared with implementing all functions in one process, multiple slave processes also reduce the coupling between functional modules. If a problem occurs in the execution of one slave process, it will not affect the other slave processes to implement their corresponding functions. Thus, while meeting the video sharing requirements, it also enhances the stability of the vehicle product and takes into account the confidentiality of the product technology. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1 This is a schematic flowchart of a video data sharing method provided in the first embodiment of this application;

[0038] Figure 2 This is another schematic flowchart of a video data sharing method provided in the first embodiment of this application;

[0039] Figure 3 This is another schematic flowchart of a video data sharing method provided in the first embodiment of this application;

[0040] Figure 4This is a schematic flowchart of a video data sharing method provided in the second embodiment of this application;

[0041] Figure 5 This is a schematic flowchart of step S203 in a video data sharing method provided in the second embodiment of this application;

[0042] Figure 6 This is a schematic diagram of the video data sharing device provided in the third embodiment of this application;

[0043] Figure 7 This is a schematic diagram of the video data sharing system provided in the third embodiment of this application;

[0044] Figure 8 This is a schematic diagram of the vehicle-mounted device provided in the fourth embodiment of this application. Detailed Implementation

[0045] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0046] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0047] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0048] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0049] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0050] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0051] Please see Figure 1 , Figure 1 This is a schematic flowchart of a video data sharing method provided in the first embodiment of this application. In this embodiment, both the main process and the slave process are processes within the overall software of the vehicle-mounted device. The execution of both the main process and the slave process is implemented by the vehicle-mounted device. Therefore, the actual execution entity of the video data sharing method is the vehicle-mounted device, or the processor, microprocessor, etc., within the vehicle-mounted device. To facilitate understanding of how video data is shared and how the shared video data is accessed, the execution process of the video data sharing method by the vehicle-mounted device in this embodiment will be described from the perspectives of both the main process and the slave process. This will more clearly express the interaction process between the main process and the slave process and facilitate understanding of the principle of video data sharing. The video data sharing method includes:

[0052] S101: Obtain the target video channel from the process, generate and send a connection request to the main process based on the target video channel.

[0053] In this embodiment of the application, the process responsible for basic functions in the overall software of the vehicle-mounted device is referred to as the main process. The basic functions include, but are not limited to, the acquisition of audio and video data and encoding and recording.

[0054] In this embodiment of the application, the process responsible for other functions in the overall software of the vehicle-mounted device is referred to as the slave process. Other functions, such as intelligent recognition, intelligent analysis, and surround view display, can be integrated into the overall software by different companies through technical cooperation.

[0055] In this embodiment, the target video channel can be understood in the following two ways:

[0056] The first type refers to any one, any number of, or all video channels among a set of video channels. Generally, the number of video channels is equal to the number of video capture devices (i.e., cameras) deployed.

[0057] The second type involves a target video channel that differs from the concept of a video channel mentioned above. Simply put, it's a merged video channel. Originally, video frames from different video channels were written separately into their respective buffer blocks, requiring separate access and retrieval. The merging here involves writing a single video frame from several video channels into the same buffer block in a predetermined order. Accessing such a buffer block is called accessing the buffer block corresponding to the target video channel. Therefore, the process generates and sends a connection request to the main process based on the target video channel. Its purpose is to indicate to the main process that it wants to access the aforementioned buffer block and retrieve the video frames assembled in a predetermined order.

[0058] In this embodiment, the subsequent steps will be described based on a first understanding of the target video channel; in the second embodiment, the steps will be described based on a second understanding of the target video channel.

[0059] Typically, the correspondence between cameras and video channels is pre-defined. For example, channel 0 refers to the front camera, and channel 1 refers to the rear camera. In this embodiment, the process first determines which camera (or cameras) it wants to acquire video data from, and then obtains the target video channel based on the correspondence between the camera and the video channel.

[0060] The slave process generates and sends a connection request to the master process based on the target video channel.

[0061] S102: The main process responds to the connection request, obtains the target video channel, and at least obtains the video access handle corresponding to the target video channel, and sends the video access handle corresponding to the target video channel to the slave process.

[0062] The main process responds to the connection request by parsing the request and obtaining the target video channel.

[0063] The main process obtains at least the corresponding video access handle based on the target video channel. This video access handle, also known as a video handle in the V4L2 framework, can be understood as a file descriptor. Through the video access handle, video-related functions can be accessed. The main process can also set the number of buffer blocks corresponding to each video channel through the video access handle corresponding to each video channel.

[0064] Specifically, after the main process obtains the video access handle corresponding to the target video channel, it sends it to the slave process via Linux sharing.

[0065] Before the S102 master process responds to a connection request, it includes:

[0066] The main process receives video data from each video channel and writes the video data from each video channel into the corresponding buffer block for each video channel; the video data from each video channel consists of video frames from that video channel.

[0067] The main process receives the video data from each video channel and writes the video data from each video channel into the corresponding buffer block for each video channel. In this embodiment, the video channels include at least the target video channel.

[0068] The number of cache blocks corresponding to each video channel is determined by the functions supported by the whole machine software. The number of cache blocks will vary depending on the functions supported. Under the V4L2 framework, it is possible to set the number of cache blocks for each video channel. The kernel will maintain a queue to store the cache blocks.

[0069] Following the connection request, the S102 master process responds to the following:

[0070] In response to the connection success command, the main process allocates shared memory for each video channel and writes the video frame description information for each video frame into the corresponding shared memory.

[0071] After a successful connection, the main process will allocate shared memory for each video channel and write the video frame description information for each video frame into the corresponding shared memory for the slave process to read.

[0072] In one optional implementation, the video frame description information corresponding to the video frame includes at least memory address offset information, and may also include the following: data start address, frame width, frame height, frame length, DMA handle, image format, and acquisition timestamp, etc.

[0073] The following is an optional data structure for video frame description information:

[0074] struct DataElem{

[0075] uint8_t*data; / * Starting address of YUV data * /

[0076] int32_t width; / * Frame width * /

[0077] int32_t height; / * Frame height * /

[0078] int32_t length; / * Frame length * /

[0079] int32_t fd; / *v4l2 DMA handle* /

[0080] int32_t videoLoss; / * Whether to check if video loss frames * /

[0081] int32_t offset; / * Memory offset * /

[0082] uint64_t pts; / ** Timestamp, unit: microseconds** /

[0083] int32_t fmt; / * Image format * /

[0084] }

[0085] A video access handle (i.e., a video handle) is available for access software to operate.

[0086] DMA handles are available for hardware operations, such as copying video data. DMA handles can be obtained based on video access handles.

[0087] In an optional implementation, in S102, in addition to sending the video access handle corresponding to the target video channel to the slave process, the shared memory handle, the number of video channels, and the number of cache blocks corresponding to each video channel are also obtained, and the above data are sent to the slave process together.

[0088] S103: Receive the video access handle corresponding to the target video channel from the process, and obtain the video frame description information corresponding to the video frame under the target video channel from the shared memory corresponding to the target video channel.

[0089] Specifically, the process receives the video access handle corresponding to the target video channel and reads the video frame description information corresponding to the video frame under the target video channel from the shared memory corresponding to the target video channel through the shared memory handle.

[0090] S104: The process obtains at least the virtual access address of the video frame in the target video channel based on the video access handle corresponding to the target video channel and the video frame description information corresponding to the video frame in the target video channel. Then, it accesses the video frame in the buffer block corresponding to the target video channel through the virtual access address of the video frame in the target video channel.

[0091] As mentioned above, the video frame description information corresponding to a video frame includes at least memory address offset information. In S104, the process can obtain the virtual access address corresponding to the video frame under the target video channel based on the video access handle corresponding to the target video channel, the memory address offset information, and the preset address mapping method.

[0092] It can be determined that the virtual access addresses obtained by different processes when pre-accessing the video frames in the buffer block corresponding to the target video channel are different.

[0093] The default address mapping method can refer to the mmap method in the Linux API.

[0094] In this embodiment, the processes access video frames in the cache block corresponding to the target video channel through different virtual access addresses, without affecting each other's video frame access process.

[0095] For further management of process access to video frames, please refer to [link / reference]. Figure 2 , Figure 2 This is another schematic flowchart of a video data sharing method provided in the first embodiment of this application, which further includes:

[0096] S105: If the process accesses a video frame in the buffer block corresponding to the target video channel, the process triggers an increment of the reference count value of the video frame description information corresponding to the video frame.

[0097] S106: If the process stops accessing the video frame in the buffer block corresponding to the target video channel, the process triggers a decrement of the reference count value of the video frame description information corresponding to the video frame.

[0098] In this embodiment, reference counting is maintained. If the process accesses a video frame in the cache block corresponding to the target video channel, the process triggers an increment of the reference count value of the video frame description information corresponding to the video frame. If the process stops accessing the video frame in the cache block corresponding to the target video channel, the process triggers a decrement of the reference count value of the video frame description information corresponding to the video frame.

[0099] In other words, the reference count value provides a clear and convenient understanding of the number of slave processes currently accessing the video frame.

[0100] In an optional implementation, the video frame description information corresponding to the video frame is stored in a queue in the shared memory corresponding to the target video channel. The method further includes:

[0101] S107: If the number of video frame descriptions stored in the shared memory corresponding to the target video channel exceeds a preset number, the main process releases the video frame descriptions in the shared memory corresponding to the target video channel according to the preset release priority rules. The preset release priority rules, from high to low, are: video frame descriptions with a reference count of zero at the head of the queue, video frame descriptions with a reference count of zero at the tail of the queue, and video frame descriptions waiting to enter the queue.

[0102] In this embodiment, when the shared memory corresponding to the target video channel is full, the main process releases the video frame description information in the shared memory corresponding to the target video channel according to the preset release priority rules.

[0103] Specifically, the preset release priority rule, from high to low, is: video frame description information with a reference count of zero at the head of the queue, video frame description information with a reference count of zero at the tail of the queue, and video frame description information waiting to enter the queue. In this embodiment, video frame description information with higher priority will be released first, and correspondingly, the cache corresponding to the video frame can be rewritten.

[0104] Please see Figure 3 , Figure 3 This is another schematic flowchart of a video data sharing method provided in the first embodiment of this application, the method further comprising:

[0105] S108: In response to the release request sent by the slave process, the master process obtains the reference count value of the target video frame and the video frame description information corresponding to the target video frame.

[0106] S109: If the reference count of the video frame description information corresponding to the target video frame is zero, the main process releases the cache corresponding to the target video frame.

[0107] Video frames may have mutually exclusive usage requirements. For example, during a surround stitching process, it may be necessary to add a bounding box to the video frame to highlight certain objects or people. This will modify the video frame. When other processes access this video frame, the modification may cause abnormal algorithm execution. Therefore, if a slave process wants to use video frames mutually exclusively, it can send a release request to the master process, specifying the video frame that it wants to use mutually exclusively, i.e., specifying the target video frame. In response to the release request sent by the slave process, the master process obtains the reference count value of the target video frame and its corresponding video frame description information. If the reference count value of the target video frame's corresponding video frame description information is zero, the master process releases the cache corresponding to the target video frame.

[0108] Obviously, if the reference count of the video frame description information corresponding to the target video frame is not zero, the current slave process cannot modify the target video frame.

[0109] In this embodiment, each slave process can connect to the master process and access video frames in the cache block based on the video access handle shared by the master process and the video frame description information stored in the shared memory. Specifically, the slave process first generates and sends a connection request to the slave process based on the target video channel. The master process responds to the connection request, obtains the target video channel, and at least obtains the video access handle corresponding to the target video channel. It then sends the video access handle corresponding to the target video channel to the slave process. It can be understood that for any number of slave processes requesting a connection, the master process will send the corresponding video access handle to the slave process. After obtaining the video access handle corresponding to the target video channel, the slave process also needs to obtain the video frame description information corresponding to the video frames under the target video channel from the shared memory corresponding to the target video channel. Then, based on the video access handle corresponding to the target video channel and the video frame description information corresponding to the video frames under the target video channel, it obtains at least the virtual access address corresponding to the video frames under the target video channel. Thus, it can access the video frames in the cache block corresponding to the target video channel through the virtual access address corresponding to the video frames under the target video channel. In the above method, each slave process can access the same video frame through different virtual access addresses, thereby realizing the sharing of video data among multiple slave processes. This solves the technical problem that multiple processes cannot access video data at the same time. Compared with implementing all functions in one process, multiple slave processes also reduce the coupling between functional modules. If a problem occurs in the execution of one slave process, it will not affect the other slave processes to implement their corresponding functions. Thus, while meeting the video sharing requirements, it also enhances the stability of the vehicle product and takes into account the confidentiality of the product technology.

[0110] Please see Figure 4 , Figure 4 This is a schematic flowchart of a video data sharing method provided in the second embodiment of this application, the method including:

[0111] S201: Obtain the target video channel from the process, generate and send a connection request to the main process based on the target video channel.

[0112] S202: The main process receives video data from each video channel; the video data from each video channel consists of video frames from that video channel.

[0113] S203: The main process adds timestamps to the video frames under each video channel, and constructs a video frame package based on the video frames under each video channel and the timestamps corresponding to the video frames under each video channel; wherein, a video frame package includes one video frame under each video channel.

[0114] S204: For each video frame package constructed, the main process obtains the spliced ​​video frame according to the video frame package and the preset surround view splicing order, and writes the spliced ​​video frame into the cache block corresponding to the target video channel; the cache block corresponding to the target video channel is the cache block retrieved from the cache pool.

[0115] S205: The master process responds to the connection request, obtains the target video channel, and at least obtains the video access handle corresponding to the target video channel, and sends the video access handle corresponding to the target video channel to the slave process.

[0116] S206: Receive the video access handle corresponding to the target video channel from the process, and obtain the video frame description information corresponding to the video frame under the target video channel from the shared memory corresponding to the target video channel.

[0117] S207: The process obtains at least the virtual access address of the video frame in the target video channel based on the video access handle corresponding to the target video channel and the video frame description information corresponding to the video frame in the target video channel. Then, it accesses the video frame in the buffer block corresponding to the target video channel through the virtual access address of the video frame in the target video channel.

[0118] Steps S201 and S205 to S207 can be referred to the description of S101 to S104 in the first embodiment. The description will mainly focus on S202 to S204, and any differences will be pointed out.

[0119] Regarding step S201, the target video channel in this embodiment can be simply understood as a fused video channel. As mentioned earlier, the video frames of each video channel were originally written separately into their respective buffer blocks. Video frames from different video channels needed to be accessed and retrieved separately. The fusion in this embodiment means writing a video frame from several video channels into the same buffer block in a predetermined order. Accessing such a buffer block is called accessing the buffer block corresponding to the target video channel. In this embodiment, the slave process generates and sends a connection request to the master process based on the target video channel. Its purpose is to indicate to the master process that it wants to access the aforementioned buffer block and retrieve the video frames spliced ​​in a predetermined order.

[0120] Regarding steps S202 to S203, the main process receives video data from each video channel and adds timestamps to the video frames of each video channel.

[0121] The timestamp can be the capture timestamp of the video frame.

[0122] Then, the main process constructs video frame packages based on the video frames under each video channel and the timestamps corresponding to the video frames under each video channel.

[0123] A video frame packet contains one video frame from each video channel. For example, if there are four video channels, then a video frame packet contains four video frames.

[0124] In one optional implementation, please refer to Figure 5 , Figure 5 This is a schematic flowchart of S203 in a video data sharing method provided in the second embodiment of this application. S203 involves constructing a video frame package based on the video frames under each video channel and the timestamps corresponding to the video frames under each video channel, including:

[0125] S2031: The main process determines the reference video frame and the timestamp corresponding to the reference video frame based on the timestamps of the video frames under each video channel.

[0126] S2032: The main process obtains the time interval between the timestamp corresponding to the video frame under each video channel and the timestamp corresponding to the reference video frame.

[0127] S2033: If the time interval is not less than the preset time interval threshold, the main process discards the corresponding video frame, waits for the next video frame in the video channel where the corresponding video frame is located, and repeats the above steps until all time intervals are less than the preset time interval threshold, and obtains the video frame package.

[0128] Regarding step S2031, the main process obtains the median of the timestamps based on the timestamps corresponding to the video frames under each video channel, and determines the video frame corresponding to the timestamp closest to the median as the reference video frame, thereby determining the timestamp corresponding to the reference video frame.

[0129] Regarding step S2032, the main process calculates the time interval between the timestamp corresponding to the video frame under each video channel and the timestamp corresponding to the reference video frame.

[0130] Regarding step S2033, if the time interval is not less than a preset time interval threshold, the main process discards the corresponding video frame. This preset time interval threshold is T. Optionally, if the frame rate is 25 frames / second and the frame interval is 1 second / 25 = 40 milliseconds, then T should generally be less than half the frame interval, i.e., T should generally be less than 20 milliseconds.

[0131] After discarding the corresponding video frame, wait for the next video frame in the video channel where the corresponding video frame is located. For example, if the video frame under video channel 0 is discarded, wait for the next video frame under video channel 0. Then, repeat the steps S2031 to S2033 above until all time intervals are less than the preset time interval threshold, and obtain the video frame package.

[0132] Regarding step S204, for each video frame package constructed, the main process will retrieve a cache block from the cache pool. In this embodiment, one cache block is sufficient to hold one video frame from each video channel.

[0133] The main process obtains the spliced ​​video frames based on the video frame wrapping and the preset panoramic view splicing order, and writes the spliced ​​video frames into the buffer block, which can also be understood as copying them to the buffer block corresponding to the target video channel.

[0134] Regarding step S205, the main process responds to the connection request, obtains the target video channel, and at least obtains the video access handle corresponding to the target video channel, and sends the video access handle corresponding to the target video channel to the slave process.

[0135] After S205, in response to the connection success command, the main process allocates shared memory corresponding to the target video channel and writes the video frame description information corresponding to the spliced ​​video frames into the shared memory corresponding to the target video channel.

[0136] Here, the process of allocating shared memory to achieve video data sharing is the same as in the first embodiment.

[0137] The difference is that the first embodiment shares video data from one or several video channels.

[0138] In the second embodiment, the shared video frames are the spliced ​​video frames, which can also be understood as the video frames copied to the buffer block corresponding to the target video channel according to the preset panoramic view splicing order.

[0139] If surround view stitching is implemented in the first embodiment, the process needs to access the video data under several video channels separately, then copy it to a cache according to the preset surround view stitching order, and then render it to the user.

[0140] In the second embodiment, the process can directly access the cache block corresponding to the target video channel to obtain the spliced ​​video frames, which can then be rendered and presented to the user.

[0141] It should also be noted that the steps related to S105 to S109 can all be implemented based on S201 to S207 in this embodiment. The implementation methods and purposes are the same, so they will not be repeated here.

[0142] In this embodiment, a video frame package is constructed before video data sharing. Based on the video frame package and a preset panoramic view stitching order, a stitched video frame is obtained and written to the cache block corresponding to the target video channel. This allows the process to retrieve the stitched video frame and render it to the screen when accessing the cache block via a virtual access address. Since it's impossible to guarantee that all cameras output video frames simultaneously, and there may be a lack of synchronization between video frames during video data sharing, constructing a video frame package before video data sharing in panoramic stitching applications effectively avoids discontinuous images during panoramic stitching. Furthermore, copying the video frames within the video frame package to the same cache block according to the preset panoramic view stitching order allows for quick image rendering by simply accessing the cache block before presentation, avoiding time jitter caused by the process copying itself and improving the overall presentation effect of panoramic stitching.

[0143] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0144] Please see Figure 6 , Figure 6 This is a schematic diagram of a video data sharing device provided in the third embodiment of this application. The included modules are used for execution... Figure 1 The steps in the corresponding embodiments. Please refer to the details. Figure 1 The relevant descriptions in the corresponding embodiments are shown below. For ease of explanation, only the parts relevant to this embodiment are shown. See also... Figure 6 The video data sharing device 6 includes:

[0145] The acquisition module 61 is used to acquire the target video channel, generate and send a connection request to the main process based on the target video channel;

[0146] The response module 62 is used to respond to the connection request, obtain the target video channel, and at least obtain the video access handle corresponding to the target video channel, and send the video access handle corresponding to the target video channel to the slave process.

[0147] The shared module 63 is used to receive the video access handle corresponding to the target video channel and obtain the video frame description information corresponding to the video frame under the target video channel from the shared memory corresponding to the target video channel.

[0148] Access module 64 is used to obtain at least the virtual access address of the video frame in the target video channel based on the video access handle corresponding to the target video channel and the video frame description information corresponding to the video frame in the target video channel, and access the video frame in the buffer block corresponding to the target video channel through the virtual access address of the video frame in the target video channel.

[0149] Furthermore, the device 6 also includes: a first writing module, used to receive video data under each video channel and write the video data under each video channel into the buffer block corresponding to each video channel; wherein, the video data under the video channel consists of video frames under the video channel; wherein, the video channel includes at least a target video channel.

[0150] Furthermore, the device 6 also includes: a first connection module, used to, in response to a connection success command, allocate shared memory corresponding to each video channel, and write the video frame description information corresponding to the video frames under each video channel into the corresponding shared memory respectively.

[0151] Furthermore, the device 6 also includes: a receiving module for receiving video data from each video channel; wherein the video data from each video channel consists of video frames from that video channel;

[0152] The construction module is used to add timestamps to the video frames under each video channel and construct video frame packages based on the video frames under each video channel and the timestamps corresponding to the video frames under each video channel; wherein, a video frame package includes one video frame from each video channel.

[0153] The second writing module is used to construct a video frame package, obtain the spliced ​​video frame according to the video frame package and the preset panoramic view splicing order, and write the spliced ​​video frame to the cache block corresponding to the target video channel; the cache block corresponding to the target video channel is a cache block retrieved from the cache pool.

[0154] Furthermore, the device 6 also includes: a second connection module, used to, in response to a connection success command, allocate shared memory corresponding to the target video channel and write the video frame description information corresponding to the spliced ​​video frames into the shared memory corresponding to the target video channel.

[0155] Furthermore, the construction module is specifically used to: determine the reference video frame and its corresponding timestamp based on the timestamps of the video frames in each video channel; obtain the time interval between the timestamps of the video frames in each video channel and the timestamps of the reference video frame; if the time interval is not less than a preset time interval threshold, discard the corresponding video frame, wait for the next video frame in the video channel where the corresponding video frame is located, and repeat the above steps until all time intervals are less than the preset time interval threshold, thus obtaining a video frame package.

[0156] Furthermore, the device 6 also includes: a first counting module, used to trigger an increase in the reference count value of the video frame description information corresponding to the video frame if the process accesses a video frame in the buffer block corresponding to the target video channel; and a second counting module, used to trigger a decrease in the reference count value of the video frame description information corresponding to the video frame if the process stops accessing the video frame in the buffer block corresponding to the target video channel.

[0157] Furthermore, the device 6 also includes: a first release module, used to release the video frame description information in the shared memory corresponding to the target video channel according to a preset release priority rule when the number of video frame description information stored in the shared memory corresponding to the target video channel exceeds a preset number; wherein, the preset release priority rule from high to low is video frame description information with a reference count of zero at the head of the queue, video frame description information with a reference count of zero at the tail of the queue, and video frame description information waiting to enter the queue.

[0158] Furthermore, the device 6 also includes: a release response module, used to obtain a reference count value of the target video frame and the video frame description information corresponding to the target video frame in response to a release request sent from the process;

[0159] The second release module is used to release the cache corresponding to the target video frame if the reference count value of the video frame description information corresponding to the target video frame is zero.

[0160] It should be noted that the information interaction and execution process between the above modules are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.

[0161] Please see Figure 7 , Figure 7 This is a schematic diagram of a video data sharing system provided in the third embodiment of this application, including a video acquisition device 71 and a vehicle-mounted device 72; the video acquisition device 71 is used to acquire video data under each video channel and send the video data under each video channel to the vehicle-mounted device 72;

[0162] The vehicle-mounted device 72 is equipped with a whole-machine software 721. The functions of the whole-machine software 721 are implemented by a main process 7211 and a slave process 7212. The vehicle-mounted device implements the video data sharing method as described in the first embodiment and / or the second embodiment through the main process 7211 and the slave process 7212.

[0163] Since this system and the method embodiments of this application are based on the same concept, their specific functions and technical effects can be found in the method embodiments section, and will not be repeated here.

[0164] Please see Figure 8 , Figure 8 This is a schematic diagram of the vehicle-mounted device provided in the fourth embodiment of this application. Figure 8 As shown, the vehicle-mounted device 8 of this embodiment includes: a processor 80, a memory 81, and a computer program 82 stored in the memory 81 and executable on the processor 80, such as a video data sharing program. When the processor 80 executes the computer program 82, it implements the steps in the various video data sharing method embodiments described above, for example... Figure 1 Steps S101 to S104 are shown. Alternatively, when the processor 80 executes the computer program 82, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 6 The functions of the acquisition module 61 to the access module 64 are shown.

[0165] For example, the computer program 82 can be divided into one or more modules / units, which are stored in the memory 81 and executed by the processor 80 to complete this application. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program 82 in the vehicle-mounted device 8. For example, the computer program 82 can be divided into an acquisition module, a response module, a sharing module, and an access module, with the specific functions of each module as follows:

[0166] The acquisition module is used to acquire the target video channel, generate and send a connection request to the main process based on the target video channel;

[0167] The response module is used to respond to the connection request, obtain the target video channel, and at least obtain the video access handle corresponding to the target video channel, and send the video access handle corresponding to the target video channel to the slave process.

[0168] The shared module is used to receive the video access handle corresponding to the target video channel and obtain the video frame description information corresponding to the video frame under the target video channel from the shared memory corresponding to the target video channel.

[0169] The access module is used to obtain, from the process, at least the virtual access address of the video frame corresponding to the target video channel based on the video access handle corresponding to the target video channel and the video frame description information corresponding to the video frame under the target video channel, and then access the video frame in the buffer block corresponding to the target video channel through the virtual access address of the video frame under the target video channel.

[0170] The vehicle-mounted device 8 may include, but is not limited to, a processor 80 and a memory 81. Those skilled in the art will understand that... Figure 8 This is merely an example of vehicle-mounted device 8 and does not constitute a limitation on vehicle-mounted device 8. It may include more or fewer components than shown, or combine certain components, or different components. For example, vehicle-mounted device 8 may also include input / output devices, network access devices, buses, etc.

[0171] The processor 80 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0172] The memory 81 can be an internal storage unit of the vehicle-mounted device 8, such as a hard drive or memory. The memory 81 can also be an external storage device of the vehicle-mounted device 8, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the vehicle-mounted device 8 can include both internal and external storage units. The memory 81 is used to store the computer program and other programs and data required by the vehicle-mounted device 8. The memory 81 can also be used to temporarily store data that has been output or will be output.

[0173] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0174] This application also provides a network device, which includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the processor executes the computer program to implement the steps in any of the above method embodiments.

[0175] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various method embodiments above.

[0176] This application provides a computer program product that, when run on a mobile terminal, enables the mobile terminal to implement the steps described in the above-described method embodiments.

[0177] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a photographing device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0178] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0179] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0180] In the embodiments provided in this application, it should be understood that the disclosed apparatus / network devices and methods can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0181] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0182] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A video data sharing method, characterized in that, include: Obtain the target video channel from the process, and generate and send a connection request to the main process based on the target video channel; In response to the connection request, the main process obtains the target video channel and at least the video access handle corresponding to the target video channel, and sends the video access handle corresponding to the target video channel to the slave process. The process receives the video access handle corresponding to the target video channel from the process, and obtains the video frame description information corresponding to the video frame under the target video channel from the shared memory corresponding to the target video channel; The slave process obtains at least the virtual access address corresponding to the video frame under the target video channel based on the video access handle corresponding to the target video channel and the video frame description information corresponding to the video frame under the target video channel, and accesses the video frame in the cache block corresponding to the target video channel through the virtual access address corresponding to the video frame under the target video channel.

2. The video data sharing method as described in claim 1, characterized in that, The main process responds to the connection request by including: The main process receives video data from each video channel and writes the video data from each video channel into the corresponding buffer block of each video channel; wherein, the video data from each video channel consists of video frames from that video channel; wherein, the video channel includes at least the target video channel.

3. The video data sharing method as described in claim 2, characterized in that, The main process responds to the connection request by including: In response to a successful connection command, the main process allocates shared memory for each video channel and writes the video frame description information for each video frame in the respective video channel into the corresponding shared memory.

4. The video data sharing method as described in claim 1, characterized in that, The main process responds to the connection request by including: The main process receives video data from each video channel; wherein, the video data from each video channel is composed of video frames from that video channel. The main process adds timestamps to the video frames under each video channel, and constructs a video frame package based on the video frames under each video channel and the timestamps corresponding to the video frames under each video channel; wherein, a video frame package includes one video frame from each video channel. For each video frame package constructed, the main process obtains the spliced ​​video frame according to the video frame package and the preset surround view splicing order, and writes the spliced ​​video frame into the cache block corresponding to the target video channel; the cache block corresponding to the target video channel is a cache block retrieved from the cache pool.

5. The video data sharing method as described in claim 4, characterized in that, The main process responds to the connection request by including: In response to a successful connection command, the main process allocates shared memory corresponding to the target video channel and writes the video frame description information corresponding to the spliced ​​video frames into the shared memory corresponding to the target video channel.

6. The video data sharing method as described in claim 4, characterized in that, The step of constructing a video frame package based on the video frames under each video channel and the timestamps corresponding to the video frames under each video channel includes: The main process determines the reference video frame and the timestamp corresponding to the reference video frame based on the timestamps corresponding to the video frames under each video channel. The main process obtains the time interval between the timestamp corresponding to the video frame under each video channel and the timestamp corresponding to the reference video frame. If the time interval is not less than a preset time interval threshold, the main process discards the corresponding video frame, waits for the next video frame in the video channel where the corresponding video frame is located, and repeats the above steps until all time intervals are less than the preset time interval threshold, thus obtaining the video frame package.

7. The video data sharing method according to any one of claims 1 to 6, characterized in that, Also includes: If the slave process accesses a video frame in the cache block corresponding to the target video channel, the slave process triggers an increment of the reference count value of the video frame description information corresponding to the video frame; If the slave process stops accessing the video frames in the cache block corresponding to the target video channel, the slave process triggers a reduction in the reference count value of the video frame description information corresponding to the video frame.

8. A video data sharing method as described in claim 7, characterized in that, The video frame description information corresponding to the video frame is stored in the shared memory corresponding to the target video channel in the form of a queue, and also includes: If the number of video frame descriptions stored in the shared memory corresponding to the target video channel exceeds a preset number, the main process releases the video frame descriptions in the shared memory corresponding to the target video channel according to a preset release priority rule. The preset release priority rule, from high to low, is: video frame descriptions with a reference count of zero at the head of the queue, video frame descriptions with a reference count of zero at the tail of the queue, and video frame descriptions waiting to enter the queue.

9. A video data sharing method as described in any one of claims 1 to 6, characterized in that, Also includes: In response to the release request sent by the slave process, the main process obtains the reference count value of the target video frame and the video frame description information corresponding to the target video frame; If the reference count of the video frame description information corresponding to the target video frame is zero, then the main process releases the cache corresponding to the target video frame.

10. A video data sharing system, characterized in that, include: Video capture equipment and vehicle-mounted equipment; The video acquisition device is used to acquire video data from each video channel and send the video data from each video channel to the vehicle-mounted device. The vehicle-mounted device is equipped with system software, the functions of which are implemented by a main process and a slave process. The vehicle-mounted device implements the video data sharing method as described in any one of claims 1 to 9 through the main process and the slave process.

Images