Camera module and video data processing method

Abstract

The invention discloses a camera module and a video data processing method. The camera module comprises a physical camera; the first video acquisition module is used for encoding the original video data according to a first data protocol to obtain first video data, and transmitting the first video data to the first display equipment; the video data format conversion module is used for converting the target video data into universal serial bus (USB) video data supported by a USB protocol and sending the USB video data to the USB module; the target video data source comprises a first equipment file corresponding to a physical camera or a second equipment file corresponding to a virtual camera; and the USB module is used for packaging the USB video data according to the data transmission structure corresponding to the USB protocol and sending the USB video data to the second display equipment. According to the embodiment of the invention, the video data is sent to the display equipment connected based on different communication protocols, and the flexibility and universality of the camera in diversified application scenes are improved.

Description

Technical Field

[0001] This application belongs to the field of electronic technology, and in particular relates to a camera module and a video data processing method. Background Technology

[0002] With the development of technology and the increasing demands of users, the functions and types of cameras are also gradually increasing. For example, cameras can include security cameras and USB Video Class (UVC) cameras.

[0003] Security cameras can acquire video data from relevant areas (such as the living room or doorway), encode the audio and video data to ensure that the encapsulated video data meets the corresponding network protocols, thereby ensuring that the video data can be sent to the user's monitoring device via the Internet, enabling the user to conduct security monitoring.

[0004] However, in related technologies, security cameras can only send video data to monitoring equipment via the Internet, and cannot simultaneously send the acquired video data to other display devices. Therefore, this restricts the flexibility and versatility of security cameras in diverse application scenarios. Summary of the Invention

[0005] This application provides a camera module and a video data processing method that can send video data to display devices connected based on different communication protocols, thereby improving the flexibility and versatility of the camera in diverse application scenarios.

[0006] In a first aspect, an embodiment of this application provides a camera module, comprising: a physical camera for acquiring raw video data; a first video acquisition module connected to the physical camera for encoding the raw video data obtained from a first device file corresponding to the physical camera according to a first data protocol to obtain first video data, and transmitting the first video data to a first display device; a video data format conversion module for converting target video data obtained from a target data source into USB video data supported by the Universal Serial Bus (USB) protocol, and sending the USB video data to a USB module; the target video data source includes a first device file corresponding to the physical camera, or a second device file corresponding to a virtual camera; the virtual camera is created by copying raw video data when a USB connection is established between the second display device and the camera module; and a USB module for encapsulating the USB video data according to the data transmission structure corresponding to the USB protocol to obtain a target data packet, and sending it to the second display device.

[0007] In one embodiment, the USB module includes: a video data processing module for encapsulating USB video data to obtain a USB video data packet; and a data structure processing module for encapsulating the USB video data packet according to the data transmission structure corresponding to the USB protocol to obtain a target data packet.

[0008] In one embodiment, the camera module further includes: a camera status determination module connected to a video data format conversion module and a USB module; the USB module is further configured to: receive a data processing request sent by a second display device, decrypt the data processing request according to the data transmission structure corresponding to the USB protocol to obtain a video data processing instruction, and send the video data processing instruction to the video data format conversion module; and, upon receiving a data processing request, send a status determination request to the camera status determination module; the camera status determination module is configured to, upon receiving the status determination request, determine the status of the physical camera, obtain identification information indicating whether the physical camera is occupied, and send the identification information to the video data format conversion module; wherein, the identification information includes first identification information indicating that the physical camera is occupied, or second identification information indicating that the physical camera is not occupied; the video data format conversion module is further configured to, upon receiving a video data processing instruction, convert the video data processing instruction to obtain a first driving instruction capable of driving the physical camera, and, upon receiving the first identification information, determine that the target video data source is a second device file; upon receiving the second identification information, determine that the target video data source is a first device file; and send the driving instruction to the target video data source.

[0009] In one embodiment, the first driving instruction includes a video acquisition instruction and / or a control instruction; the target video data source is used to acquire video data according to the video acquisition instruction upon receiving the video acquisition instruction; and / or, the target video data source is used to control the camera according to the data acquisition instruction upon receiving the control instruction.

[0010] In one embodiment, the camera module further includes a virtual camera creation module, used to create a virtual camera when a USB connection is established between the second display device and the camera module, and the virtual camera corresponding to the second device file is obtained by copying the video data captured by the physical camera.

[0011] In one embodiment, the camera module further includes: a microphone module for acquiring raw audio data; an audio acquisition module connected to the microphone module for storing raw audio data from the audio device file corresponding to the microphone module into a cache module, encoding the raw audio data stored in the cache module according to a first data protocol to obtain first audio data, and transmitting the first audio data to a first display device; an audio data format conversion module for converting target audio data obtained from a target audio data source into USB audio data supported by the Universal Serial Bus (USB) protocol; the target audio data source includes the cache module or the audio device file corresponding to the microphone module; and a USB module for encapsulating the USB audio data according to the data transmission structure corresponding to the USB protocol to obtain a target data packet, and sending it to a second display device.

[0012] In one embodiment, the USB module further includes: an audio data processing module for encapsulating USB audio data to obtain a USB audio data packet; and a data structure processing module for encapsulating the USB audio data packet according to the data transmission structure corresponding to the USB protocol to obtain a target data packet.

[0013] In one embodiment, the camera module further includes: a microphone module status determination module; the USB module is further configured to: receive a data processing request sent by a second display device, and deseal the data processing request according to the data transmission structure corresponding to the USB protocol to obtain an audio data processing instruction, and send the audio data processing instruction to an audio data format conversion module; and, upon receiving a data processing request, send a status determination request to the microphone module status determination module; the microphone module status determination module is configured to, upon receiving the status determination request, determine the status of the microphone module, obtain identification information indicating whether the microphone module is occupied, and send the identification information to the audio data format conversion module; wherein, the identification information includes third identification information indicating that the microphone module is occupied, or fourth identification information indicating that the microphone module is not occupied; the audio data format conversion module is further configured to, upon receiving an audio data processing instruction, convert the audio data processing instruction to obtain a second driving instruction capable of driving the microphone module, and, upon receiving the third identification information, determine that the target audio data source is a cache module; upon receiving the fourth identification information, determine that the target audio data source is an audio device file; and send the second driving instruction to the target audio data source.

[0014] Secondly, embodiments of this application provide a video data processing method applied to a camera module as described in the first aspect or any embodiment of the first aspect. The method includes: a first video acquisition module acquiring raw video data from a first device file corresponding to a physical camera, encoding it according to a first data protocol to obtain first video data, and transmitting the first video data to a first display device; a video data format conversion module converting target video data acquired from a target video data source into USB video data supported by the Universal Serial Bus (USB) protocol, and sending the USB video data to a USB module; wherein the target video data source includes a first device file corresponding to a physical camera, or a second device file corresponding to a virtual camera; and upon receiving the USB video data, the USB module encapsulates the USB video data according to the data transmission structure corresponding to the USB protocol to obtain a target data packet, and sends it to a second display device.

[0015] The camera module and video data processing method of this application embodiment can send video data acquired by the camera to a first display device, including a monitoring device, through a first video acquisition module, and send the video data to a second display device, including a display device connected based on the USB protocol, through a video data format conversion module and a USB module. Therefore, the camera module provided in this application embodiment can send video data to different devices. Furthermore, the first video acquisition module can encode the raw video data obtained from the first device file corresponding to the physical camera according to a first data protocol, thereby enabling the video data to be transmitted to the monitoring device and ensuring the correctness of video data transmission on the monitoring device side. The video data format conversion module converts the target video data obtained from the target video data source into USB video data supported by the Universal Serial Bus (USB) protocol and sends the USB video data to the USB module; the USB module encapsulates the USB video data according to the data transmission structure corresponding to the USB protocol to obtain a target data packet, and sends it to the second display device, thus enabling the video data to be transmitted to the second display device and ensuring the correctness of video data transmission on the second display device side. Therefore, the camera module provided in this application embodiment can send video data to display devices connected based on different communication protocols, thereby improving the flexibility and versatility of the camera in diverse application scenarios. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1This application shows a schematic diagram of the architecture of a camera module according to one embodiment.

[0018] Figure 2 This invention provides a schematic diagram of the architecture of another camera module according to one embodiment of the present application.

[0019] Figure 3 This illustration shows a schematic diagram of the structure of a UVC data packet provided in one embodiment of this application;

[0020] Figure 4 This illustration shows a schematic diagram of the structure of a URB data packet according to an embodiment of this application;

[0021] Figure 5 This illustration shows a schematic diagram of the architecture of another camera module provided in one embodiment of this application;

[0022] Figure 6 This illustration shows a schematic diagram of the architecture of another camera module provided in one embodiment of this application;

[0023] Figure 7 This illustration shows a schematic diagram of the architecture of yet another camera module provided in one embodiment of this application;

[0024] Figure 8 This invention provides a schematic diagram of the architecture of another camera module according to one embodiment of the present application.

[0025] Figure 9 This illustration shows a schematic diagram of the architecture of another camera module provided in one embodiment of this application;

[0026] Figure 10 The diagram shows a flowchart of a video data processing method according to an embodiment of this application. Detailed Implementation

[0027] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.

[0028] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0029] With the development of technology and the increasing demands of users, the functions and types of cameras have gradually increased. For example, to meet users' needs for real-time monitoring of target areas (such as living rooms and doorways), engineers have designed security cameras. Security cameras can transmit video data acquired by the camera to the user's monitoring equipment in real time via the internet, enabling the user to monitor the status of the target area. Security cameras can be built on a Linux system, and under the Linux system architecture, the kernel driver framework V4L2 (Video for Linux 2), used to process video devices in the Linux kernel, encodes the video data acquired by the camera according to video encoding standards, and encapsulates the encoded video data according to the Secure Reliable Transport (SRT) data format. Furthermore, the encapsulated video data is transmitted to the user's monitoring equipment via the internet, allowing the user to obtain the video data and realize the monitoring function of the security camera.

[0030] Among them, video coding standards may include Advanced Video Coding (AVC, or H.264) or High Efficiency Video Coding (HEVC, or H.265).

[0031] However, in related technologies, security cameras can only send video data to monitoring equipment via the internet, and cannot simultaneously send the acquired video data to other display devices. For example, when a security camera is installed in the living room, it can only send the acquired video data to the monitoring equipment, and cannot send the video data to other display devices, such as a television connected to the camera via a USB cable. In this case, if the television needs to acquire data from the camera for video calls, recording, or other functions, it can only be done by adding a new camera. This limits the flexibility and versatility of security cameras in diverse application scenarios.

[0032] To address the problems of the prior art, this application provides a camera module and a video data processing method. The camera module provided in this application is described below.

[0033] Figure 1 A schematic diagram of the architecture of a camera module provided in one embodiment of this application is shown.

[0034] like Figure 1 As shown, the camera module 100 includes:

[0035] Physical camera 101 is used to collect raw video data;

[0036] The first video acquisition module 102 is connected to the physical camera 101 and is used to encode the raw video data obtained from the first device file corresponding to the physical camera 101 according to the first data protocol to obtain the first video data, and transmit the first video data to the first display device.

[0037] The video data format conversion module 103 is used to convert target video data obtained from the target video data source into USB video data supported by the Universal Serial Bus (USB) protocol, and send the USB video data to the USB module 104; the target video data source includes a first device file corresponding to the physical camera 101, or a second device file corresponding to the virtual camera 105; the virtual camera 105 is created by copying the original video data when the second display device establishes a USB connection with the camera module 100;

[0038] USB module 104 is used to encapsulate USB video data according to the data transmission structure corresponding to the USB protocol, obtain the target video data packet, and send it to the second display device.

[0039] For example, raw video data can be acquired via a physical camera 101. In one example, the physical camera may include a lens and an image sensor. The lens collects light and focuses it onto the image sensor. The image sensor converts the focused light signal into an electrical signal, thereby enabling the video signal to be transmitted to a display device. In yet another example, the raw video data may be video data acquired under a V4L2 architecture.

[0040] For example, the first video acquisition module 102 is connected to the physical camera 101 and is used to encode the raw video data obtained from the first device file corresponding to the physical camera 101 according to the first data protocol to obtain the first video data, and transmit the first video data to the first display device.

[0041] For example, the first device file can be the connection interface between the system kernel of the camera module and the bracket of the physical camera hardware device. The physical camera can be controlled through the first device file. In one example, the physical camera can be controlled to acquire raw video data through the first device file. The raw video data can be video data acquired by the physical camera.

[0042] For example, the first video acquisition module 102 can encode the raw video data according to a first data protocol to obtain first video data that satisfies the first data protocol. In one example, the first data protocol may be the SRT protocol.

[0043] In one example, the raw video data captured by the physical camera 101 is encoded using H.264 or H.265, and the encoded video data is encapsulated according to the data format corresponding to SRT. The encapsulated first video data is then transmitted to a first display device via the Internet. For example, the first display device may be a user's monitoring device.

[0044] For example, the video data format conversion module 103 can convert the video data acquired by the camera so that the converted data format meets the corresponding data transmission protocol.

[0045] For example, the video data format conversion module 103 can convert video data into USB video data supported by the Universal Serial Bus (USB) protocol and send the USB data to the USB module 104.

[0046] The video data obtained by the video data format conversion module 103 can be video data obtained from the target video data source.

[0047] For example, the target video data source may include a first device file corresponding to the physical camera 101, or a second device file corresponding to the virtual camera 105.

[0048] The virtual camera 105 is created by copying the original video data when a USB connection is established between the second display device and the camera module 100. In one example, the second device file in the virtual camera can copy the original video data through memory mapping.

[0049] For example, the USB module 104 is used to encapsulate USB video data according to the data transmission structure corresponding to the USB protocol, obtain a target video data packet, and send it to the second display device.

[0050] In one example, the USB module 104 can encapsulate the USB video data according to the data transmission structure corresponding to the USB protocol to obtain a USB request block (URB), and send the URB to the second display device.

[0051] For example, the connection between the above modules can be a communication connection.

[0052] In this embodiment, video data acquired by a camera can be sent to a first display device (including a monitoring device) via a first video acquisition module, and then sent to a second display device (including a display device connected via the USB protocol) via a video data format conversion module and a USB module. Therefore, the camera module provided in this embodiment can send video data to different devices. Furthermore, the first video acquisition module can encode the raw video data obtained from the first device file corresponding to the physical camera according to a first data protocol, thereby enabling the video data to be transmitted to the monitoring device and ensuring correct video data transmission on the monitoring device side. The video data format conversion module converts the target video data obtained from the target video data source into USB video data supported by the Universal Serial Bus (USB) protocol and sends the USB video data to the USB module. The USB module encapsulates the USB video data according to the data transmission structure corresponding to the USB protocol to obtain the target video data packet, and sends it to the second display device, thus enabling the video data to be transmitted to the second display device and ensuring correct video data transmission on the second display device side. Therefore, the camera module provided in this embodiment can send video data to display devices connected based on different communication protocols, improving the flexibility and versatility of the camera in diverse application scenarios.

[0053] In order to enable video data to be transmitted via USB module, as another implementation of this application, another implementation of the camera module is also provided, as detailed in the following embodiments.

[0054] Figure 2 A schematic diagram of the architecture of another camera module provided in one embodiment of this application is shown. Figure 2 As shown, the USB module 104 in the camera module 100 also includes:

[0055] The video data processing module 1041 is used to encapsulate USB video data to obtain USB video data packets;

[0056] The data structure processing module 1042 is used to encapsulate USB video data packets according to the data transmission structure corresponding to the USB protocol to obtain target video data packets.

[0057] For example, the video data processing module 1041 can encapsulate USB video data to obtain a USB video data packet.

[0058] In one example, the video data processing module 1041 can be a USB Video Class (UVC) module. The UVC module encapsulates the video data to obtain UVC data packets. These UVC data packets are encapsulated according to the UVC specification format. For example... Figure 3 As shown, the UCV specification format can be achieved by adding a 12-byte UVC header to uncompressed video data. Figure 3 A schematic diagram of the structure of a UVC data packet provided in one embodiment of this application is shown.

[0059] For example, the data structure processing module 1042 can encapsulate the USB video data packet according to the data transmission structure corresponding to the USB protocol to obtain the target video data packet.

[0060] For example, the data structure processing module 1042 can encapsulate the UVC data packet according to the USB protocol to obtain the URB data packet. Wherein, as... Figure 4 As shown, a URB data packet can include a USB header and a UVC data packet. Figure 4 A schematic diagram of the structure of a URB data packet provided in one embodiment of this application is shown.

[0061] In this embodiment, the video data processing module 1041 encapsulates the video data into a USB video data packet, and further, the USB video data packet is subdivided to obtain the target video data packet, so that the target video data packet can be transmitted via the USB protocol.

[0062] For example, the camera module 100 can obtain video data based on the data processing request sent by the second display device after receiving the data processing request. As another implementation of this application, this application also provides another implementation of the camera module, as detailed in the following embodiments.

[0063] Figure 5 A schematic diagram of the architecture of another camera module provided in one embodiment of this application is shown. Figure 5 As shown, the camera module 100 also includes:

[0064] The camera status determination module 106 is connected to the video data format conversion module 103 and the USB module 104.

[0065] The USB module 104 in the camera module 100 is also used to receive data processing requests sent by the second display device, decrypt the data processing requests according to the data transmission structure corresponding to the USB protocol, obtain video data processing instructions, and send the video data processing instructions to the video data format conversion module 103. The USB module 104 is also used to send a status determination request to the camera status determination module 106 when a data processing request is received.

[0066] The camera status determination module 106 is used to determine the status of the physical camera 101 after receiving a status determination request, obtain identification information indicating whether the physical camera 101 is occupied, and send the identification information to the video data format conversion module 103. The identification information includes first identification information indicating that the physical camera 101 is occupied, or second identification information indicating that the physical camera 101 is not occupied.

[0067] The video data format conversion module 103 is further configured to, upon receiving a video data processing instruction, convert the video data processing instruction to obtain a first driving instruction capable of driving the physical camera 101 or the virtual camera 105; and, upon receiving first identification information, determine that the target video data source is a second device file; upon receiving second identification information, determine that the target video data source is a first device file; and send the first driving instruction to the target video data source.

[0068] For example, after receiving a data processing request from the second display device, the camera module obtains the corresponding video data based on the data processing request, thereby enabling the camera module to respond to the data processing request.

[0069] For example, the USB module 104 can receive data processing requests sent by the second display device. The data processing request received by the USB module can be a URB data packet containing the data processing request. Furthermore, after receiving the URB data packet, the USB module can decrypt the URB data packet to obtain video data processing instructions, and send the video data processing instructions to the video data format conversion module 103. In one example, the video data processing instructions can be instructions in the UVC data format.

[0070] Furthermore, when the USB module 104 receives a data processing request from the second display device, it can simultaneously send a status determination request to the camera status determination module 106.

[0071] For example, after receiving a status determination request sent by the USB module 104, the camera status determination module 106 can determine the status of the physical camera 101, obtain identification information as to whether the physical camera 101 is occupied, and send the identification information to the video data format conversion module 103.

[0072] In one example, the camera status determination module 106 can store physical camera configuration information, physical camera status information, etc. After receiving a status determination request sent by the USB module 104, the camera status determination module 106 can determine whether the physical camera is occupied based on the physical camera configuration information and physical camera status information. It can be understood that the physical camera being occupied can mean that the first video acquisition module 102 is acquiring video data through the physical camera; the physical camera not being occupied can mean that the first video acquisition module 102 is not acquiring video data through the physical camera.

[0073] For example, after determining that the physical camera is occupied, the camera status determination module 106 can send first identification information to the video data format conversion module 103; after determining that the physical camera is not occupied, the camera status determination module 106 can send second identification information to the video data format conversion module 103.

[0074] For example, after receiving the video data processing instruction sent by the USB module 104, the video data format conversion module 103 can decrypt the video data processing instruction to obtain the first driving instruction corresponding to the V4L2 architecture. Control of the physical or virtual camera is achieved by sending the first driving instruction to the target video data source. It is understood that in this application, the camera can be controlled through the device file corresponding to the target video data source; for example, a physical camera can be controlled through a first device file, and a virtual camera can be controlled through a second device file.

[0075] In some optional embodiments, the first driving instruction may include a video acquisition instruction and / or a control instruction. Furthermore, the target video data source is used to acquire video data according to the video acquisition instruction upon receiving it; and / or, the target video data source is used to control the camera according to the data acquisition instruction upon receiving the control instruction.

[0076] In one example, the operation instructions may include operations such as setting the camera resolution.

[0077] The camera module provided in this application embodiment can not only send video data to display devices under different communication protocols, but also control the camera through display devices under different protocols, thereby improving the flexibility and versatility of the camera in diverse application scenarios.

[0078] For example, upon receiving the identification information sent by the camera status determination module 106, the video data format conversion module 103 can determine the target video data source based on the identification information. In one example, if the video data format conversion module 103 receives the first identification information, it determines the target video data source to be a second device file; if the video data format conversion module 103 receives the second identification information, it determines the target video data source to be a first device file. Furthermore, a first driving command can be sent to the target video data source, thereby enabling the acquisition of video data or the control of the camera through the device file in the target data source.

[0079] In one example, if it is determined that the physical camera is already in use, video data can only be obtained from the second device file, and the camera cannot be manipulated, such as changing the camera resolution.

[0080] In another example, the data processing request sent by the second display device may also include a request to obtain a descriptor. This request may include obtaining a UVC configuration descriptor, a UVC interface descriptor, a UVC endpoint descriptor, etc. Furthermore, the request to obtain the descriptor can be sent from the second display device to the USB module 104 in the form of a USB data packet. The USB module 104 decapsulates the data packet to obtain the corresponding UVC format request and sends this request to the camera status determination module 106. After determining the descriptor information corresponding to the descriptor request, the camera status determination module 106 feeds the descriptor information back to the USB module. After being encapsulated by the USB module, the information is returned to the second display module.

[0081] In another example, the data processing request may also include instructions to control the camera's movement in all directions. Since the UVC specification does not include specific provisions for UVC camera movement, the camera movement function can be implemented by extending the instructions. For example, the movement control of the physical camera can be achieved by directly sending the instructions to control the camera's movement in all directions to the first device file.

[0082] In this embodiment, the target source file is determined by the camera status determination module, thereby ensuring that both the first and second display devices can acquire video data under any circumstances. Furthermore, a data processing request is received via a USB module, and the data processing instructions are decrypted according to the USB protocol to obtain video data processing instructions. These instructions are then sent to a video data format conversion module, which decrypts the instructions to obtain a first driving instruction. Based on this first driving instruction, video data acquisition is achieved.

[0083] It is understandable that in Linux systems, different processes are not allowed to access a device file simultaneously. Therefore, in this embodiment, to ensure that both display devices can obtain the video data acquired by the camera, a virtual camera is constructed, and a second device file corresponding to the virtual camera is constructed, so that the second display device can obtain the video data when needed. Therefore, as another implementation of this application, this application also provides another implementation of the camera module, as detailed in the following embodiments.

[0084] Figure 6 A schematic diagram of the architecture of another camera module provided in one embodiment of this application is shown. Figure 6 As shown, the camera module 100 also includes:

[0085] The virtual camera creation module 107 is used to create a virtual camera when a USB connection is established between the second display device and the camera module. The virtual camera corresponding to the second device file is obtained by copying the video data captured by the physical camera.

[0086] For example, the virtual camera creation module 107 can be a V4L2 loopback driver (i.e., a V4L2 lookback). In one example, after the camera module is powered on, during the Linux system initialization process of the camera module, the V4L2 loopback driver is loaded and a virtual camera is created. The path to the virtual camera is set to / dev / video1, while the path to the physical camera is set to / dev / video0. Furthermore, the video data from the physical camera is output to the virtual camera via a Linux startup script, enabling the virtual camera to acquire video data.

[0087] In this embodiment of the application, the virtual camera creation module constructs a virtual camera and a second device file corresponding to the virtual camera, so that when the second display device needs video data, it can obtain video data from the second device file.

[0088] It is understood that the camera module can acquire not only video data but also audio data. To achieve audio data acquisition, as another implementation of this application, another implementation of the camera module is provided, as detailed in the following embodiments.

[0089] Figure 7 This illustration shows a schematic diagram of the architecture of another camera module provided in one embodiment of this application. For example... Figure 7 As shown, the camera module 100 also includes:

[0090] Microphone module 108 is used to capture raw audio data;

[0091] The audio acquisition module 109 is connected to the microphone module and is used to store the raw audio data from the audio device file corresponding to the microphone module into the cache module, encode the raw audio data stored in the cache module according to the first data protocol to obtain the first audio data, and transmit the first audio data to the first display device.

[0092] The audio data format conversion module 110 is also used to convert target audio data obtained from the target audio data source into USB audio data supported by the Universal Serial Bus (USB) protocol; the target audio data source includes a buffer module or an audio device file corresponding to the microphone module;

[0093] The USB module 104 is also used to encapsulate USB audio data according to the data transmission structure corresponding to the USB protocol, obtain the target audio data packet, and send it to the second display device.

[0094] For example, raw audio data can be acquired using a microphone module. In one example, the raw audio data acquired by the microphone module can be audio data obtained based on the V4L2 architecture.

[0095] For example, the audio acquisition module 109 can be connected to the microphone module 108, and the audio acquisition module 109 may contain a cache module 1091. The audio acquisition module 109 can store the raw audio data from the audio device file corresponding to the microphone module 108 into the cache module 1091, encode the raw audio data stored in the cache module 1091 according to a first data protocol to obtain first audio data, and transmit the first audio data to the first display device.

[0096] In one example, the audio acquisition module 109 can encode the raw audio data according to a first data protocol to obtain first audio data that satisfies the first data protocol. In one example, the first data protocol may be the SRT protocol.

[0097] In one example, the raw audio data captured by the physical camera 101 is processed by Advanced Audio Coding (AAC) to obtain encoded audio data. The encoded audio data is then encapsulated according to the data format corresponding to SRT. The encapsulated first audio data is transmitted to a first display device via the Internet. For example, the first display device may be a user's monitoring device.

[0098] For example, the audio data format conversion module 110 can convert the audio data acquired by the microphone module so that the converted data format meets the corresponding data transmission protocol.

[0099] For example, the audio data format conversion module 110 can convert audio data into USB audio data supported by the Universal Serial Bus (USB) protocol and send the USB data to the USB module 104.

[0100] The video data obtained by the video data format conversion module 103 can be video data obtained from the target video data source.

[0101] For example, the target audio data source may include the cache module 1091 or the audio device file corresponding to the microphone module 108.

[0102] In one example, the USB module 104 can encapsulate the USB audio data according to the data transmission structure corresponding to the USB protocol to obtain a USB request block (URB), and send the URB to the second display device.

[0103] In this embodiment, audio data acquired by the camera can be sent to a first display device (including a monitoring device) via an audio acquisition module, and then sent to a second display device (including a display device connected via the USB protocol) via an audio data format conversion module and a USB module. Therefore, the camera module provided in this embodiment can send audio data to different devices. Furthermore, the audio acquisition module can encode the raw audio data from the audio device file corresponding to the microphone module according to a first data protocol, enabling the audio data to be transmitted to the monitoring device and ensuring correct audio data transmission on the monitoring device side. The audio data format conversion module converts the audio data obtained from the target audio data source into USB audio data supported by the Universal Serial Bus (USB) protocol and sends the USB audio data to the USB module. The USB module encapsulates the USB audio data according to the data transmission structure corresponding to the USB protocol to obtain the target audio data packet, and sends it to the second display device, ensuring correct audio data transmission on the second display device side. Therefore, the camera module provided in this embodiment can also send audio data to display devices connected based on different communication protocols, improving the flexibility and versatility of the camera in diverse application scenarios.

[0104] In order to enable audio data to be transmitted via USB module, as another implementation of this application, another implementation of the camera module is also provided, as detailed in the following embodiments.

[0105] Figure 8 A schematic diagram of the architecture of another camera module provided in one embodiment of this application is shown. Figure 8 As shown, the USB module 104 in the camera module 100 also includes:

[0106] The audio data processing module 1043 is used to encapsulate USB audio data to obtain USB audio data packets;

[0107] The data structure processing module 1042 is also used to encapsulate the USB audio data packet according to the data transmission structure corresponding to the USB protocol to obtain the target audio data packet.

[0108] For example, the audio / video data processing module 1043 can encapsulate USB audio data to obtain a USB audio data packet.

[0109] In one example, the audio data processing module 1043 can be a USB Audio Class (UAC) module. The UAC module encapsulates the audio data to obtain a UAC data packet. The UAC data packet is encapsulated in the UAC specification format.

[0110] For example, the data structure processing module 1042 can encapsulate the USB audio data packet according to the data transmission structure corresponding to the USB protocol to obtain the target audio data packet.

[0111] For example, the data structure processing module 1042 can encapsulate the UAC data packet according to the USB protocol to obtain a URB data packet. The USB audio data packet may include a UAC header and uncompressed audio data, and the target audio data packet may include a USB header and a UAC data packet. It is understood that the structure of the UAC data packet and target audio data packet corresponding to video data is similar to that of the UAC data packet and target audio data packet corresponding to video data, and will not be elaborated further here.

[0112] In this embodiment, the audio data processing module 1043 encapsulates the audio data into a USB audio data packet, and further, the USB audio data packet is subdivided to obtain the target audio data packet, so that the target audio data packet can be transmitted via the USB protocol.

[0113] For example, the camera module 100 can obtain audio data based on the data processing request sent by the second display device after receiving the data processing request. As another implementation of this application, this application also provides another implementation of the camera module, as detailed in the following embodiments.

[0114] Figure 9 A schematic diagram of the architecture of another camera module provided in one embodiment of this application is shown. Figure 9 As shown, the camera module 100 also includes:

[0115] Microphone module status determination module 111;

[0116] USB module 104 is also used to: receive a data processing request sent by the second display device, deseal the data processing request according to the data transmission structure corresponding to the USB protocol to obtain an audio data processing instruction, and send the audio data processing instruction to the audio data format conversion module 110; and send a status determination request to the microphone module status determination module 111 when a data processing request is received.

[0117] The microphone module status determination module 111 is used to determine the status of the microphone module after receiving a status determination request, obtain identification information indicating whether the microphone module is occupied, and send the identification information to the audio data format conversion module 110; wherein, the identification information includes third identification information indicating that the microphone module is occupied, or fourth identification information indicating that the microphone module is not occupied.

[0118] The audio data format conversion module 110 is also used to convert the audio data processing instruction upon receiving it, so as to obtain a second driving instruction that can drive the microphone module 108.

[0119] Furthermore, when the audio data format conversion module receives the third identification information, it determines that the target audio data source is the cache module 1091; when the audio data format conversion module receives the fourth identification information, it determines that the target audio data source is the audio device file; and sends the second driving instruction to the target audio data source.

[0120] For example, after receiving a data processing request from the second display device, the camera module obtains the corresponding audio data based on the data processing request, thereby enabling the camera module to respond to the data processing request.

[0121] For example, the USB module 104 can receive data processing requests sent by the second display device. The data processing request received by the USB module can be a URB data packet containing the data processing request. Furthermore, after receiving the URB data packet, the USB module can decrypt the URB data packet to obtain audio data processing instructions, and send the audio data processing instructions to the audio data format conversion module 110. In one example, the audio data processing instructions can be instructions in UAC data format.

[0122] Furthermore, when the USB module 104 receives a data processing request sent by the second display device, it can simultaneously send a status determination request to the microphone module status determination module 111.

[0123] For example, after receiving the status determination request sent by the USB module 104, the microphone module status determination module 111 can determine the status of the microphone module 108, obtain the identification information of whether the microphone module 108 is occupied, and send the identification information to the audio data format conversion module 110.

[0124] In one example, the microphone module status determination module 110 can store microphone module configuration information, microphone module status information, etc. After receiving a status determination request sent by the USB module 104, the microphone module status determination module 110 can determine whether the microphone module is occupied based on the microphone module configuration information and microphone module status information. It can be understood that the microphone module being occupied means that the audio acquisition module 109 is acquiring audio data through the microphone module; the microphone module not being occupied means that the audio acquisition module 109 is not acquiring audio data through the microphone module.

[0125] For example, after determining that the microphone module is occupied, the microphone module status determination module 110 can send third identification information to the audio data format conversion module 110; after determining that the microphone module is not occupied, the microphone module status determination module 110 can send fourth identification information to the audio data format conversion module 110.

[0126] For example, after receiving the audio data processing instruction sent by the USB module 104, the audio data format conversion module 110 can decrypt the audio data processing instruction to obtain the second driving instruction corresponding to the V4L2 architecture. Control of the microphone module is achieved by sending the second driving instruction to the target audio data source. It is understood that in this application, the microphone module can be controlled through the device file corresponding to the target audio data source, for example, by controlling the microphone module through an audio device file.

[0127] For example, upon receiving the identification information sent by the microphone module status determination module 110, the audio data format conversion module 110 can determine the target audio data source based on the identification information. In one example, if the audio data format conversion module 110 receives the third identification information, it determines that the target audio data source is the cache module; if the audio data format conversion module 110 receives the fourth identification information, it determines that the target audio data source is the audio device file. Furthermore, a second driving command can be sent to the target audio data source to acquire audio data or to control the microphone module.

[0128] In one example, if it is determined that the microphone module is already in use, audio data can only be retrieved from the cache module, and the camera cannot be controlled, such as changing the camera resolution.

[0129] In another example, the data processing request sent by the second display device may also include a request to obtain a descriptor. This request may include obtaining a UAC configuration descriptor, a UAC interface descriptor, a UAC endpoint descriptor, etc. Furthermore, the request to obtain a descriptor can be sent from the second display device to the USB module 104 in the form of a USB data packet. The USB module 104 decapsulates the data packet to obtain the corresponding UAC format request and sends it to the camera status determination module 110. After determining the descriptor information corresponding to the descriptor request, the camera status determination module 110 feeds the descriptor information back to the USB module. After being encapsulated by the USB module, the information is returned to the second display module.

[0130] In this embodiment, the target source file is determined by the camera status determination module, ensuring that both the first and second display devices can acquire audio data under any circumstances. Furthermore, a data processing request is received via the USB module, and the data processing instructions are decrypted according to the USB protocol to obtain audio data processing instructions. These instructions are then sent to the audio data format conversion module, which decrypts the instructions to obtain a first driving instruction. Based on this second driving instruction, the audio data is then acquired.

[0131] Based on the camera module provided in the above embodiments, this application also provides specific implementations of the video data processing method. Please refer to the following embodiments.

[0132] Figure 10 A flowchart illustrating a video data processing method according to an embodiment of this application is shown. Figure 10 As shown, the video data processing method includes the following steps:

[0133] S101, the first video acquisition module will obtain the original video data from the first device file corresponding to the physical camera, encode it according to the first data protocol to obtain the first video data, and transmit the first video data to the first display device.

[0134] For example, the first video acquisition module can obtain raw video data from the first device file corresponding to the physical camera, encode the raw video data according to the first data protocol to obtain the first video data, and further, transmit the first video data to the first display device.

[0135] S102, the video data format conversion module converts the target video data obtained from the target video data source into USB video data supported by the Universal Serial Bus (USB) protocol, and sends the USB video data to the USB module.

[0136] The target video data source includes either a first device file corresponding to a physical camera or a second device file corresponding to a virtual camera.

[0137] For example, the video data format conversion module can convert target video data obtained from the target video data source into USB video data supported by the Universal Serial Bus (USB) protocol, and send the USB video data to the USB module.

[0138] In one example, the video data format conversion module can convert video data acquired under the V4L2 architecture into USB video data under the USB protocol.

[0139] S103. Upon receiving USB video data, the USB module encapsulates the USB video data according to the data transmission structure corresponding to the USB protocol to obtain the target video data packet, and sends it to the second display device.

[0140] For example, a USB module can encapsulate USB video data so that the USB video data conforms to the data transmission structure corresponding to the USB protocol.

[0141] In this embodiment, video data acquired by a camera can be sent to a first display device (including a monitoring device) via a first video acquisition module, and then sent to a second display device (including a display device connected via the USB protocol) via a video data format conversion module and a USB module. Therefore, the camera module provided in this embodiment can send video data to different devices. Furthermore, the first video acquisition module can encode the raw video data obtained from the first device file corresponding to the physical camera according to a first data protocol, thereby enabling the video data to be transmitted to the monitoring device and ensuring correct video data transmission on the monitoring device side. The video data format conversion module converts the target video data obtained from the target video data source into USB video data supported by the Universal Serial Bus (USB) protocol and sends the USB video data to the USB module. The USB module encapsulates the USB video data according to the data transmission structure corresponding to the USB protocol to obtain the target video data packet, and sends it to the second display device, thus enabling the video data to be transmitted to the second display device and ensuring correct video data transmission on the second display device side. Therefore, the camera module provided in this embodiment can send video data to display devices connected based on different communication protocols, improving the flexibility and versatility of the camera in diverse application scenarios.

[0142] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0143] The above description is merely a specific implementation of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A camera module, comprising: include: Physical cameras are used to capture raw video data; The first video acquisition module is connected to the physical camera and is used to encode the raw video data obtained from the first device file corresponding to the physical camera according to the first data protocol to obtain the first video data, and transmit the first video data to the first display device. The video data format conversion module is used to convert target video data obtained from a target video data source into USB video data supported by the Universal Serial Bus (USB) protocol, and send the USB video data to the USB module; the target video data source includes a first device file corresponding to a physical camera, or a second device file corresponding to a virtual camera; the virtual camera is created by copying the original video data when a second display device establishes a USB connection with the camera module; The USB module is used to encapsulate the USB video data according to the data transmission structure corresponding to the USB protocol to obtain a target video data packet, and send it to the second display device.

2. The camera module of claim 1, wherein, The USB module includes: The video data processing module is used to encapsulate the USB video data to obtain a USB video data packet; The data structure processing module is used to encapsulate the USB video data packet according to the data transmission structure corresponding to the USB protocol to obtain the target video data packet.

3. The camera module of claim 1, wherein, The camera module also includes: The camera status determination module is connected to the video data format conversion module and the USB module; The USB module is further configured to: receive a data processing request sent by the second display device, deseal the data processing request according to the data transmission structure corresponding to the USB protocol to obtain a video data processing instruction, and send the video data processing instruction to the video data format conversion module; and, upon receiving the data processing request, send a status determination request to the camera status determination module. The camera status determination module is used to determine the status of the physical camera after receiving the status determination request, obtain identification information indicating whether the physical camera is occupied, and send the identification information to the video data format conversion module; wherein, the identification information includes first identification information indicating that the physical camera is occupied, or second identification information indicating that the physical camera is not occupied; The video data format conversion module is further configured to, upon receiving the video data processing instruction, convert the video data processing instruction to obtain a first driving instruction capable of driving the physical camera or the virtual camera. Furthermore, when the video data format conversion module receives the first identification information, it determines that the target video data source is a second device file; when the video data format conversion module receives the second identification information, it determines that the target video data source is a first device file; and sends the first driving instruction to the target video data source.

4. The camera module according to claim 3, characterized in that, The first driving command includes video capture commands and / or control commands; The target video data source is used to acquire video data according to the video acquisition instruction upon receiving the video acquisition instruction; And / or, The target video data source is used to control the camera according to the control command when the control command is received.

5. The camera module according to claim 3, characterized in that, The camera module also includes: The virtual camera creation module is used to create the virtual camera when the second display device and the camera module establish a USB connection, and the virtual camera corresponding to the second device file is obtained by copying the video data captured by the physical camera.

6. The camera module according to claim 1, characterized in that, The camera module also includes: Microphone module, used to capture raw audio data; An audio acquisition module, connected to the microphone module, is used to store the original audio data from the audio device file corresponding to the microphone module into a cache module, encode the original audio data stored in the cache module according to a first data protocol to obtain first audio data, and transmit the first audio data to a first display device; The audio data format conversion module is also used to convert target audio data obtained from the target audio data source into USB audio data supported by the Universal Serial Bus (USB) protocol; the target audio data source includes a cache module or an audio device file corresponding to the microphone module; The USB module is also used to encapsulate the USB audio data according to the data transmission structure corresponding to the USB protocol to obtain a target audio data packet, and send it to the second display device.

7. The camera module according to claim 6, characterized in that, The USB module also includes: An audio data processing module is used to encapsulate the USB audio data to obtain a USB audio data packet; The data structure processing module is also used to encapsulate the USB audio data packet according to the data transmission structure corresponding to the USB protocol to obtain the target audio data packet.

8. The camera module according to claim 6, characterized in that, The camera module also includes: Microphone module status determination module; The USB module is further configured to: receive a data processing request sent by the second display device, deseal the data processing request according to the data transmission structure corresponding to the USB protocol to obtain an audio data processing instruction, and send the audio data processing instruction to the audio data format conversion module; and, upon receiving the data processing request, send a status determination request to the microphone module status determination module. The microphone module status determination module is used to determine the status of the microphone module after receiving the status determination request, obtain identification information indicating whether the microphone module is occupied, and send the identification information to the audio data format conversion module; wherein, the identification information includes third identification information indicating that the microphone module is occupied, or fourth identification information indicating that the microphone module is not occupied; The audio data format conversion module is further configured to, upon receiving the audio data processing instruction, convert the audio data processing instruction to obtain a second driving instruction capable of driving the microphone module. Furthermore, when the audio data format conversion module receives the third identification information, it determines that the target audio data source is a cache module; when the audio data format conversion module receives the fourth identification information, it determines that the target audio data source is an audio device file; and sends the second driving instruction to the target audio data source.

9. A video data processing method, characterized in that, Applied to a camera module as described in any one of claims 1-8, the method includes: The first video acquisition module will obtain the original video data from the first device file corresponding to the physical camera, encode it according to the first data protocol to obtain the first video data, and transmit the first video data to the first display device. The video data format conversion module converts the target video data obtained from the target video data source into USB video data supported by the Universal Serial Bus (USB) protocol, and sends the USB video data to the USB module; wherein, the target video data source includes a first device file corresponding to a physical camera, or a second device file corresponding to a virtual camera; Upon receiving the USB video data, the USB module encapsulates the USB video data according to the data transmission structure corresponding to the USB protocol to obtain the target video data packet, and sends it to the second display device.

Images