Camera apparatus supporting two-way video call

Abstract

Disclosed in the present application is a camera apparatus supporting two-way video call, which is used for acquiring first audio data and first image data, and performing data transmission with a mobile terminal by means of a cloud. The camera apparatus comprises a communication module, a data processing module, an image display module and an audio playback module; the camera apparatus acquires, by means of the communication module, second audio data and second image data uploaded to the cloud by the mobile terminal. The data processing module is used for performing data processing on the acquired second audio data and second image data, the image display module is used for acquiring and displaying the processed second image data, and the audio playback module is used for acquiring and playing back the processed second audio data. By means of the arrangement, the camera apparatus can play back the second image data and the second audio data acquired by the mobile terminal, thus improving the efficiency of information transmission between the camera apparatus and the mobile terminal, and optimizing the call experience of users at the camera apparatus end.

Description

Camera device that supports two-way video calling Technical Field

[0001] The present application relates to the field of IoT smart home, and in particular to a camera device that supports two-way video calls. Background Art

[0002] Smart home cameras typically have built-in microphones for voice input. Mobile phones can view the camera's real-time audio and video feed. However, the camera itself cannot see the mobile phone's live audio and video feed, creating a poor user experience. This is especially true when communicating via the camera, as existing cameras only support one-way audio and video transmission, making it difficult for the camera user to clearly convey their desired message to the mobile user.

[0003] Summary of the Invention

[0004] In order to address the deficiencies of the prior art, the purpose of the present application is to provide a camera device that supports two-way video calls, which improves the efficiency of information transmission between devices.

[0005] To achieve the above objectives, this application adopts the following technical solutions:

[0006] A camera device supporting two-way video calls is used to obtain first audio data and first image data, and transmit the data to a mobile terminal via a cloud, so that the mobile terminal receives and plays the first audio data and the first image data. The camera device includes:

[0007] a communication module, used for information exchange between the camera device and the cloud, wherein the camera device obtains the second audio data and the second image data uploaded to the cloud by the mobile terminal through the communication module;

[0008] a data processing module, configured to perform data processing on the acquired second audio data and second image data, wherein the data processing includes at least one of decoding, synchronizing, and rendering the second audio data and second image data;

[0009] a display module, configured to acquire and display the processed second image data;

[0010] The audio playback module is used to obtain and play the processed second audio data.

[0011] Furthermore, the camera device includes a base and a camera module that can rotate relative to the base, the camera module is used to obtain the first image data, and the display module is fixedly arranged on the base.

[0012] Furthermore, the camera device sends the device information to the cloud through the communication module and stores it in a communication list established in the cloud, so that the camera device can send a video call request to any device in the communication list, wherein the communication list includes at least the device information of the camera device and the device information of the mobile terminal.

[0013] Furthermore, the camera device is connected to any display device through the cloud, wherein the display device includes at least one of a central console and a fixed terminal, and the device information of the display device is stored in a communication list, so that the camera device sends a video call request to the display device through the cloud.

[0014] Furthermore, the camera device sends a video call request to the cloud through the MQTT protocol, so that at least one device in the communication list obtains the video call request sent by the cloud.

[0015] Furthermore, at least two camera devices are connected via cloud communication, and at least two camera devices can send collected audio and video data to each other;

[0016] At least two camera devices include a sending end and at least one receiving end. The sending end sends a video call request to the cloud through the MQTT protocol and receives feedback information sent by the cloud. The feedback information includes at least one of the connection, waiting, rejection, and hang-up operations sent by the receiving end.

[0017] Furthermore, the data processing module includes:

[0018] a feature extraction unit, configured to select a region representing a user's hand in the first image data and extract gesture feature information within the region;

[0019] The recognition unit is used to obtain the current gesture based on the gesture feature information. When the current gesture matches the preset gesture type, the recognition unit generates a call instruction, and the communication module sends a video call request to the mobile terminal in response to the call instruction.

[0020] Furthermore, the recognition unit is used to perform curve fitting on the current gesture to obtain a polygonal approximation of the current gesture. The recognition unit determines the changing trend of the current gesture based on the number of edges and the changing trend of the edge number of the polygonal approximation, and determines the predicted gesture after the change based on the changing trend of the current gesture. When the predicted gesture matches the preset gesture type, the camera device executes any one of the following instructions including sending a video call request, hanging up, and rejecting the call.

[0021] Furthermore, the camera device further includes a synchronization module, and the camera device performs periodic clock synchronization with the cloud through the synchronization module.

[0022] Furthermore, the camera device obtains the timestamp recorded on the second video data and the timestamp recorded on the second audio data through the cloud;

[0023] The data processing module matches the timestamp of the second video data with the timestamp of the second audio data, and plays the second video data and the second audio data respectively through the display module and the audio playing module to synchronize the second video data and the second audio data.

[0024] In an embodiment of the present invention, the camera device plays the acquired audio and video data through the display module and the audio playback module, so that two-way information interaction can be carried out between the camera device and the mobile terminal, thereby improving the call experience of the user at the camera device end and ensuring the effectiveness of information transmission between the users at both ends. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] FIG1 is a schematic structural diagram of a camera device in an embodiment of the present application.

[0026] FIG2 is a schematic diagram of signal connection between a camera device and a mobile terminal in an embodiment of the present application.

[0027] FIG3 is a partial structural block diagram of a camera device in an embodiment of the present application.

[0028] FIG4 is a timing diagram of a video call of a camera device according to an embodiment of the present application.

[0029] FIG5 is a structural block diagram of a data processing module in an embodiment of the present application. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the specific implementation of the present application will be clearly and completely described below in conjunction with the drawings in the implementation of the present application.

[0031] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. "First", "second", "third", and "fourth" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first", "second", "third", and "fourth" may explicitly or implicitly include at least one of such features. The singular forms "a", "an", and "the" used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0032] As shown in Figures 1 and 2, the present application provides a camera device 100 that supports two-way video calling. The camera device 100 includes a camera module 11 and a sound acquisition module 12. The camera device 100 obtains first image data through the camera module 11, and the camera device 100 obtains first audio data through the sound acquisition module 12. The camera device 100 transmits data to a mobile terminal 300 via a cloud 200. The data transmitted includes at least the first image data and / or first audio data obtained by the camera device 100, so that the mobile terminal 300 plays the first image data and / or first audio data. The mobile terminal 300 can be a mobile device such as a mobile phone or a tablet computer.

[0033] For example, the camera device 100 is connected to a wireless network, and the mobile terminal 300 can send a control command such as wake-up or sleep to the camera device 100. After receiving the control command, the camera device 100 performs the corresponding operation. When the control command is a wake-up command, the camera device 100 begins to collect first image data and / or first audio data and uploads it to the cloud 200 via the wireless network. The user on the mobile terminal 300 can access the cloud 200 by connecting to the wireless network or using mobile data to view the first image data and / or first audio data collected by the camera device 100. When the control command is a sleep command, the camera device 100 stops working and enters a sleep state.

[0034] It should be noted that the mobile terminal 300 is used to collect the second image data and the second audio data. When a signal transmission channel is established between the camera device 100 and the mobile terminal 300, the mobile terminal 300 uploads the acquired second image data and the second audio data to the cloud 200 by connecting to a wireless network or using mobile data.

[0035] As an implementation, the camera device 100 includes a base 13 and a display module 14. The camera module 11 is rotatably mounted on the base 13. The display module 14 is fixedly mounted on the base 13 and is configured to display the second image data. This ensures that the display module 14 can stably display the second image data while the camera module 11 is capturing the first image data. The display module 14 may be a liquid crystal display.

[0036] Furthermore, the camera device 100 also includes a communication module 15, a data processing module 16, and an audio playback module 17. The communication module 15 can be a short-range wireless communication module 15 such as a WIFI module, a Bluetooth module, or a Zigbee module. The camera device 100 transmits data to the cloud 200 via the communication module 15.

[0037] The data processing module 16 is configured to encode the first audio data and the first image data collected by the camera device 100. When the camera device 100 conducts a video call with the mobile terminal 300 via the cloud 200, the encoding process of the data processing module 16 is as follows: the data processing module 16 adds a timestamp to the first audio data and the first image data, encodes the timestamp-added first audio data and the first image data, and transmits the encoded data to the cloud 200.

[0038] In an embodiment of the present application, the camera device 100 sends the device information of the camera device 100 to the cloud 200 through the communication module 15, and stores it in a communication list established in the cloud 200, so that the camera device 100 can send a video call request to any device in the communication list, wherein the communication list includes at least the device information of the camera device 100 and the device information of the mobile terminal 300.

[0039] Exemplarily, the camera device 100 sends a video call request to the cloud 200 through the communication module 15 , and the cloud 200 sends the video call request to the mobile terminal 300 in the communication list according to a pre-set communication list.

[0040] It should be noted that when the camera device 100 conducts a video call with the mobile terminal 300 via the cloud 200, the mobile terminal 300 collects the second audio data and the second image data and encodes the collected second audio data and the second image data. The encoding process performed by the mobile terminal 300 includes adding a timestamp to the second audio data and the second image data, as described above, so that the camera device 100 can synchronize the second audio data and the second image data after obtaining the second audio data and the second image data. The mobile terminal 300 uploads the encoded second audio data and the second image data to the cloud 200 via a wireless network or using mobile data.

[0041] Furthermore, the data processing module 16 is further configured to decode, synchronize, and render the second audio data and the second image data collected by the mobile terminal 300 .

[0042] In an embodiment of the present application, the data processing module 16 is respectively connected to the display module 14 and the audio playback module 17 for communication. The display module 14 receives the second image data after decoding, synchronization, and rendering, and performs corresponding display on its display interface.

[0043] The audio playback module 17 receives and plays the second audio data after being decoded, synchronized, and rendered. The audio playback module 17 may be a speaker integrated in the camera device 100 .

[0044] As shown in FIG3 , the camera device 100 further includes a synchronization module 18 , through which the camera device 100 periodically synchronizes its clock with the cloud 200 . This avoids data conflicts between the camera device 100 and the mobile terminal 300 , thereby ensuring data consistency between the camera device 100 and the mobile terminal 300 .

[0045] Exemplarily, the camera device 100 periodically receives a time signal sent by the cloud 200 through the communication module 15 . The synchronization module 18 may be a timer. The synchronization module 18 calibrates the current time according to the time signal to achieve clock synchronization between the camera device 100 and the cloud 200 .

[0046] Through the above settings, a video call between the camera device 100 and the mobile terminal 300 is realized. Compared with the one-way call between the mobile terminal 300 and the camera device 100 in the prior art, the camera device 100 provided in this application has higher information transmission efficiency, thereby improving the product usage experience of the user of the camera device 100.

[0047] The present application also provides another embodiment in which the camera device 100 is communicatively connected to any display device via the cloud 200, wherein the display device includes at least a central console, a fixed terminal, etc. The device information of the display device is stored in a communication list, so that the camera device 100 can send a video call request to the display device in the communication list via the cloud 200.

[0048] For example, the center console is often used to control multiple IoT devices in a home environment. The center console is generally equipped with a display screen and a speaker. When the center console receives a video call request sent by the camera device 100 through the cloud 200, the first image data obtained by the camera device 100 is displayed through the display screen of the center console, and the first audio data obtained by the camera device 100 is played through the speaker of the center console.

[0049] In some examples, the fixed terminal can be a computer, a television with IoT capabilities, etc.

[0050] Optionally, the camera device 100 sends a video call request to the cloud 200 via the MQTT (Message Queuing Telemetry Transport) protocol, so that at least one device in the communication list obtains the video call request sent by the cloud 200, thereby realizing information interaction between multiple devices.

[0051] It should be noted that the MQTT protocol is a low-overhead, low-bandwidth instant messaging protocol built on the TCP / IP protocol stack. It can provide real-time and reliable messaging services for remote device connections with less code and bandwidth.

[0052] The present application also provides another embodiment in which at least two camera devices 100 are communicatively connected via a cloud 200 , and the at least two camera devices 100 can send collected audio and video data to each other;

[0053] As shown in Figure 4, at least two camera devices 100 include a sending end 101 and at least one receiving end 102. The sending end 101 sends a video call request to the cloud 200 through the MQTT protocol. The cloud sends the video call request to the receiving end 102 and obtains feedback information sent by the receiving end 102. The feedback information includes at least one of the connect, wait, reject, and hang up instructions sent by the receiving end 102.

[0054] For example, after initiating a video call request, the sending end 101 can cancel the call, thereby stopping the cloud 200 from sending the video call request to the receiving end 102 .

[0055] When the receiving end 102 is making a video call with another device, such as the mobile terminal 300 , a display device, or any other device of the receiving end 102 , the cloud 200 feeds back to the sending end 101 that the call is busy.

[0056] When the sending end 101 initiates a video call request and the waiting time exceeds a preset time threshold, the cloud 200 feeds back a call timeout to the sending end 101, and at the same time, the sending end 101 obtains a no-answer reply sent by the receiving end 102 to the cloud 200.

[0057] The sending end 101 can obtain the ringing response fed back by the receiving end 102 through the cloud 200, so that the sending end 101 can determine that the receiving end 102 is not busy based on the feedback information.

[0058] When the receiving end 102 responds to the video call request, the feedback information includes an answering reply sent by the receiving end 102 , and the sending end 101 establishes a signal transmission channel with the receiving end 102 in response to the answering reply.

[0059] The above is the sending end 101 obtaining the feedback information sent by the receiving end 102 or the cloud 200, and performing corresponding actions based on the feedback information. It will not be repeated here. Any feedback information that can reflect the current status of the sending end 101 and the receiving end 102 is within the scope of protection required by this application.

[0060] Through the above settings, in addition to the aforementioned mobile terminal 300 and display device, multi-terminal information interaction can also be achieved through multiple camera devices 100, and the efficiency of information transmission during video calls can be guaranteed, thereby improving the effectiveness and integrity of information transmission during multi-terminal information interaction.

[0061] Since the overall structure of camera products on the market is relatively compact, especially smart home camera products need to meet market requirements and realize miniaturization. The camera device 100 provided in this application has advantages such as small structure and easy installation. However, after the display module 14 is integrated into the camera device 100, it is more difficult for the user to perform functional operations on the camera device 100.

[0062] As an implementation, the data processing module 16 further includes a feature extraction unit 161 and a recognition unit 162. Feature extraction unit 161 is configured to select an area representing the user's hand in the first image data and extract gesture feature information within the selected area. Recognition unit 162 obtains the current gesture based on the gesture feature information. If the current gesture matches a preset gesture type, recognition unit 162 generates a call instruction. In response to the call instruction, communication module 15 transmits a video call request to mobile terminal 300 via cloud 200.

[0063] Specifically, the recognition unit 162 is used to perform curve fitting on the current gesture to obtain a polygonal approximation of the current gesture. The recognition unit 162 determines the changing trend of the current gesture based on the number of edges and the changing trend of the edge number of the polygonal approximation, and determines the predicted gesture after the change based on the changing trend of the current gesture. When the predicted gesture matches the preset gesture type, the camera device 100 executes any one of the operations including sending a video call request, hanging up, and rejecting.

[0064] It should be noted that the number of fingers can be determined according to the number of sides of the polygonal approximation graph and the length of each side.

[0065] For example, the changing trend of the current gesture is mainly realized based on the Camshift algorithm, which comprehensively utilizes the color, area and contour features of the gesture image to determine the outer rectangle of the current gesture.

[0066] In summary, the present application utilizes a camera device 100 integrated with a display module 14 to enable the camera device 100 to conduct a two-way voice and video call with a peer device. Furthermore, the camera device 100 can also display the second audio data and second image data acquired by a mobile terminal 300, a display device, or other devices, thereby improving the call experience of the user at the camera device 100. Furthermore, the camera device 100 capable of two-way voice and video calls has better information transmission efficiency.

[0067] It should be understood that those skilled in the art can make improvements or changes based on the above description, and all such improvements and changes should fall within the scope of protection of the claims appended to this application.

Claims

1. A camera device supporting two-way video calls, the camera device being used to obtain first audio data and first image data, and to transmit data with a mobile terminal through a cloud, so that the mobile terminal receives and plays the first audio data and the first image data, characterized in that: The camera device comprises: A communication module, used for information exchange between the camera device and the cloud, wherein the camera device obtains the second audio data and the second image data uploaded to the cloud by the mobile terminal through the communication module; a data processing module, configured to perform data processing on the acquired second audio data and the second image data, wherein the data processing includes at least one of decoding, synchronizing, and rendering the second audio data and the second image data; A display module, used for acquiring and displaying the processed second image data; The audio playing module is used to obtain and play the processed second audio data.

2. The camera device according to claim 1, characterized in that: The camera device includes a base and a camera module that can rotate relative to the base, the camera module is used to obtain the first image data, and the display module is fixedly arranged on the base.

3. The camera device according to claim 1, characterized in that: The camera device sends the device information to the cloud through the communication module and stores it in a communication list established in the cloud, so that the camera device can send a video call request to any device in the communication list, wherein the communication list includes at least the device information of the camera device and the device information of the mobile terminal.

4. The camera device according to claim 3, characterized in that: The camera device is communicatively connected to any display device via the cloud, wherein the display device includes at least one of a central console and a fixed terminal, and the device information of the display device is stored in the communication list, so that the camera device sends a video call request to the display device via the cloud.

5. The camera device according to claim 4, characterized in that: The camera device sends a video call request to the cloud through the MQTT protocol, so that at least one device in the communication list obtains the video call request sent by the cloud.

6. The camera device according to claim 1, characterized in that: At least two of the camera devices are connected via the cloud communication, and at least two of the camera devices can send collected audio and video data to each other; At least two of the camera devices include a transmitting end and at least one receiving end. The transmitting end sends a video call request to the cloud through the MQTT protocol and receives feedback information sent by the cloud. The feedback information includes at least one of the connect, wait, reject, and hang up instructions sent by the receiving end.

7. The camera device according to claim 1, characterized in that: The data processing module comprises: a feature extraction unit, configured to select a region representing a user's hand in the first image data, and extract gesture feature information in the region; The recognition unit is used to obtain the current gesture according to the gesture feature information. When the current gesture matches the preset gesture type, the recognition unit generates a call instruction, and the communication module sends a video call request to the mobile terminal in response to the call instruction.

8. The camera device according to claim 7, characterized in that: The recognition unit is used to perform curve fitting on the current gesture to obtain a polygonal approximation of the current gesture. The recognition unit determines a changing trend of the current gesture based on the number of edges of the polygonal approximation and the changing trend of the number of edges, and determines a predicted gesture after the change based on the changing trend of the current gesture. When the predicted gesture matches the preset gesture type, the camera device executes any one of the operations including sending a video call request, hanging up, and rejecting the call.

9. The camera device according to claim 1, characterized in that: The camera device further comprises a synchronization module, and the camera device performs periodic clock synchronization with the cloud through the synchronization module.

10. The camera device according to claim 1, characterized in that: The camera device obtains, through the cloud, a timestamp recorded on the second video data and a timestamp recorded on the second audio data; The data processing module combines the timestamp of the second video data with the timestamp of the second audio data The timestamp corresponds to that of the video data, and the second video data and the second audio data are played respectively through the display module and the audio playing module to synchronize the second video data with the second audio data.

Images