Data stream pushing method, system and device and electronic equipment

Abstract

The embodiment of the invention provides a data flow pushing method, system and device and electronic equipment. In the embodiment of the invention, when the transmitting end needs to simultaneously push the preview frame and the video frame to the receiving end, the transmitting end preferentially pushes the preview frame, detects whether the available bandwidth for transmitting the video frame exists at present, and only pushes the video frame under the condition that the available bandwidth exists, so that the condition that the transmission of the preview frame is interrupted due to insufficient bandwidth can be avoided.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to a data stream pushing method, system, device and electronic equipment. BACKGROUND

[0002] The intelligent doorbell set generally includes a visual doorbell and a smart screen. The so-called visual doorbell (referred to as doorbell) is a doorbell with a camera, which can record the video when the doorbell is pressed, and can also view the situation at the door through the obtained real-time stream. In a specific application, the visual doorbell is configured outside the door, and the smart screen is configured inside the door. When the doorbell is started, two data streams will be pushed to the smart screen, one of which is a preview stream for previewing the situation at the door, and the other is a video stream. However, since the two data streams share the same wireless bandwidth during transmission, there may be a situation that part of the preview frames are interrupted due to insufficient bandwidth, resulting in a situation that the display content on the smart screen is stuck, blooming, etc. SUMMARY

[0003] Therefore, the present application provides a data stream pushing method, system, device and electronic equipment to avoid transmission interruption of preview frames.

[0004] The present application provides a data stream transmission method, which is applied to a sending end, and the method comprises the following steps:

[0005] Pushing a currently generated preview frame to a receiving end;

[0006] When each preview frame is pushed to the receiving end, if it is found that a set condition is met between the sending end and the receiving end, checking whether a current video frame to be pushed matches one of the preview frames that have been pushed; wherein the set condition refers to the fact that there is currently available bandwidth for pushing the video frame;

[0007] When the current video frame to be pushed matches one of the preview frames that have been pushed, suppressing the pushing of the current video frame, and when there is still a next video frame to be pushed, taking the next video frame as the current video frame, and returning to the step of checking whether the current video frame to be pushed matches one of the preview frames that have been pushed;

[0008] When the current video frame to be pushed does not match any of the preview frames that have been pushed, pushing the current video frame.

[0009] The present application also provides a data stream transmission method, which is applied to a receiving end, and the method comprises the following steps:

[0010] When a preview frame pushed by the sending end is received, caching the currently received preview frame in a preconfigured first buffer;

[0011] When receiving the video frame pushed by the sending end, the currently received video frame is cached into a pre-configured second buffer;

[0012] If it is determined that the video frame cached in the second buffer and the preview frame cached in the first buffer meet preset requirements, recording writing based on the video frame cached in the second buffer and the preview frame cached in the first buffer is triggered; the preset requirements refer to that the video frame cached in the second buffer and the preview frame cached in the first buffer are continuous in time stamps.

[0013] The embodiment of the application further provides a data stream transmission system, which comprises a sending end and a receiving end;

[0014] The sending end performs according to the first data stream transmission method provided above;

[0015] The receiving end performs according to the second data stream transmission method provided above.

[0016] The embodiment of the application further provides a data stream transmission device, which is configured in the sending end, and the device comprises:

[0017] A first pushing module is configured to push a currently generated preview frame to the receiving end;

[0018] A checking module is configured to, when pushing each preview frame to the receiving end, check whether a current video frame to be pushed matches one of the preview frames that have been pushed, if it is found that a set condition between the sending end and the receiving end is met; the set condition refers to that there is currently available bandwidth for pushing the video frame;

[0019] A suppressing module is configured to, when the current video frame to be pushed matches one of the preview frames that have been pushed, suppress pushing the current video frame, and when there is a next video frame to be pushed, take the next video frame as the current video frame, and return to the step of checking whether the current video frame to be pushed matches one of the preview frames that have been pushed;

[0020] A second pushing module is configured to, when the current video frame to be pushed does not match any of the preview frames that have been pushed, push the current video frame.

[0021] The embodiment of the application further provides another data stream transmission device, which is configured in the receiving end, and the device comprises:

[0022] A first caching module is configured to, when receiving the preview frame pushed by the sending end, cache the currently received preview frame into a pre-configured first buffer;

[0023] A second caching module is configured to, when receiving the video frame pushed by the sending end, cache the currently received video frame into a pre-configured second buffer;

[0024] The writing module is configured to trigger writing of the video based on the video frame stored in the second buffer and the preview frame stored in the first buffer if it is determined that the video frame stored in the second buffer and the preview frame stored in the first buffer meet preset requirements; the preset requirements refer to that the video frame stored in the second buffer and the preview frame stored in the first buffer are continuous in time stamp.

[0025] The embodiments of the present application also provide an electronic device, comprising:

[0026] The processor and a memory for storing computer program instructions, the computer program instructions, when executed by the processor, cause the processor to perform the steps of the above method.

[0027] The embodiments of the present application also provide a computer readable storage medium, the storage medium stores computer program instructions, when the computer program instructions are executed, the steps of the above method can be implemented.

[0028] From the above technical solutions, in the embodiment, when the sending end needs to push the preview frame and the video frame to the receiving end at the same time, the sending end will preferentially push the preview frame, and detect whether there is available bandwidth for transmitting the video frame, and only in the case that there is available bandwidth, the video frame is pushed, so that the situation that the transmission of the preview frame is interrupted due to insufficient bandwidth can be avoided;

[0029] Further, when the video frame is pushed, the embodiment also checks whether the current video frame to be pushed matches one of the preview frames that have been pushed, wherein if the current video frame matches one of the preview frames that have been pushed, it means that the current video frame and one of the preview frames that have been pushed are repeated in content, and on this premise, the embodiment only pushes the current video frame when the above does not match, so that the repeated pushing of the data frame with the same content can be avoided, the transmission of redundant data is effectively reduced, and the bandwidth utilization is improved. BRIEF DESCRIPTION OF DRAWINGS

[0030] The accompanying drawings, which are incorporated herein and form part of the specification, illustrate embodiments consistent with the present application and, together with the description, further serve to explain the principles of the application.

[0031] Figure 1 A flowchart of a data stream transmission method provided by the embodiments of the present application.

[0032] Figure 2 A flowchart of another data stream transmission method provided by the embodiments of the present application.

[0033] Figure 3 An implementation schematic diagram of a doorbell kit provided by the embodiments of the present application.

[0034] Figure 4An implementation schematic diagram of a video recording stream and a preview stream provided by an embodiment of the present application.

[0035] Figure 5 An implementation schematic diagram of a data stream transmission method provided by an embodiment of the present application.

[0036] Figure 6 An implementation schematic diagram of a data stream transmission system provided by an embodiment of the present application.

[0037] Figure 7 An implementation schematic diagram of a data stream transmission device provided by an embodiment of the present application.

[0038] Figure 8 An implementation schematic diagram of another data stream transmission device provided by an embodiment of the present application.

[0039] Figure 9 An implementation schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0040] To facilitate understanding of the technical solutions of the embodiments of the present application, the technical concepts related to the embodiments of the present application are explained below.

[0041] Preview: Obtain the real-time stream of the camera through the protocol.

[0042] Video recording: Obtain the video recording stream of the camera through the protocol, generally record when the event occurs, and the video recording may have a pre-recording requirement, such as a pre-recording time of 4-6 seconds.

[0043] Camera doorbell: that is, a video doorbell, which can be placed at the door and has a camera, which can record the video when the bell is pressed, and can also view the situation at the door through the real-time stream.

[0044] Smart screen: used with the camera doorbell, which can be placed indoors and ring when the bell is pressed to remind people inside; it also provides WIFI, intercom, video recording saving and other functions.

[0045] In order for those skilled in the art to better understand the technical solutions provided by the embodiments of the present application, and to make the above-mentioned purposes, features and advantages of the embodiments of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present application will be further described in detail below with reference to the drawings.

[0046] Reference Figure 1 , Figure 1A flowchart of a data stream transmission method provided by an embodiment of the present application. The flowchart describes the data stream transmission between a sending end and a receiving end from the perspective of the sending end. The forms of the sending end and the receiving end are not specifically limited herein. For example, as an embodiment, the sending end and the receiving end can belong to the same doorbell set. The sending end is a doorbell deployed at a door, and the receiving end is a display screen deployed indoors.

[0047] As shown in Figure 1 , the flowchart can include the following steps:

[0048] S101, push a currently generated preview frame to the receiving end.

[0049] In this embodiment, when the sending end receives an external preview instruction, it is triggered to generate a preview frame and push the currently generated preview frame to the receiving end, so as to ensure the real-time of the preview. Here, the preview frame is generated by the camera module (such as a camera) deployed by the sending end when receiving the external preview instruction.

[0050] Optionally, as an embodiment, the received external preview instruction has multiple generation methods. Here, taking the sending end as a doorbell deployed at a door and the receiving end as a display screen deployed indoors as an example, the user can trigger the generation of the preview instruction through the display screen (for example, the user can trigger the generation of the preview instruction through the preview button on the display screen). For example, the user can trigger the generation of the preview instruction through an associated application program in communication with the sending end (for example, the user can trigger the generation of the preview instruction through the preview button on the application program interface).

[0051] As an embodiment, the sending end will stop pushing the preview frame to the receiving end after receiving an external preview stop instruction. Here, the generation method of the preview stop instruction is similar to that of the preview instruction, and the related description of the generation method of the preview instruction is referred to herein, which will not be repeated.

[0052] S102, when pushing each preview frame to the receiving end, if it is found that the sending end and the receiving end meet the set condition, check whether the current video frame to be pushed matches one of the pushed preview frames.

[0053] In this embodiment, the set condition refers to the current available bandwidth for pushing the video frame. Here, how to determine that the sending end and the receiving end meet the set condition is not specifically limited. For example, as an embodiment, when the sending end pushes each preview frame to the receiving end, if it is found that the bandwidth required for pushing the preview frame is less than the wireless bandwidth between the sending end and the receiving end for pushing the data frame, it can be considered that there is remaining bandwidth available for the transmission of other data frames (such as video frames), that is, there is currently available bandwidth for pushing the video frame, that is, the sending end and the receiving end meet the set condition.

[0054] For example, assuming that the total bandwidth currently available is 200K, and the bandwidth required for transmitting each preview frame is 100K, then there is 100K of bandwidth remaining for transmitting video frames. If the bandwidth required for transmitting each preview frame is 200K or more, then there is no bandwidth remaining for transmitting video frames.

[0055] Optionally, as an embodiment, the sending end triggers the deployed camera module to generate video frames upon detecting a video start event, until a video stop condition is reached. As an embodiment, one of the video start events is that the doorbell is triggered to start. For example, someone presses the doorbell to trigger the doorbell to start, or the user initiates preview through the display screen or the associated application to trigger the doorbell to start, etc. As another embodiment, one of the video start events is that the deployed sensor module (such as a radar sensor, etc.) of the doorbell recognizes a target object. Here, the target object can refer to a person, an object, etc.

[0056] As an embodiment, the above-mentioned video stop condition can be flexibly configured according to actual needs. For example, the video stop condition can be that the video time reaches a set time, which can be 1 minute, 2 minutes, etc.

[0057] In this embodiment, both the preview frames and the video frames are generated by the camera module deployed by the sending end. In actual applications, there can be a case where a data frame generated at a certain moment can be used as both a preview frame and a video frame, that is, the preview frame and the video frame generated at the same moment can be considered as the same data frame. In order to reduce the transmission of redundant data and improve the utilization of bandwidth, the repeated data frames do not need to be pushed repeatedly. Based on this, when each preview frame is pushed to the receiving end, if it is found that the set condition is met between the sending end and the receiving end, and there is at least one video frame to be pushed, it can be checked whether the current video frame to be pushed matches one of the pushed preview frames, so as to determine whether the current video frame needs to be pushed according to the matching result, thereby avoiding the repeated pushing of the pushed data frame. As for how to determine whether the current video frame needs to be pushed according to the matching result, there can be various implementation manners, which will be described by way of example in the following, and will not be described here in detail.

[0058] As an embodiment, the above pushing the current generated preview frame to the receiving end further comprises: recording frame information of the pushed preview frame for subsequent matching with the to-be-pushed video frame. The specific content of the frame information is not limited here, and can be used as an identifier representing the preview frame, such as at least including a timestamp. On this basis, as an embodiment, there are many specific implementation manners for checking whether the current to-be-pushed video frame matches one of the pushed preview frames, such as checking whether the frame information of the current to-be-pushed video frame matches the frame information of one of the pushed preview frames; since the frame information is an identifier representing a data frame, if the frame information of the current to-be-pushed video frame matches the frame information of one of the pushed preview frames, it can be determined that the current to-be-pushed video frame matches one of the pushed preview frames; if the frame information of the current to-be-pushed video frame does not match the frame information of one of the pushed preview frames, it can be determined that the current to-be-pushed video frame does not match any of the pushed preview frames.

[0059] As an embodiment, in the case where the frame information at least includes a timestamp, it can be checked whether the difference between the timestamp of the current to-be-pushed video frame and the timestamp of one of the pushed preview frames is within a set time difference range; if the difference is within the set time difference range, it can be indicated that the current to-be-pushed video frame and one of the pushed preview frames are the same data frame, and it can be determined that the current to-be-pushed video frame matches one of the pushed preview frames; if the difference is not within the set time difference range, it can be indicated that the current to-be-pushed video frame and one of the pushed preview frames are not the same data frame, and it can be determined that the current to-be-pushed video frame does not match one of the pushed preview frames. Here, the set time difference range can be a range that is flexibly set according to actual needs in advance.

[0060] For example, assuming that the timestamp is the time point at which the data frame is generated, since there will be a certain error in the generation of the timestamp, in order to avoid the influence of this error on the subsequent timestamp matching judgment, a set time difference range can be pre-configured, such as the set time difference range can be between 0ms and 2ms (i.e. greater than or equal to 0ms and less than or equal to 2ms); if the timestamp of the current to-be-pushed video frame is 1 minute 66ms and one of the pushed preview frames is 1 minute 67ms, the difference between the timestamp of the current to-be-pushed video frame and the timestamp of one of the pushed preview frames is 1ms, and the difference is within the set time difference range, so it can be indicated that the current to-be-pushed video frame matches one of the pushed preview frames.

[0061] S103, when the current video frame to be pushed matches one of the pushed preview frames, suppressing pushing the current video frame, and when there is a next video frame to be pushed, taking the next video frame as the current video frame, and returning to the step of checking whether the current video frame to be pushed matches one of the pushed preview frames.

[0062] In the embodiment, when each preview frame is pushed to the receiving end, if it is found that there is available bandwidth between the sending end and the receiving end for pushing the video frame, it can be checked whether the current video frame to be pushed matches one of the pushed preview frames; if yes, it means that the same preview frame as the current video frame has been pushed, at this time, in order to avoid transmission of redundant data and improve bandwidth utilization, the current video frame does not need to be pushed repeatedly, i.e., pushing the current video frame is suppressed; if it is found that there is a next video frame to be pushed at this time, the next video frame can be taken as the current video frame, and the step of checking whether the current video frame to be pushed matches one of the pushed preview frames is returned, so as to continue to judge whether the next video frame needs to be pushed.

[0063] S104, when the current video frame to be pushed does not match the pushed preview frames, pushing the current video frame.

[0064] In the embodiment, when the current video frame to be pushed does not match the pushed preview frames, it means that the current video frame has not been pushed, and thus the current video frame can be pushed to the receiving end.

[0065] Thus, the process shown in Figure 1 is completed.

[0066] As can be seen from the process shown in Figure 1 , in the embodiment, when each preview frame is pushed to the receiving end, the video frame to be pushed is pushed only when there is available bandwidth between the sending end and the receiving end, which can ensure real-time pushing of the preview stream and also ensure priority pushing of the preview frame, so as to avoid the case that transmission of part of the preview frames of the preview stream is interrupted due to insufficient bandwidth; and by using the available bandwidth remaining in transmission of the preview frame to transmit the video frame to be pushed, the utilization of the bandwidth can be effectively improved. Further, in the embodiment, when the available bandwidth for pushing the video frame exists, whether the current video frame to be pushed matches one of the pushed preview frames is checked, and if not, the current video frame is pushed, which can avoid repeated pushing of the repeated frame, thereby effectively reducing transmission of redundant data and improving the utilization of the bandwidth.

[0067] The data stream transmission method provided by the embodiment is described below from the perspective of the receiving end:

[0068] Referring to Figure 2 ,Figure 2 Another flowchart of a data stream transmission method is provided in the embodiments of the present application. As shown in the flowchart, the method can include the following steps: Figure 2

[0069] S201, when receiving a preview frame pushed by the sending end, cache the currently received preview frame in a preconfigured first buffer.

[0070] In the embodiments, when the receiving end receives a preview frame pushed by the sending end, the currently received preview frame can be cached in a preconfigured first buffer. Optionally, as an embodiment, after the currently received preview frame is cached in the preconfigured first buffer, the currently received preview frame can be read from the first buffer and displayed.

[0071] S202, when receiving a recording frame pushed by the sending end, cache the currently received recording frame in a preconfigured second buffer.

[0072] In the embodiments, when the receiving end receives a recording frame pushed by the sending end, the currently received recording frame can be cached in a preconfigured second buffer for subsequent recording writing in combination with the preview frame in the first buffer.

[0073] S203, if it is determined that the recording frame cached in the second buffer and the preview frame cached in the first buffer meet a preset requirement, trigger recording writing based on the recording frame cached in the second buffer and the preview frame cached in the first buffer.

[0074] In the embodiments, the preset requirement refers to that the recording frame cached in the second buffer and the preview frame cached in the first buffer are continuous in the timestamp. If it is determined that the recording frame cached in the second buffer and the preview frame cached in the first buffer meet the preset requirement, it indicates that the recording frame cached in the second buffer and the preview frame cached in the first buffer are continuous in the timestamp, that is, a data stream is formed. At this time, recording writing can be triggered based on the recording frame cached in the second buffer and the preview frame cached in the first buffer, such as writing to a designated storage medium (such as a memory card of a peripheral storage medium) to form part of a data stream of a recording stream.

[0075] ​Optionally, as an embodiment, there are many specific implementation manners for determining that the recorded video frames cached in the second buffer and the preview frames cached in the first buffer meet the preset requirement, for example, the recorded video frames cached in the second buffer and the preview frames cached in the first buffer are sorted according to the time stamps in chronological order; if the difference between the time stamps of any two adjacent data frames in the sorted data frames is a set value, it is determined that the recorded video frames cached in the second buffer and the preview frames cached in the first buffer meet the preset requirement; otherwise, it is determined that the recorded video frames cached in the second buffer and the preview frames cached in the first buffer do not meet the preset requirement.

[0076] Here, the set value can be determined according to the generation interval time of the two adjacent data frames, for example, if the generation interval time of the two adjacent data frames is 66 ms, the set value can be set to 66 ms. It should be noted that the sending end generates data frames (such as preview frames or recorded video frames) at a preconfigured set frame rate, so the generation interval time of any two adjacent data frames in the generated data frames is the same, that is, the difference between the time stamps of any two adjacent data frames is the same. Based on this, if the difference between the time stamps of any two adjacent data frames in the sorted data frames is a set value, it can be determined that the recorded video frames cached in the second buffer and the preview frames cached in the first buffer are continuous in time stamps, that is, meet the preset requirement.

[0077] For example, taking the time stamps 1, 2, 3, 4, etc. as an example, assuming that the set value is 1, if the time stamps of the preview frame 1, the preview frame 2 and the preview frame 3 cached in the first buffer are 3, 4 and 5 respectively, and the time stamps of the recorded video frame 1 and the recorded video frame 2 cached in the second buffer are 1 and 2 respectively, the order of the sorted data frames is recorded video frame 1, recorded video frame 2, preview frame 1, preview frame 2, preview frame 3, and it can be seen that the difference between the time stamps of any two adjacent data frames in the sorted data frames is 1, which indicates that the recorded video frames cached in the second buffer and the preview frames cached in the first buffer are continuous in time stamps. If the time stamps of the preview frame 1, the preview frame 2 and the preview frame 3 cached in the first buffer are 3, 4 and 5 respectively, and the time stamp of the recorded video frame 1 cached in the second buffer is 1, the order of the sorted data frames is recorded video frame 1, preview frame 1, preview frame 2, preview frame 3, and the difference between the time stamp of the recorded video frame 1 and the time stamp of the preview frame 1 is 2, which indicates that the recorded video frames cached in the second buffer and the preview frames cached in the first buffer are not continuous in time stamps.

[0078] Optionally, as an embodiment, the specific implementation of the above-mentioned triggering the recording writing based on the recording frames cached in the second buffer and the preview frames cached in the first buffer has many ways, for example, the recording frames cached in the second buffer and the preview frames cached in the first buffer can be sequentially written to the specified storage medium according to the time stamp order. As another embodiment, after the recording writing based on the recording frames cached in the second buffer and the preview frames cached in the first buffer, if it is found that there are new data frames in the first buffer and / or the second buffer, the new data frames are written to the specified storage medium.

[0079] In the embodiment, as an embodiment, after the preview and recording end, if it is found that the data frames cached in the first buffer and the second buffer have been all written to the specified storage medium, the contents cached in the first buffer and the second buffer can be deleted to release the space of the first buffer and the second buffer for next use.

[0080] At this point, the completion Figure 2 flow shown in the figure.

[0081] As can be seen from the Figure 2 flow shown in the figure, the embodiment can constitute a complete recording stream by using the preview frames cached in the first buffer as the recording frames of the sending end which are suppressed to be pushed, and then when the recording frames cached in the second buffer and the preview frames cached in the first buffer are continuous in the time stamp, the recording writing of the recording frames cached in the second buffer and the preview frames cached in the first buffer is performed.

[0082] In order to facilitate the understanding of the specific implementation process of the above-mentioned data stream transmission method, the following will be described by specific embodiments.

[0083] In the embodiment of the present application, referring to Figure 3As shown, the doorbell kit includes a visual doorbell (i.e., a sending end) and a smart screen (i.e., a receiving end), the visual doorbell is deployed at the door, and the smart screen (i.e., a display screen) is deployed indoors. When someone presses the button of the doorbell, the smart screen starts ringing, and the doorbell pushes the generated video stream to the smart screen, and the smart screen saves the video stream in the TF card (i.e., a peripheral memory card) of the smart screen after receiving the video stream. In one practical application scenario, after someone presses the doorbell, the doorbell pushes a message (i.e., a prompt message indicating that someone presses the doorbell) to the associated smart screen and application (APP). After receiving a preview instruction, the doorbell pushes a preview stream to the smart screen, and the screen of the smart screen can display the picture content at this time. In this scenario, the doorbell needs to push two streams (i.e., a preview stream and a video stream) to the smart screen at the same time, one for preview and one for video recording. The two streams are essentially PS streams (PS stream usually refers to one of the video encoding formats, which may refer to MPEG-2 PS (Program Stream) or MPEG-4 Part2 video stream format), so there is a possibility of partial data stream duplication. Among them, the services (i.e., video recording service and preview service) are different, and the requirements for the two are also different; the preview stream requires real-time, while the video stream generally requires the content of the video to be saved for subsequent playback. For example, if video recording and preview are started at the same time, the preview pushes the data stream after the current time, while the video recording starts pushing the stream after [current time - pre-recording time] (i.e., starts pushing the pre-recording stream). Since the preview is real-time and has a talkback function, the priority of the preview stream needs to be guaranteed, i.e., the preview stream is pushed first; while the video recording stream does not have such a high real-time requirement, and can be transmitted slowly, such as having a lower priority than the preview stream, to avoid occupying the bandwidth required for the transmission of the preview stream.

[0084] Referring to Figure 4 As shown, it can be seen that the video stream overlaps with the preview stream except for the shadow part (i.e., the pre-recording stream part) on the left side of the dashed line. The overlapping part of the preview stream has been pushed, so in this embodiment, the part of the video stream that overlaps with the preview stream can not need to be pushed repeatedly, which can reduce redundant data transmission and maximize the utilization of bandwidth. Generally, the preview stream is transmitted at a speed according to the code stream generation speed, such as 15 frames per second. When the video is sent, the time has accumulated for several seconds (i.e., the pre-recording part), but it is not necessary to transmit all at once, and it can be transmitted at a speed of 15 frames per second or even lower, as long as the bandwidth required for the transmission of the preview stream is not occupied.

[0085] In this embodiment, the data stream transmission is exemplarily described through the following three scenarios:

[0086] Scenario one, someone rings the doorbell: at this time, video recording must be triggered, so the generated video frames can be pushed to the smart screen from the doorbell activation. After the call message (i.e. the message indicating that someone rings the doorbell) is sent to the smart screen, the smart screen obtains the stream from the doorbell based on external instructions (such as preview instructions triggered by the user through the preview button on the smart screen). In order to respond more timely, I-frames (i.e. key frames) are actively inserted in the data stream obtained by the doorbell, and then when the preview frames and video frames are pushed, the preview frames are pushed with high priority. If there is available bandwidth for pushing video frames between the sending end and the receiving end when the preview frames are pushed, it is determined whether the current video frame to be pushed matches one of the already pushed preview frames. If they match, the current video frame is suppressed from being pushed, and when there is a next video frame to be pushed, the next video frame is taken as the current video frame for further determination. If the current video frame to be pushed does not match one of the already pushed preview frames, the current video frame is pushed. In this way, the part of the video frames in the video stream that overlaps with the already pushed preview frames does not need to be pushed repeatedly, thereby reducing the transmission of redundant data and improving the bandwidth utilization.

[0087] Scenario two, the user actively previews the device (i.e. the doorbell) through the screen or the APP, and the device pushes the currently generated preview frames to the smart screen. When the device is actively previewed, the sensing module configured on the doorbell will trigger video recording to generate video frames if it detects a target object (such as a person, an object, etc.), and will not trigger video recording if it does not detect a target object. In the above process, the part of the video frames in the video stream that overlaps with the already pushed preview frames does not need to be pushed repeatedly. Only when the preview ends but the video does not end, the generated video frames are continuously pushed to the smart screen until the video ends.

[0088] Scenario three, someone lingers at the door, and the sensing module of the doorbell recognizes that someone is at the door. At this time, the doorbell is awakened, triggering the start of video recording to generate video frames, and the generated video frames are pushed to the smart screen. When the user performs preview stream acquisition, the preview is triggered to generate preview frames, and the currently generated preview frames are pushed to the smart screen. The implementation process of pushing the preview frames to the smart screen can be referred to the related content described in the above embodiments, which will not be repeated here. In this embodiment, the preview frames are pushed first, and the video frames can be sent with appropriate delay to avoid affecting the real-time pushing of the preview frames due to the video frames occupying too much bandwidth.

[0089] From the above three scenarios, it can be seen that the pushing of the data stream mainly has two cases of preview and recording, the preview needs real-time response, and the recording decides whether to push according to the preview pushing situation. The code stream is organized in the form of frames, the doorbell locally maintains a list of sent preview frames, the preview stream has high priority, the preview frame is sent first, and the non-preview recording frame can be sent slowly, and the part of the recording frame that is repeated with the sent preview frame in the recording stream does not need to be pushed. The smart screen at the receiving end needs to reorganize the recording stream according to the received data frame through the timestamp method (for how to reorganize, please refer to the related content described in the above method embodiment). The preview stream itself is linear and does not need to be reorganized. For the reorganized part of the recording stream, the smart screen can save it first, and then start writing the recording stream into the TF card of the smart screen after the recording stream is collected.

[0090] Referring to Figure 5 in the actual application process of data stream pushing:

[0091] 1) Each data frame corresponds to a timestamp (i.e., the generated data frame carries a timestamp), assuming 15 frames per second, then the difference between frames is 1000 / 15=66ms, frames with the same timestamp may be of different types, such as video frames and audio frames; different numbers in the figure represent data frames with different timestamps;

[0092] 2) The doorbell also pushes two streams (i.e., recording stream and preview stream) at the same time, and the smart screen is pre-configured with two buffer areas (i.e., recording temporary buffer area and preview temporary buffer area), when it is found that the recording frames in the recording temporary buffer area and the preview frames in the preview temporary buffer area are continuous in the timestamp, it can be indicated that the recording stream has been continuous, at this time, the recording writing based on the recording frames in the recording temporary buffer area and the preview frames in the preview temporary buffer area can be triggered, such as the recording frames in the recording temporary buffer area and the preview frames in the preview temporary buffer area can be sequentially written into the TF card according to the order of the timestamp;

[0093] 3) In the process of pushing the recording frame, it is judged whether the recording frame needs to be pushed (i.e., whether the recording frame matches an already recorded and pushed preview frame), if not, it is skipped; otherwise, the recording frame is pushed to the smart screen;

[0094] 4) After receiving the preview frame, the receiving end first caches the preview frame in the preview temporary buffer area, then reads the preview frame from the preview temporary buffer area and displays it on the screen of the smart screen to prompt the user.

[0095] The embodiment gives priority to pushing the preview stream to ensure the real-time performance of the preview stream, and the video stream is sent slowly, without occupying the bandwidth required for sending the preview stream, so that the maximum shared code stream can be achieved through the out-of-order sending of frames. Whether any video frame in the video stream needs to be pushed is determined by the recorded sent preview frame, so that repeated pushing of repeated frames can be avoided, thereby reducing the transmission of redundant data and improving the utilization of bandwidth.

[0096] So far, the method provided by the embodiment of the application has been described, and the system and device provided by the embodiment of the application will be described below.

[0097] Referring to Figure 6 , Figure 6 FIG. 1 is a structural schematic diagram of a data stream transmission system provided by an embodiment of the application. As shown in Figure 6 , the data stream transmission system 600 includes a sending end 601 and a receiving end 602;

[0098] The sending end 601 performs the method as described in Figure 1 .

[0099] The receiving end 602 performs the method as described in Figure 2 .

[0100] As an embodiment, the embodiment also provides a data stream transmission device. For example, referring to Figure 7 , Figure 7 FIG. 2 is a structural schematic diagram of a data stream transmission device provided by an embodiment of the application. The device corresponds to the process as shown in Figure 1 . As shown in Figure 7 , the data stream transmission device 700 is configured in the sending end, and the data stream transmission device 700 includes:

[0101] A first pushing module 701 is configured to push a currently generated preview frame to the receiving end;

[0102] A checking module 702 is configured to, when pushing each preview frame to the receiving end, check whether a current video frame to be pushed matches one of the pushed preview frames if it is found that a set condition is met between the sending end and the receiving end, wherein the set condition refers to that there is currently available bandwidth for pushing the video frame;

[0103] A suppressing module 703 is configured to, when the current video frame to be pushed matches one of the pushed preview frames, suppress pushing the current video frame, and when there is a next video frame to be pushed, take the next video frame as the current video frame, and return to the step of checking whether the current video frame to be pushed matches one of the pushed preview frames;

[0104] The second push module 704 is used to push the current recording frame when the current recording frame to be pushed does not match the preview frames that have already been pushed.

[0105] As an example, the first push module 701 further records the frame information of the pushed preview frame.

[0106] As an example, the inspection module 702 further checks whether the frame information of the current recording frame to be pushed matches the frame information of one of the already pushed preview frames. If they match, it is determined that the current recording frame to be pushed matches one of the already pushed preview frames; otherwise, it is determined that the current recording frame to be pushed does not match any of the already pushed preview frames.

[0107] As one example, the frame information includes at least a timestamp;

[0108] Checking whether the frame information of the current recorded frame to be pushed matches the frame information of one of the already pushed preview frames includes:

[0109] Check if the difference between the timestamp of the current recorded frame to be pushed and the timestamp of one of the already pushed preview frames is within the set time difference range.

[0110] As one example, the sending end and the receiving end belong to the same doorbell package;

[0111] The sending end is a doorbell deployed at the entrance, and the receiving end is a display screen deployed indoors.

[0112] This concludes the process. Figure 7 Structural description of the device shown.

[0113] This application also provides a structural diagram of another data stream transmission device. For example, see... Figure 8 , Figure 8 This is a schematic diagram of another data stream transmission device provided in an embodiment of this application. This device corresponds to… Figure 2 The process is shown below. Figure 8 As shown, the data stream transmission device 800 is configured at the receiving end, and the data stream transmission device 800 includes:

[0114] The first caching module 801 is used to cache the currently received preview frame into a pre-configured first buffer when a preview frame pushed by the sending end is received.

[0115] The second buffer module 802 is used to buffer the currently received video frame into a pre-configured second buffer when a video frame pushed by the sending end is received.

[0116] The writing module 803 is used to trigger recording writing based on the video frames cached in the second buffer and the preview frames cached in the first buffer if it is determined that the video frames cached in the second buffer and the preview frames cached in the first buffer meet the preset requirements; the preset requirements refer to the video frames cached in the second buffer and the preview frames cached in the first buffer being consecutive in timestamp.

[0117] As an example, the writing module 803 further sorts the video frames cached in the second buffer and the preview frames cached in the first buffer according to the order of their timestamps.

[0118] If the difference between the timestamps of any two adjacent data frames in the sorted data frames is a set value, then the video frames cached in the second buffer and the preview frames cached in the first buffer are determined to meet the preset requirements.

[0119] Otherwise, it is determined that the video frames cached in the second buffer and the preview frames cached in the first buffer do not meet the preset requirements.

[0120] As an example, the writing module 803 further writes the video frames cached in the second buffer and the preview frames cached in the first buffer to the designated storage medium in chronological order of timestamps.

[0121] As an example, the first cache module 801 further reads the currently received preview frame from the first buffer and displays the currently received preview frame.

[0122] This concludes the process. Figure 8 Structural description of the device shown.

[0123] The specific implementation process of the functions and roles of each unit in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0124] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0125] Please see Figure 9A hardware structure schematic diagram of an electronic device is provided for an exemplary embodiment of the present application. The electronic device can include a processor 901, a communication interface 902, a memory 903 and a communication bus 904. The processor 901, the communication interface 902 and the memory 903 complete the communication among each other through the communication bus 904. Among them, the memory 903 stores a computer program; the processor 901 can execute the steps of the method described in the above embodiment by executing the program stored in the memory 903. The electronic device can also include other hardware according to the actual function of the electronic device, which will not be described here.

[0126] Correspondingly, the embodiments of the present application also provide a computer readable storage medium, and the computer readable storage medium stores a plurality of computer instructions. When the computer instructions are executed by a processor, the method disclosed in the above examples of the present application can be implemented.

[0127] For example, the computer readable storage medium can be a RAM (Random Access Memory), a volatile memory, a non-volatile memory, a flash memory, a storage drive (such as a hard disk drive), a solid state disk, any type of storage disk (such as an optical disk, a DVD, etc.), or similar storage medium, or a combination thereof. The processor and the memory can be supplemented by or incorporated into a special logic circuit.

[0128] The above is only a preferred embodiment of the present application, and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the scope of protection of the present application.

Claims

1. A data stream transmission method, characterized in that, This method is applied to the sending end, and includes: Push the currently generated preview frame to the receiving end; When pushing each preview frame to the receiving end, if it is found that the sending end and the receiving end meet the set conditions, then it is checked whether the current recording frame to be pushed matches one of the preview frames that has been pushed; wherein, the set conditions refer to the existence of available bandwidth for pushing recording frames. When the current recording frame to be pushed matches one of the already pushed preview frames, the push of the current recording frame is suppressed. If there is a next recording frame to be pushed, the next recording frame is used as the current recording frame, and the process returns to the step of checking whether the current recording frame to be pushed matches one of the already pushed preview frames. If the current recording frame to be pushed does not match any of the previously pushed preview frames, then push the current recording frame.

2. The method according to claim 1, characterized in that, The step of pushing the currently generated preview frame to the receiving end further includes: recording the frame information of the pushed preview frame; The step of checking whether the current video frame to be pushed matches one of the already pushed preview frames includes: Check if the frame information of the current recording frame to be pushed matches the frame information of one of the already pushed preview frames. If they match, it is determined that the current recording frame to be pushed matches one of the already pushed preview frames. If they do not match, it is determined that the current recording frame to be pushed does not match any of the already pushed preview frames.

3. The method according to claim 2, characterized in that, The frame information includes at least a timestamp; The step of checking whether the frame information of the current recorded frame to be pushed matches the frame information of one of the already pushed preview frames includes: Check if the difference between the timestamp of the current recorded frame to be pushed and the timestamp of one of the already pushed preview frames is within the set time difference range.

4. The method according to claim 1, characterized in that, The transmitting end and the receiving end belong to the same doorbell package; The transmitting end is a doorbell deployed at the entrance, and the receiving end is a display screen deployed indoors.

5. A data stream transmission method, characterized in that, This method is applied at the receiving end, and includes: When a preview frame is received from the sender, the currently received preview frame is cached in the pre-configured first buffer. When a video frame is received from the sending end, the currently received video frame is buffered in the pre-configured second buffer. If it is determined that the video frames cached in the second buffer and the preview frames cached in the first buffer meet the preset requirements, then recording is triggered based on the video frames cached in the second buffer and the preview frames cached in the first buffer; the preset requirements refer to the video frames cached in the second buffer and the preview frames cached in the first buffer being consecutive in timestamp.

6. The method according to claim 5, characterized in that, The step of determining that the video frames cached in the second buffer and the preview frames cached in the first buffer meet the preset requirements includes: The recorded frames cached in the second buffer and the preview frames cached in the first buffer are sorted according to the order of their timestamps to obtain a sequence. If the difference between the timestamps of any two adjacent frames in the sequence is within a set difference range, then the video frames cached in the second buffer and the preview frames cached in the first buffer are determined to meet the preset requirements; otherwise, the video frames cached in the second buffer and the preview frames cached in the first buffer are determined not to meet the preset requirements.

7. The method according to claim 5, characterized in that, The triggering of recording and writing based on the video frames cached in the second buffer and the preview frames cached in the first buffer includes: Based on the order of timestamps, the video frames cached in the second buffer and the preview frames cached in the first buffer are sequentially written to the specified storage medium.

8. A data stream transmission system, characterized in that, The system includes a transmitter and a receiver; The transmitting end performs the method according to any one of claims 1 to 4; The receiving end performs the method as described in any one of claims 5 to 7.

9. A data stream transmission device, characterized in that, The device is configured at the transmitting end and includes: The first push module is used to push the currently generated preview frame to the receiving end; The inspection module is used to check whether the current recording frame to be pushed matches one of the already pushed preview frames if a set condition is found between the sending end and the receiving end when pushing each preview frame to the receiving end; wherein, the set condition refers to the existence of available bandwidth for pushing recording frames. The suppression module is used to suppress the push of the current recording frame when it matches one of the already pushed preview frames, and when there is a next recording frame to be pushed, to use the next recording frame as the current recording frame and return to the step of checking whether the current recording frame to be pushed matches one of the already pushed preview frames. The second push module is used to push the current recording frame when the current recording frame to be pushed does not match the preview frames that have already been pushed.

10. A data stream transmission device, characterized in that, The device is configured at the receiving end and includes: The first caching module is used to cache the currently received preview frame into a pre-configured first buffer when a preview frame pushed by the sending end is received. The second buffer module is used to buffer the currently received video frame into a pre-configured second buffer when a video frame pushed by the sending end is received. The writing module is used to trigger recording writing based on the video frames cached in the second buffer and the preview frames cached in the first buffer if it is determined that the video frames cached in the second buffer and the preview frames cached in the first buffer meet the preset requirements; the preset requirements refer to the video frames cached in the second buffer and the preview frames cached in the first buffer being consecutive in timestamp.

11. An electronic device, characterized in that, The electronic device includes: Processor; and A memory storing computer program instructions that, when executed by the processor, cause the processor to perform the steps of the method according to any one of claims 1 to 4 or 5 to 7.

Images