Monitoring video adjustment method, device, electronic device and computer readable medium

Abstract

The embodiments of the present disclosure disclose a monitoring video adjustment method, device, electronic device and computer-readable medium. A specific implementation of the method includes: in response to determining that the monitoring video captured by the image acquisition device is to be sent to the target terminal, estimating the bandwidth between the image acquisition device and the target terminal through web real-time communication technology; determining the encoding parameters of the encoder based on the estimated bandwidth and the attribute information of the target terminal; according to the encoding parameters, encoding the monitoring video using the encoder to adjust the transmission bit rate of the monitoring video to achieve real-time transmission of video data; and sending the encoded monitoring video to the target terminal. This implementation proposes a simple and lightweight monitoring video adaptive adjustment method, which helps to reduce the occurrence of video packet loss, freezes, etc., and achieves smooth transmission of monitoring screens.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of video processing technology, and in particular to a monitoring video adjustment method, device, electronic device, and computer-readable medium. Background Art

[0002] With the rapid development of Internet technology, more and more families are installing home surveillance cameras, and the images of surveillance videos are becoming more and more high-definition. However, the inventors found that in the existing home surveillance technology, the original surveillance video collected by the camera is usually sent directly to the mobile phone. The clearer the collected video images, the more network bandwidth resources are required. Therefore, there is usually the following technical problem: when the network bandwidth resources are tight, there will be situations such as loss of surveillance video data and image freezes, which cannot meet the user needs of real-time transmission.

[0003] In addition, WebRTC (Web Real Time Communications) has been widely used in browsers to achieve cross-platform communication functions. According to the public content of the technology, the technology can not only perform bandwidth estimation, but also has a video processing engine (Video Engine). The engine is an overall framework that includes a series of video processing, providing a solution for the entire process from camera video acquisition to video information network transmission to video display. However, the prior art lacks relevant practical application technologies, especially applications between home surveillance cameras and mobile phones. Therefore, if you want to use this technology to process surveillance videos to solve the above problems and achieve real-time transmission, you need to write and debug program files according to its public standard files. This often requires a lot of manpower, material resources and time.

[0004] In the process of using technical solutions to solve the above technical problems, the following technical problem 2 is often accompanied: Although the bandwidth estimation using WebRTC is more accurate, the estimation process is more complicated and often takes a certain amount of time. This cannot meet the real-time encoding adjustment of video data. In response to the above technical problem 2, the conventional solution is generally to use it to estimate the bandwidth before sending the surveillance video, or to estimate the bandwidth periodically. However, the above conventional solution still has the following problem: Since the bandwidth often changes in real time, the bandwidth is likely to be different from the estimated bandwidth during the sending of the surveillance video or before the next bandwidth estimation. If the bandwidth suddenly becomes tight, the surveillance video is still encoded according to the previously estimated bandwidth, and jamming will still occur.

[0005] Continuing, in the process of adopting technical solutions to solve the above technical problem 2, the following technical problem 3 is often accompanied: when the relevant technology encodes the video data, it often uniformly encodes the entire video picture. This will make the video picture clear as a whole, or blurry as a whole. In this way, when the user needs to carefully view a certain partial picture, due to the overall uniform encoding, the user cannot clearly view the partial picture. And these partial pictures are often what the user focuses on.

[0006] The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known in this country to a person of ordinary skill in the art. Summary of the invention

[0007] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0008] Some embodiments of the present disclosure propose a surveillance video adjustment method, a surveillance video adjustment device, an electronic device, a computer-readable medium, and a computer program product to solve one or more of the technical problems mentioned in the above background technology section.

[0009] In a first aspect, some embodiments of the present disclosure provide a monitoring video adjustment method, comprising: in response to determining that the monitoring video captured by an image acquisition device is to be sent to a target terminal, estimating the bandwidth between the image acquisition device and the target terminal through real-time web communication technology; determining the encoding parameters of the encoder based on the estimated bandwidth and attribute information of the target terminal; encoding the monitoring video using the encoder according to the encoding parameters to adjust the transmission bit rate of the monitoring video to achieve real-time transmission of video data; and sending the encoded monitoring video to the target terminal through real-time web communication technology.

[0010] In some embodiments, the encoding parameters of the encoder are determined based on the estimated bandwidth and attribute information of the target terminal, including: obtaining configuration data of the target terminal, and determining a resolution range of video data playable on the target terminal based on the obtained configuration data; determining a bit rate range corresponding to the resolution range of playable video data; and determining the encoding parameters of the encoder based on a comparison result of the estimated bandwidth and the bit rate range.

[0011] In some embodiments, the encoding parameters of the encoder are determined based on the comparison result of the estimated bandwidth and the bit rate range, including: in response to determining that the estimated bit rate indicated by the estimated bandwidth is within the bit rate range, the estimated bit rate and the minimum bit rate within the bit rate range are used as the upper limit result and the lower limit result of the encoding processing of the encoder respectively; or in response to determining that the estimated bit rate indicated by the estimated bandwidth is greater than the bit rate range, at least one high resolution is selected from the resolution range corresponding to the bit rate range as the encoding processing result of the encoder.

[0012] In some embodiments, determining the encoding parameters of the encoder based on the comparison result of the estimated bandwidth and the bit rate range also includes: in response to determining that the estimated bit rate indicated by the estimated bandwidth is less than the bit rate range, using a preset bit rate as the encoding processing result of the encoder, wherein the preset bit rate is a bit rate less than or equal to the minimum bit rate in the bit rate range, including the estimated bit rate.

[0013] Taking into account the problems existing in the conventional solutions adopted for the above-mentioned technical problem 2, in the face of the above-mentioned technical problem 2, combined with the statistical analysis advantages of big data, we can decide to adopt the following solution.

[0014] In some embodiments, the method also includes: in response to detecting that the dynamic properties of the target terminal meet preset conditions during the process of sending the surveillance video, adjusting the current encoding parameters of the encoder to the minimum bit rate within the bit rate range to encode the video data that is not sent in the surveillance video.

[0015] In some embodiments, detecting that the dynamic attributes of the target terminal meet preset conditions includes at least one of the following: detecting that the signal strength of the target terminal is less than a preset threshold and lasts for a preset duration; or detecting that the positioning address of the target terminal is within a preset location range, wherein the preset location range is a location area where the signal strength is reduced; or detecting that the current time is within a preset time period.

[0016] In some embodiments, the image acquisition device is a home surveillance camera, and the target terminal includes a mobile terminal.

[0017] In a second aspect, some embodiments of the present disclosure provide a surveillance video adjustment device, comprising: a bandwidth estimation unit, configured to estimate the bandwidth between the image acquisition device and the target terminal through real-time web communication technology in response to determining that the surveillance video captured by the image acquisition device is to be sent to the target terminal; a coding parameter determination unit, configured to determine the coding parameters of the encoder based on the estimated bandwidth and attribute information of the target terminal; a coding processing unit, configured to encode the surveillance video using the encoder according to the coding parameters to adjust the transmission bit rate of the surveillance video to achieve real-time transmission of video data; a sending unit, configured to send the encoded surveillance video to the target terminal through real-time web communication technology.

[0018] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; an encoder for encoding video data; and a storage device on which one or more programs are stored, and when the one or more programs are executed by one or more processors, the one or more processors implement the monitoring video adjustment method described in any implementation method in the above-mentioned first aspect.

[0019] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the monitoring video adjustment method described in any implementation manner in the above-mentioned first aspect is implemented.

[0020] In a fifth aspect, some embodiments of the present disclosure provide a computer program product, including a computer program, which, when executed by a processor, implements the monitoring video adjustment method described in any implementation manner in the above-mentioned first aspect.

[0021] The above-mentioned embodiments of the present disclosure have the following beneficial effects: the monitoring video adjustment method of some embodiments of the present disclosure can realize adaptive adjustment of the monitoring video and achieve the effect of real-time smooth transmission. Specifically, the reason for the freezing of the monitoring video is that in the existing home monitoring technology, the original data collected by the camera is usually sent directly to the mobile phone terminal. As the collected video screen page becomes higher in definition, more bandwidth resources are required to complete the monitoring video transmission. In this way, when the bandwidth resources are tight, the monitoring video information will be lost or frozen.

[0022] Based on this, in some embodiments of the present disclosure, the monitoring video adjustment method can use web real-time communication technology to estimate the bandwidth before sending the monitoring video. Then, the encoding parameters of the encoder are determined according to the estimated bandwidth to perform secondary encoding processing on the monitoring video. Finally, the encoded monitoring video is sent to the terminal. In this way, the monitoring video can be compressed according to the estimated bandwidth, the amount of monitoring video data to be transmitted is reduced, and adaptive adjustment of the monitoring video is achieved to achieve the purpose of smooth transmission.

[0023] In addition, since the video encoding and decoding process of web real-time communication technology is relatively complex, it is necessary to consider multiple factors such as congestion control, image quality, and dynamic frame rate. Therefore, the resource requirements are relatively high. In addition, there is a lack of relevant practical application technology records in public information. Considering that in home monitoring scenarios, it is usually mainly used for family members and pet care. And the user is very familiar with the environment at home. Therefore, the clarity requirements for video images are often less than other scenarios such as security monitoring and road traffic monitoring. Therefore, this solution proposes a simplified and lightweight video encoding processing technology. Although the encoding processing effect may not be as good as the encoding processing effect of WebRTC, a simple and efficient processing process can be achieved to achieve real-time and smooth transmission of monitoring videos. In addition, this lightweight processing method has much lower requirements for resources such as CPU (central processing unit) than WebRTC. At the same time, this solution can support multiple video encoding methods, and WebRTC does not support H265 video encoding. However, most existing home monitoring uses H265 encoding. It is precisely because the home monitoring scene does not require high video clarity, so this solution can meet the usage requirements of this scene while reducing processing costs, and is more applicable and practical. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0025] Figure 1 is a schematic diagram of an application scenario of the monitoring video adjustment method of some embodiments of the present disclosure;

[0026] Figure 2 is a flow chart of some embodiments of the monitoring video adjustment method disclosed in the present invention;

[0027] Figure 3 are flow charts of other embodiments of the monitoring video adjustment method disclosed herein;

[0028] Figure 4 It is a schematic diagram of the structure of some embodiments of the monitoring video adjustment device disclosed in the present invention;

[0029] Figure 5 It is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0030] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0031] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0032] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0033] In addition, the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".

[0034] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0035] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0036] Figure 1 It is a schematic diagram of an application scenario of the monitoring video adjustment method of some embodiments of the present disclosure.

[0037] exist Figure 1 In the application scenario, the service device 101 can communicate with the image acquisition device 102 and the terminal device 103. When the user wants to view the surveillance video collected by the image acquisition device 102 through the terminal device 103, first, the service device 101 can estimate the bandwidth through the real-time communication technology of the web page, and then determine the encoding parameters. Then, the service device 101 can encode the surveillance video using the encoder. Thus, the encoded surveillance video is sent to the terminal device 103.

[0038] It should be noted that the above-mentioned service device 101 can be hardware or software. When the service device is hardware, it can be implemented as a distributed cluster consisting of multiple servers or terminal devices, or it can be implemented as a single server or a single terminal device. When the service device is embodied as software, it can be installed in the hardware devices listed above. It can be implemented as multiple software or software modules for providing distributed services, for example, or it can be implemented as a single software or software module. No specific limitation is made here.

[0039] It should be understood that Figure 1 The number of each device in the figure is only for illustration. According to the implementation requirements, there may be any number of service devices, image acquisition devices and terminal devices.

[0040] Continue to see Figure 2 , which shows a process 200 of some embodiments of the monitoring video adjustment method according to the present disclosure. The method may include the following steps:

[0041] Step 201, in response to determining to send the surveillance video captured by the image capture device to the target terminal, the bandwidth between the image capture device and the target terminal is estimated through the web real-time communication technology.

[0042] In some embodiments, the execution subject of the monitoring video adjustment method of the present disclosure (such as Figure 1 The service device 101 shown in FIG. 1 can be connected to other electronic devices (such as Figure 1 The image acquisition device 102 and the terminal device 103 in the figure are connected for communication. Here, the image acquisition device can be a device that can realize the image acquisition function, such as a surveillance camera. Among them, the surveillance camera can include a home surveillance camera, a security surveillance camera, a road traffic surveillance camera, etc. Home surveillance cameras are usually used for home monitoring, such as at home or in a small personal shop. In this case, the scenes and personnel are often relatively fixed, with relatively few changes, and the monitoring range is small. Security surveillance cameras are usually used for security protection monitoring, such as in public places such as shopping malls, scenic spots, and office buildings. In this case, the mobility of personnel is often large and the monitoring range is wide. Road traffic surveillance cameras are usually used for road traffic monitoring, such as speed measurement, line crossing, traffic lights, etc. Violation photography. The traffic volume in this case is often large. Here, the terminal device can be any device with a video display function, such as a mobile terminal (such as a mobile phone, a smart watch, a tablet computer, etc.), a notebook, a personal desktop computer, etc.

[0043] In some embodiments, a user can use a terminal device to view the surveillance video collected by the image acquisition device. As an example, a user can send a viewing instruction through a terminal device, such as triggering (such as clicking) a control representing viewing a surveillance video, or opening a related application, etc. Here, when the execution subject receives the above-mentioned viewing instruction, it can determine that the surveillance video collected by the image acquisition device needs to be sent to the target terminal. The target terminal here is usually a terminal device that has established a connection with the image acquisition device and needs to view the surveillance video. In this case, the execution subject can estimate the bandwidth between the image acquisition device and the target terminal through web real-time communication technology (WebRTC).

[0044] It should be noted that bandwidth estimation generally refers to the process of measuring and estimating the bandwidth on a network link through a series of algorithms and techniques. Bandwidth usually refers to the ability to transmit data from one point to another in a unit of time, usually in bits per second (bps). The purpose of bandwidth estimation is usually to evaluate the actual transmission speed of a network link, thereby providing reference data to ensure the reasonable allocation of network resources and performance optimization. WebRTC is a real-time communication technology that allows network applications or sites to establish peer-to-peer connections between browsers without the help of an intermediary, and to achieve the transmission of video streams and / or audio streams or any other data.

[0045] In addition, WebRTC also has a bandwidth estimation function. As an example, WebRTC can perform bandwidth estimation based on delay gradient, and can also perform bandwidth estimation based on packet loss. In addition, the final bandwidth estimation value can also be calculated in combination with delay and packet loss. The specific bandwidth estimation principle will not be repeated here. It can be understood that WebRTC has a relatively complete and mature set of data processing standards and processes, and has been widely used in video and audio data transmission on web pages. Therefore, more accurate bandwidth estimation can be achieved through WebRTC. This can make the subsequent encoding processing of surveillance videos more efficient. This is conducive to the real-time and smooth transmission of surveillance videos.

[0046] Step 202: Determine encoding parameters of the encoder based on the estimated bandwidth and attribute information of the target terminal.

[0047] In some embodiments, the execution subject can determine the encoding parameters of the encoder based on the estimated bandwidth (i.e., bandwidth estimation value) obtained in step 201 and the attribute information of the target terminal. The attribute information can represent the information of the functional attributes of the terminal, especially the attributes related to the video playback function. The encoding parameters can be parameters related to the compression encoding process of the video data, such as the encoding method (i.e., video compression standard) adopted, the video bit rate after encoding, the video format after encoding, etc.

[0048] As an example, the execution subject can determine the encoding parameters based on the estimated bandwidth. For example, the estimated bandwidth is converted into a bit rate representation, that is, the estimated bandwidth is multiplied by 1024. The result obtained by multiplying the bit rate by a certain coefficient (less than 1) is used as the final encoding processing result of the encoder. That is, the bit rate of the video data after encoding. For another example, in order to make the monitoring video compatible with the target terminal so that the monitoring video can be played smoothly on the target terminal, the execution subject can also determine the encoding parameters of the encoder based on the estimated bandwidth and the attribute information of the target terminal.

[0049] In some embodiments, first, the execution subject can obtain the configuration data of the target terminal. The configuration data here can be software and hardware configuration data. As an example, when establishing a connection with the target terminal, the execution subject can obtain (or the target terminal sends to it) the model, factory settings, and / or installed video playback software (or browser) and other configuration data of the target terminal. Then, based on the obtained configuration data, the execution subject can determine the resolution range of the video data that can be played on the target terminal. And then the bit rate range corresponding to the resolution range of the video data that can be played can be determined. For example, based on the configuration data such as the model, the execution subject can use public resources to learn about the video playback function of the target terminal. Thereby determining the resolution range of the video data that can be played on it.

[0050] It should be noted that the resolution of video data is usually several standard ones, such as standard definition 640×480 pixels, high definition 1280×720 pixels, Blu-ray, 2K, etc., as well as resolutions that may appear in the future. Therefore, the above resolution range is usually the range between the highest playable resolution and the lowest playable resolution. Here, the execution subject can determine the corresponding bit rate range according to the product value of the resolution and the number of pixel bytes, the number of byte bits, and the number of frames. Afterwards, the execution subject can determine the encoding parameters of the encoder based on the comparison result of the estimated bandwidth and the bit rate range.

[0051] Furthermore, if it is determined that the estimated bit rate indicated by the estimated bandwidth is within the bit rate range, the estimated bit rate and the minimum bit rate within the bit rate range can be used as the upper limit result and the lower limit result of the encoder encoding process, respectively. That is to say, the network at this time is moderate, but cannot meet the requirements of sending higher resolution video data. If it is determined that the estimated bit rate indicated by the estimated bandwidth is greater than the bit rate range, at least one high resolution can be selected from the resolution range corresponding to the bit rate range as the encoding processing result of the encoder. For example, among the three resolutions, select the highest one, or select two higher ones. That is to say, the network at this time is very good and can send high-resolution video data.

[0052] Optionally, if it is determined that the estimated bit rate indicated by the estimated bandwidth is less than the bit rate range, the preset bit rate can be used as the encoding processing result of the encoder. The preset bit rate can be a bit rate that is less than or equal to the minimum bit rate in the bit rate range, including the estimated bit rate. In other words, the network at this time is poor and only lower resolution video data can be sent. That is, in addition to the upper and lower limits, a gear with a lower bit rate can also be set in the encoder.

[0053] Step 203: Encode the surveillance video using an encoder according to the encoding parameters.

[0054] In some embodiments, the execution subject may use an encoder to encode the surveillance video according to the encoding parameters determined in step 202. In this way, the surveillance video may be compressed and encoded to a code rate less than or equal to the determined code rate, thereby adjusting the transmission code rate of the surveillance video, realizing real-time transmission of video data, and reducing video freezes.

[0055] Step 204: Send the encoded surveillance video to the target terminal.

[0056] In some embodiments, the execution subject may send the encoded surveillance video to the target terminal. As an example, the execution subject may send the encoded surveillance video to the target terminal via a network. Alternatively, since the bandwidth estimation is performed by web real-time communication technology in step 201, in order to further ensure the accuracy of the estimation, the image acquisition device and the target terminal may be connected by web real-time communication technology. In this case, the execution subject may send the encoded surveillance video to the target terminal by web real-time communication technology.

[0057] It can be seen from the above description that the monitoring video adjustment method of some embodiments of the present disclosure can use web real-time communication technology to estimate the bandwidth before sending the monitoring video. Then, the encoding parameters of the encoder are determined according to the estimated bandwidth to perform secondary encoding processing on the monitoring video. Finally, the encoded monitoring video is sent to the terminal. In this way, the monitoring video can be compressed according to the estimated bandwidth, the amount of monitoring video data to be transmitted is reduced, and adaptive adjustment of the monitoring video is achieved to achieve the purpose of smooth transmission.

[0058] It is understandable that although the bandwidth estimation using WebRTC is more accurate, the estimation process is more complicated and often takes a certain amount of time. This cannot meet the real-time encoding adjustment and transmission of video data. Therefore, it can be used to estimate the bandwidth before sending the surveillance video or periodically. The image acquisition device is usually fixed in position after installation, and the communication signal is relatively stable. The position of the target terminal (especially the mobile terminal) is often moving and changing. As its position moves and changes, its signal will generally be affected and unstable. Therefore, whether the surveillance video can be sent smoothly to the target terminal is often affected by the signal of the target terminal.

[0059] Therefore, in some embodiments, during the process of sending surveillance videos, the dynamic attributes of the target terminal can be detected to achieve real-time encoding adjustment of the surveillance videos, so as to ensure the real-time and smooth transmission of the surveillance videos and reduce freezes and delays. Figure 3 In the process shown, during the process of sending the surveillance video, if it is detected that the dynamic attribute of the target terminal meets the preset conditions, the current encoding parameters of the encoder can be adjusted to the minimum bit rate within the bit rate range to encode the video data that is not sent in the surveillance video. Among them, the dynamic attribute generally refers to the attribute whose value will change over time, such as signal strength, location address, etc. In other words, in these cases, the execution subject can compress and encode the surveillance video to the lowest resolution that can be played by the target terminal.

[0060] Here, the dynamic attribute meeting the preset conditions may include at least one of the following: detecting that the signal strength of the target terminal is less than a preset threshold and the duration reaches a preset duration; or detecting that the location address of the target terminal is within a preset location range, wherein the preset location range is a location area where the signal strength decreases; or detecting that the current time is within a preset time period. Among them, the preset threshold, preset duration, etc. can be set according to actual needs. The signal strength and the method of obtaining the location address here are also not limited. For example, the signal strength can be obtained through the signal icon displayed on the target terminal. For example, a full grid indicates that the signal strength is 100%; one grid indicates that the signal strength is 10%.

[0061] In some embodiments, the above-mentioned preset position range can be obtained in the following manner: through big data analysis, determine the first position range where the signal strength of the target signal type is less than the average signal strength, wherein the target signal type is the signal type used for communication by the target terminal, such as a certain mobile, a certain communication, or a certain letter, etc.; obtain the historical movement data and the corresponding historical signal data of the target terminal, and determine the correspondence between the mobile position and the signal change through data analysis; according to the determined correspondence, select the mobile position with a signal strength less than a preset threshold as the second position range; according to the base station setting position of the target signal type signal, determine the third position range not covered by the base station; based on the first position range, the second position range and the third position range, determine the preset position range.

[0062] It should be noted that the second location range usually uses the location that the user of the target terminal has been to or often goes to (or passes through), such as the location of the commute route. In this way, the above-mentioned preset location range is determined according to the second location range, which can improve the accuracy and reference of the determined location. And the encoding processing of the monitoring video can be performed in this way, which can ensure the accuracy of the necessary encoding processing and reduce the number of unnecessary encoding processing. While ensuring the smooth transmission of the monitoring video, the picture quality of the monitoring video is improved as much as possible. In addition, through the first location range and the third location range, the location that the user has not been to but may go to in the future can be supplemented. In this way, the number and coverage of the preset location range can be expanded. At the same time, the signal of the same signal type as the target terminal is used as the determination standard of the first and third location ranges, which can further improve the reference of the first and third location ranges. Thereby further improving the accuracy of the preset location range. Here, the execution subject can determine all the locations included in the three location ranges as within the preset location range. Alternatively, the execution subject can filter out the common locations in the first location range and the third location range. Then the common location and the second location range are determined as within the preset location range.

[0063] Optionally, the preset time period can be obtained in the following manner: obtaining the historical mobile data and the corresponding historical signal data of the target terminal; performing statistical analysis on the historical mobile data and the historical signal data to determine the correlation between the mobile position, signal, and time; based on the determined correlation, selecting a time period that characterizes the periodic decrease in signal strength as the preset time period. As an example, a user using the target terminal travels from A to B between 8 and 9 o'clock every weekday. And when passing through C, the signal strength will weaken, and the time passing through C is approximately 8:30 to 8:45. In this case, the execution entity can determine 8:30 to 8:45 on weekdays as the preset time period.

[0064] It is understandable that for commuting users, their movement trajectories and time are often relatively fixed. Therefore, it is easy to count the time periods when the signal strength periodically decreases. If the user views the surveillance video during this time period, encoding processing is usually required. This can further improve the accuracy of the necessary encoding processing. And by performing encoding processing based on the judgment of the preset time period, the judgment process can be simplified, and the judgment data required to be obtained can be reduced, which is conducive to improving the timeliness of the encoding processing, thereby ensuring the timeliness of the video transmission.

[0065] It is understandable that when the related technology encodes the video data, it often uniformly encodes the entire video picture. This will lead to the following technical problem three: the video picture is very clear as a whole, or it is blurred as a whole. In this way, when the user needs to carefully view a certain partial picture, the user cannot clearly view the partial picture due to the overall uniform encoding. And these partial pictures are often what the user focuses on.

[0066] Based on this, in some application scenarios, in order to ensure smooth transmission of surveillance video while improving the clarity of the focused screen, the execution subject can also perform encoding processing of the surveillance video according to the user's operation behavior on the target terminal. Specifically, first, in response to detecting a trigger operation on the video screen of the surveillance video displayed on the target terminal, the target screen area indicated by the trigger operation is determined. Then, according to the target screen area and encoding parameters, the surveillance video is encoded. Among them, in the video screen of the surveillance video, the resolution of the target screen area can be greater than or equal to other screen areas. The above-mentioned trigger operation can be an operation that represents screen selection, such as clicking on an object (including a person) in the video screen, or enlarging an area, or boxing an object or area, etc.

[0067] As an example, the encoder may include a first encoder and a second encoder. When the execution subject receives the video data of the surveillance video, it may input it into the first encoder or the second encoder according to the video screen position represented by the video data. For example, the video data of the target screen area is input into the first encoder, and the video data of other screen areas may be input into the second encoder. And the transmission bit rate of the surveillance video after encoding and processing by the two encoders is not greater than the encoding parameters determined above. It is understandable that when the network is good, the clarity of the video screen can be guaranteed. At this time, the resolution of the two screen areas can be the same. And one of the encoders can be used for encoding processing. When the network is poor, two encoders can be used for processing separately. At the same time, it is prioritized to ensure that the target screen area has a higher resolution to ensure the clarity of the target screen area.

[0068] Further references Figure 4 , as a response to the above Figures 1 to 3In order to realize the method shown in the figure, the present disclosure provides some embodiments of a monitoring video adjustment device. These device embodiments are similar to Figures 1 to 3 The device can be specifically applied to various electronic devices.

[0069] like Figure 4 As shown, the surveillance video adjustment device 400 of some embodiments may include: a bandwidth estimation unit 401, configured to estimate the bandwidth between the image acquisition device and the target terminal through web real-time communication technology in response to determining that the surveillance video acquired by the image acquisition device is to be sent to the target terminal; an encoding parameter determination unit 402, configured to determine the encoding parameters of the encoder based on the estimated bandwidth and the attribute information of the target terminal; an encoding processing unit 403, configured to encode the surveillance video using the encoder according to the encoding parameters to adjust the transmission bit rate of the surveillance video to achieve real-time transmission of video data; a sending unit 404, configured to send the encoded surveillance video to the target terminal.

[0070] In some embodiments, the encoding parameter determination unit 402 can be configured to obtain configuration data of the target terminal, determine the resolution range of video data that can be played on the target terminal based on the obtained configuration data; determine the bit rate range corresponding to the resolution range of the playable video data; and determine the encoding parameters of the encoder based on the comparison result of the estimated bandwidth and the bit rate range.

[0071] In some embodiments, the encoding parameter determination unit 402 is further configured to, in response to determining that the estimated bit rate indicated by the estimated bandwidth is within the bit rate range, use the estimated bit rate and the minimum bit rate within the bit rate range as the upper limit result and lower limit result of the encoder encoding processing, respectively; or, in response to determining that the estimated bit rate indicated by the estimated bandwidth is greater than the bit rate range, select at least one high resolution from the resolution range corresponding to the bit rate range as the encoding processing result of the encoder.

[0072] In some embodiments, the encoding parameter determination unit 402 is further configured to, in response to determining that the estimated bit rate indicated by the estimated bandwidth is less than the bit rate range, use a preset bit rate as the encoding processing result of the encoder, wherein the preset bit rate is a bit rate less than or equal to the minimum bit rate in the bit rate range, including the estimated bit rate.

[0073] In some embodiments, the surveillance video adjustment device 400 may also include a detection unit (not shown in the figure), which is configured to adjust the current encoding parameters of the encoder to the minimum bit rate within the bit rate range in response to detecting that the dynamic attributes of the target terminal meet preset conditions during the process of sending the surveillance video, so as to encode the video data that is not sent in the surveillance video.

[0074] In some embodiments, the detection unit is further configured to do at least one of the following: detect that the signal strength of the target terminal is less than a preset threshold and lasts for a preset duration; or detect that the location address of the target terminal is within a preset location range, wherein the preset location range is a location area where the signal strength is reduced; or detect that the current time is within a preset time period.

[0075] In some embodiments, the image acquisition device may be a home surveillance camera, and the target terminal may include a mobile terminal.

[0076] It is understandable that the units described in the monitoring video adjustment device 400 are similar to those described in the reference Figures 1 to 3 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the monitoring video adjustment device 400 and the units included therein, and will not be described in detail here.

[0077] Reference below Figure 5 , which shows a structural schematic diagram of an electronic device 500 suitable for implementing some embodiments of the present disclosure. Figure 5 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0078] like Figure 5 As shown, the electronic device 500 may include a processing device 501 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the electronic device 500 are also stored. The processing device 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0079] Typically, the following devices may be connected to the I / O interface 505: input devices 506 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 507 including, for example, a speaker, a vibrator, etc.; storage devices 508 including, for example, a disk, a memory card, etc.; and communication devices 509. The communication device 509 may allow the electronic device 500 to communicate with other devices wirelessly or by wire to exchange data. And an encoder 510 for encoding video data. Although Figure 5 The electronic device 500 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 5Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0080] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 509, or installed from the storage device 508, or installed from the ROM 502. When the computer program is executed by the processing device 501, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.

[0081] It should be noted that the computer-readable medium recorded in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0082] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (Hyper Text Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0083] The computer-readable medium may be included in the electronic device; or it may exist independently without being installed in the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: in response to determining to send the surveillance video captured by the image acquisition device to the target terminal, estimates the bandwidth between the image acquisition device and the target terminal through the real-time web communication technology; determines the encoding parameters of the encoder based on the estimated bandwidth and the attribute information of the target terminal; encodes the surveillance video using the encoder according to the encoding parameters to adjust the transmission bit rate of the surveillance video to realize the real-time transmission of the video data; sends the encoded surveillance video to the target terminal through the real-time web communication technology.

[0084] In addition, computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0085] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0086] The units described in some embodiments of the present disclosure may be implemented by software or by hardware. The described units may also be arranged in a processor, for example, may be described as: a processor includes a bandwidth estimation unit, a coding parameter determination unit and a coding processing unit. The names of these units do not constitute limitations on the units themselves in certain cases, for example, the bandwidth estimation unit may also be described as a "unit for estimating the bandwidth between the image acquisition device and the target terminal".

[0087] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0088] Some embodiments of the present disclosure further provide a computer program product, including a computer program, which, when executed by a processor, implements any of the above-mentioned surveillance video adjustment methods.

[0089] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.

Claims

1. A monitoring video adjustment method, comprising: In response to determining to send the surveillance video captured by the image acquisition device to the target terminal, estimating the bandwidth between the image acquisition device and the target terminal through a web real-time communication technology, wherein the image acquisition device is a home surveillance camera; Acquiring configuration data of the target terminal, and determining a resolution range of video data playable on the target terminal according to the acquired configuration data; Determine the bit rate range corresponding to the resolution range of playable video data; Determining encoding parameters of an encoder according to a comparison result of the estimated bandwidth and the bit rate range; According to the encoding parameters, the monitoring video is encoded using the encoder to adjust the transmission bit rate of the monitoring video to achieve real-time transmission of video data; Sending the encoded surveillance video to the target terminal; The step of determining the encoding parameters of the encoder according to the comparison result between the estimated bandwidth and the bit rate range includes: In response to determining that the estimated bit rate indicated by the estimated bandwidth is less than the bit rate range, using a preset bit rate as an encoding processing result of the encoder, wherein the preset bit rate is a bit rate that is less than or equal to a minimum bit rate in the bit rate range, including the estimated bit rate; In response to detecting a triggering operation on a video screen of a surveillance video displayed on a target terminal, determining a target screen area indicated by the triggering operation; Encoding the surveillance video according to the target screen area and the encoding parameters, wherein, in the video screen of the surveillance video, the resolution of the target screen area is greater than or equal to that of other screen areas, and the trigger operation is an operation representing screen selection; When video data of a surveillance video is received, the video data is input into a first encoder or a second encoder included in the encoder according to the video screen position represented by the video data, wherein the video data of a target screen area is input into the first encoder, and the video data of other screen areas are input into the second encoder, and the transmission bit rate of the surveillance video after encoding and processing by the two encoders is not greater than the determined encoding parameters. When the network is good, the clarity of the video screen is guaranteed, and the resolutions of the two screen areas are the same, and one of the encoders is used for encoding processing. When the network is poor, the two encoders are used for processing separately, and at the same time, priority is given to ensuring that the target screen area has a higher resolution to ensure the clarity of the target screen area.

2. The monitoring video adjustment method according to claim 1, wherein: The step of determining the encoding parameters of the encoder according to the comparison result between the estimated bandwidth and the bit rate range includes: In response to determining that the estimated bit rate indicated by the estimated bandwidth is within the bit rate range, using the estimated bit rate and the minimum bit rate within the bit rate range as an upper limit result and a lower limit result of encoding processing by the encoder, respectively; or In response to determining that the estimated bit rate indicated by the estimated bandwidth is greater than the bit rate range, at least one high resolution is selected from a resolution range corresponding to the bit rate range as an encoding processing result of the encoder.

3. The monitoring video adjustment method according to claim 1, wherein: The method further comprises: In response to detecting that the dynamic attribute of the target terminal meets a preset condition during the process of sending the monitoring video, the current encoding parameter of the encoder is adjusted to the minimum bit rate within the bit rate range to encode the video data that is not sent in the monitoring video.

4. The monitoring video adjustment method according to claim 3, wherein: The detecting that the dynamic attribute of the target terminal meets a preset condition includes at least one of the following: It is detected that the signal strength of the target terminal is less than a preset threshold and the duration reaches a preset duration; or It is detected that the location address of the target terminal is within a preset location range, wherein the preset location range is a location area with reduced signal strength; or It is detected that the current time is within the preset time period.

5. The monitoring video adjustment method according to any one of claims 1 to 4, wherein: The target terminal includes a mobile terminal.

6. A monitoring video adjustment device, comprising: a bandwidth estimation unit, configured to estimate the bandwidth between the image acquisition device and the target terminal through a web real-time communication technology in response to determining to send the surveillance video acquired by the image acquisition device to the target terminal, wherein the image acquisition device is a home surveillance camera; an encoding parameter determination unit configured to obtain configuration data of the target terminal, determine a resolution range of video data playable on the target terminal according to the obtained configuration data, determine a bit rate range corresponding to the resolution range of the playable video data, and determine encoding parameters of an encoder according to a comparison result of an estimated bandwidth and the bit rate range; A first encoding processing unit is configured to perform encoding processing on the monitoring video using the encoder according to the encoding parameters, so as to adjust the transmission bit rate of the monitoring video to achieve real-time transmission of video data; A sending unit, configured to send the encoded monitoring video to the target terminal through the web real-time communication technology; The step of determining the encoding parameters of the encoder according to the comparison result between the estimated bandwidth and the bit rate range includes: In response to determining that the estimated bit rate indicated by the estimated bandwidth is less than the bit rate range, using a preset bit rate as an encoding processing result of the encoder, wherein the preset bit rate is a bit rate that is less than or equal to a minimum bit rate in the bit rate range, including the estimated bit rate; a detection unit configured to, in response to detecting a triggering operation on a video screen of a surveillance video displayed on a target terminal, determine a target screen area indicated by the triggering operation; The second encoding processing unit is configured to perform encoding processing on the monitoring video according to the target picture area and the encoding parameters, wherein, in the video picture of the monitoring video, the resolution of the target picture area is greater than or equal to that of other picture areas; The input unit is configured to input the video data of the surveillance video into the first encoder or the second encoder included in the encoder according to the video screen position represented by the video data when the video data is received, wherein the video data of the target screen area is input into the first encoder, and the video data of other screen areas are input into the second encoder, and the transmission bit rate of the surveillance video after encoding and processing by the two encoders is not greater than the determined encoding parameters, and when the network is good, the clarity of the video screen is guaranteed, the resolution of the two screen areas is the same, and one of the encoders is used for encoding processing, and when the network is poor, the two encoders are used for processing separately, and at the same time, priority is given to ensuring that the target screen area has a higher resolution to ensure the clarity of the target screen area.

7. An electronic device comprising: one or more processors; An encoder, used for encoding video data; a storage device having one or more programs stored thereon, When the one or more programs are executed by the one or more processors, the one or more processors implement the monitoring video adjustment method as described in any one of claims 1-5.

8. A computer readable medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the monitoring video adjustment method as described in any one of claims 1 to 5 is implemented.

Images