Video stream push method, device and electronic device

Abstract

The present application provides a video stream push method, device and electronic device, which relates to the field of intelligent monitoring technology. In the present application, the probability value of the target object viewing the target monitoring within a preset time period is first obtained, and then the working mode of the target monitoring is determined based on the numerical interval to which the probability value belongs, and finally the video stream is pushed to the target object according to the video stream push rule corresponding to the working mode of the target monitoring. In this way, it can be ensured that the user can quickly wake up the target monitoring by clicking the browsing screen of the monitoring APP under low power consumption, and push the video stream to the monitoring APP, thereby increasing the rate of video stream push.

Description

Technical Field

[0001] The present application relates to the field of intelligent monitoring technology, and in particular to a video stream pushing method, device and electronic device. Background Art

[0002] With the rapid development of intelligent monitoring technology, more and more monitoring equipment is widely used in various industries (for example, monitoring equipment deployed in parks, subway stations, docks, etc.). Among them, monitoring equipment based on cellular (4G, 5G) networks is more widely used due to its ultra-large bandwidth and low-latency network characteristics.

[0003] On the one hand, monitoring equipment based on cellular networks does not need to lay a dedicated monitoring network. The monitoring equipment can quickly access the external network by simply connecting to the user identity card (SIM) and transmit the collected video data to the monitoring platform through the external network, realizing convenient transmission of video data.

[0004] On the other hand, with the further optimization of cellular networks and the increasing demand of users for video quality, cellular network-based monitoring equipment also makes it possible to transmit ultra-high-definition videos such as 4K and 8K.

[0005] Although cellular network-based monitoring devices have the above advantages, most of these monitoring devices are battery-based monitoring devices, and users usually have high requirements on the usage time of monitoring devices. Therefore, how to reduce the power consumption of such monitoring devices is crucial.

[0006] In traditional solutions to reduce the power consumption of monitoring devices, when the user or monitoring application (APP) does not initiate a streaming request to the monitoring device, the monitoring device is usually put into a sleep state to reduce power consumption. When the user clicks on the preview screen through the monitoring application, the monitoring device will be awakened, but this wake-up action usually takes 3-10 seconds. Currently, it is impossible to guarantee the real-time effect of quickly streaming when the user clicks on the browsing screen under low power consumption. Summary of the invention

[0007] The present invention provides a data stream push method, device and electronic device to solve the problem that the overall power consumption of low-power devices based on PIR perception is high and the remote APP randomly wakes up the monitoring device to output the data stream, which takes time. The specific technical solution is as follows:

[0008] In a first aspect, the present application provides a data stream pushing method, comprising:

[0009] Obtain the probability value of the target object viewing the target monitoring within a preset time period;

[0010] Determining a working mode of the target monitoring based on the numerical interval to which the probability value belongs;

[0011] The video stream is pushed to the target object according to the video stream pushing rule corresponding to the working mode of the target monitoring.

[0012] By acquiring perception information related to the user or monitoring APP and performing real-time AI analysis in combination with monitoring screen information, it is possible to accurately predict the probability value of the target object viewing the target monitoring within a preset time period, and then select different low-power modes based on the probability value, thereby reducing the hardware modules powered by the monitoring device and the number of wake-up times, thereby achieving the purpose of reducing device power consumption; by waking up the device in advance when the probability value is greater than 99%, the user can open the monitoring APP and immediately start streaming, reducing the wake-up delay to 0 seconds; when the probability value is lower than 99%, the heartbeat keep-alive interval and paging cycle are dynamically adjusted according to the change in the probability value, thereby achieving dual optimization of the power consumption and timeliness of the monitoring device; in the event of an error in the prediction of the AI ​​training platform, a remedial mechanism is designed (timed wake-up of the monitoring device) to avoid the long-term inability to pull APP traffic when the device is in low-power mode, and at the same time, the training results are adjusted based on the latest perception data to update the working mode of the monitoring device.

[0013] In a possible implementation, determining the target monitoring working mode based on the numerical interval to which the probability value belongs includes:

[0014] If the probability value belongs to the first numerical interval, switching the target monitoring working mode from the initial working mode to the first low power consumption working mode, and operating based on the first low power consumption working mode;

[0015] If the probability value belongs to the second numerical interval, switching the target monitoring working mode from the initial working mode to the second low power consumption working mode, and operating based on the second low power consumption working mode;

[0016] If the probability value belongs to the third numerical interval, the target monitoring working mode is switched from the initial working mode to the third low power consumption working mode, and operates based on the third low power consumption working mode.

[0017] Based on the numerical interval to which the probability value belongs, different low-power operation modes are selected, so that the streaming request sent by the APP or user can be transmitted to the device end with the lowest delay while reducing the power consumption of the monitoring device.

[0018] In a possible implementation, if the target monitoring is operated in the first low power consumption working mode, pushing the video stream to the target object according to the video stream pushing rule corresponding to the working mode of the target monitoring includes:

[0019] Setting the target monitoring to a dormant state, wherein the dormant state means waking up the main control hardware of the target monitoring periodically according to a preset first duration, and powering off other hardware of the target monitoring;

[0020] Determining whether the current duration reaches the first duration;

[0021] If so, the master control hardware and the other hardware are powered on, and it is determined whether a video stream acquisition request initiated by the target object is received within the first time period. When it is determined that the video stream acquisition request is received, the target video stream associated with the video stream acquisition request is pushed to the target object, and the working mode of the target monitoring is adjusted based on the first updated value of the probability value;

[0022] If not, the sleep state is maintained.

[0023] By designing a remedial mechanism (periodic wake-up of the monitoring device), it is possible to avoid the long-term inability to pull APP traffic when the device is in the first low-power mode. At the same time, the training results are adjusted based on the latest perception data to update the working mode of the monitoring device.

[0024] In a possible implementation, if the target monitoring is operated in the second low power consumption working mode, pushing the video stream to the target object according to the video stream pushing rule corresponding to the working mode of the target monitoring includes:

[0025] Setting the target monitoring to a wake-up state, wherein the wake-up state means powering on all hardware of the target monitoring;

[0026] Determining whether a video stream acquisition request initiated by the target object is received within a preset time period;

[0027] If yes, push the target video stream associated with the video stream acquisition request to the target object;

[0028] If not, a second updated value of the probability value is obtained, and the working mode of the target monitoring is adjusted based on the second updated value.

[0029] By waking up the target monitoring in advance when the probability value of the user using the monitoring APP to view the target monitoring within the preset time period is greater than 99%, the user can push the video stream immediately when opening the monitoring APP, which will greatly reduce the wake-up delay of the target monitoring.

[0030] In a possible implementation, if the target monitoring is operated in the third low power consumption working mode, pushing the video stream to the target object according to the video stream pushing rule corresponding to the working mode of the target monitoring includes:

[0031] Based on the current probability value of the target object viewing the target monitoring in the current time period, adjust the heartbeat keep-alive interval and paging cycle of the target monitoring in the current time period, wherein the heartbeat keep-alive interval represents the time interval for the target monitoring to periodically send heartbeat packets to the monitoring platform; the paging cycle represents the periodic time for the target monitoring to regularly check whether there is a paging message in idle mode;

[0032] When it is determined that the video stream acquisition request initiated by the target object is received, the target monitoring device is woken up according to the heartbeat keep-alive interval and the paging cycle, and the target video stream associated with the video stream acquisition request is pushed to the target object.

[0033] When the probability value is higher than 1% and lower than 99%, the heartbeat keep-alive interval and paging cycle are dynamically adjusted according to the change of the probability value, which can achieve dual optimization of the power consumption of the monitoring equipment and the timeliness of the outflow.

[0034] In a possible implementation, obtaining the probability value of the target object viewing the target monitoring within a preset time period includes:

[0035] Acquire perception data associated with the target object, wherein the perception data includes at least location information, channel state information, and behavior information;

[0036] Using the perception data to train a preset user behavior prediction model to obtain an updated target user behavior prediction model;

[0037] The perception data of the target object when currently viewing the target monitoring is input into the target user behavior prediction model to obtain the probability value of the target object viewing the target monitoring within a preset time period.

[0038] Through the above method, the user behavior prediction model can be trained and optimized to obtain the probability value of the target object viewing the target monitoring within a preset time period.

[0039] In a possible implementation, before obtaining the probability value of the target object viewing the target monitoring within a preset time period, the method further includes:

[0040] Establishing a communication connection between the target monitoring and a base station, initiating device registration to the monitoring platform through the base station, and receiving a registration message returned by the monitoring platform after successful device registration;

[0041] Based on the registration message, the initial working mode of the target monitoring is determined, wherein the initial working mode indicates that the target monitoring and the monitoring platform operate according to a preset heartbeat keep-alive interval, and the target monitoring and the base station operate according to a preset paging cycle.

[0042] Through the above method, the monitoring device can be initialized, so that the monitoring device can operate in a more moderate power consumption mode, thereby reducing the power consumption of the monitoring device.

[0043] In a second aspect, the present application provides a video stream pushing device, comprising:

[0044] Video stream processing module, used to collect video streams of the monitoring area;

[0045] The video stream transmission module is used to obtain the probability value of the target object viewing the target monitoring within a preset time period;

[0046] A power consumption management module, used for determining the working mode of the target monitoring based on the numerical interval to which the probability value belongs;

[0047] The video stream is pushed to the target object according to the video stream pushing rule corresponding to the working mode of the target monitoring.

[0048] In a possible implementation, the power consumption management module is specifically used to:

[0049] If the probability value belongs to the first numerical interval, switching the target monitoring working mode from the initial working mode to the first low power consumption working mode, and operating based on the first low power consumption working mode;

[0050] If the probability value belongs to the second numerical interval, switching the target monitoring working mode from the initial working mode to the second low power consumption working mode, and operating based on the second low power consumption working mode;

[0051] If the probability value belongs to the third numerical interval, the target monitoring working mode is switched from the initial working mode to the third low power consumption working mode, and operates based on the third low power consumption working mode.

[0052] In a possible implementation, the power consumption management module is specifically used to:

[0053] Setting the target monitoring to a dormant state, wherein the dormant state means waking up the main control hardware of the target monitoring periodically according to a preset first duration, and powering off other hardware of the target monitoring;

[0054] Determining whether the current duration reaches the first duration;

[0055] If so, the master control hardware and the other hardware are powered on, and it is determined whether a video stream acquisition request initiated by the target object is received within the first time period. When it is determined that the video stream acquisition request is received, the target video stream associated with the video stream acquisition request is pushed to the target object, and the working mode of the target monitoring is adjusted based on the first updated value of the probability value;

[0056] If not, the sleep state is maintained.

[0057] In a possible implementation, the power consumption management module is specifically used to:

[0058] Setting the target monitoring to a wake-up state, wherein the wake-up state means powering on all hardware of the target monitoring;

[0059] Determining whether a video stream acquisition request initiated by the target object is received within a preset time period;

[0060] If yes, push the target video stream associated with the video stream acquisition request to the target object;

[0061] If not, a second updated value of the probability value is obtained, and the working mode of the target monitoring is adjusted based on the second updated value.

[0062] In a possible implementation, the power consumption management module is specifically used to:

[0063] Based on the current probability value of the target object viewing the target monitoring in the current time period, adjust the heartbeat keep-alive interval and paging cycle of the target monitoring in the current time period, wherein the heartbeat keep-alive interval represents the time interval for the target monitoring to periodically send heartbeat packets to the monitoring platform; the paging cycle represents the periodic time for the target monitoring to regularly check whether there is a paging message in idle mode;

[0064] When it is determined that the video stream acquisition request initiated by the target object is received, the target monitoring device is woken up according to the heartbeat keep-alive interval and the paging cycle, and the target video stream associated with the video stream acquisition request is pushed to the target object.

[0065] In a possible implementation, the video stream transmission module is specifically used for:

[0066] Acquire perception data associated with the target object, wherein the perception data includes at least location information, channel state information, and behavior information;

[0067] Using the perception data to train a preset user behavior prediction model to obtain an updated target user behavior prediction model;

[0068] The perception data of the target object when currently viewing the target monitoring is input into the target user behavior prediction model to obtain the probability value of the target object viewing the target monitoring within a preset time period.

[0069] In a possible implementation, the video stream processing module is further used for:

[0070] Establishing a communication connection between the target monitoring and a base station, initiating device registration to the monitoring platform through the base station, and receiving a registration message returned by the monitoring platform after successful device registration;

[0071] Based on the registration message, the initial working mode of the target monitoring is determined, wherein the initial working mode indicates that the target monitoring and the monitoring platform operate according to a preset heartbeat keep-alive interval, and the target monitoring and the base station operate according to a preset paging cycle.

[0072] In a third aspect, the present application provides an electronic device, including:

[0073] Memory, used to store computer programs;

[0074] The processor is used to implement the steps of the above-mentioned data stream pushing method when executing the computer program stored in the memory.

[0075] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned data stream pushing method are implemented.

[0076] For each aspect from the second to the fourth aspect and the technical effects that may be achieved by each aspect, please refer to the above description of the technical effects that can be achieved by the first aspect or various possible schemes in the first aspect, and no further details will be given here. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] Figure 1 A flowchart of a data stream push method provided in an embodiment of the present application;

[0078] Figure 2 A schematic diagram of the data stream push system architecture provided in an embodiment of the present application;

[0079] Figure 3 A schematic diagram of the structure of a monitoring device provided in an embodiment of the present application;

[0080] Figure 4 A flow chart of device initialization provided in an embodiment of the present application;

[0081] Figure 5 A flowchart of user remote viewing, perception and model training provided in an embodiment of the present application;

[0082] Figure 6 A schematic diagram of streaming target monitoring in a first low power consumption working mode provided in an embodiment of the present application;

[0083] Figure 7 A schematic diagram of streaming target monitoring in a second low power consumption working mode provided by an embodiment of the present application;

[0084] Figure 8 A schematic diagram of streaming target monitoring in a third low power consumption working mode provided by an embodiment of the present application;

[0085] Fig. 9 A schematic diagram of the structure of a data stream push device provided in an embodiment of the present application;

[0086] Fig.10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0087] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The specific operating methods in the method embodiments can also be applied to device embodiments or system embodiments. It should be noted that in the description of the present application, "multiple" is understood as "at least two". "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A is connected to B, which can represent: A is directly connected to B and A is connected to B through C. In addition, in the description of the present application, words such as "first" and "second" are only used to distinguish the purpose of description, and cannot be understood as indicating or implying relative importance, nor can they be understood as indicating or implying order.

[0088] The embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0089] With the rapid development of intelligent monitoring technology, more and more monitoring equipment is widely used in various industries. Among them, monitoring equipment based on cellular (4G, 5G) networks is more widely used due to its ultra-large bandwidth and low-latency network characteristics.

[0090] On the one hand, monitoring equipment based on cellular networks does not need to lay a dedicated monitoring network. It can quickly access the external network by simply connecting the user identity card (SIM) to the monitoring device, and transmit the collected video data to the monitoring platform through the external network, realizing convenient transmission of video data.

[0091] On the other hand, with the further optimization of cellular networks and the increasing demand of users for video quality, cellular network-based monitoring equipment also makes it possible to transmit ultra-high-definition videos such as 4K and 8K.

[0092] Although cellular network-based monitoring devices have the above advantages, most of these monitoring devices are battery-based monitoring devices, and users usually have high requirements on the usage time of monitoring devices. Therefore, how to reduce the power consumption of such monitoring devices is crucial.

[0093] In the traditional solution to reduce the power consumption of monitoring equipment, there is a method of obtaining infrared radiation signal information in each preset area through the passive infrared (PIR) sensor in the target hunting camera, and converting the infrared radiation signal information into a preset type of electrical signal; if the threshold of the preset type of electrical signal is greater than the preset threshold, the PIR sensor generates a trigger signal, and the target hunting camera is started according to the trigger signal, and the video frame information of the preset area is obtained through the target hunting camera, and the video frame information is recognized and processed. If the recognition result is the first recognition result, the target hunting camera is controlled to be turned off; if the recognition result is the second recognition result, an alarm message is generated. Although this solution can filter out some PIR sensor false alarm problems and realize rapid recognition of target prey by using the PIR sensor to recognize the object and then filtering it twice through the image AI, the perception range of the PIR sensor is limited and long-distance perception cannot be achieved. In addition, this method requires the image and PIR sensor to work together for each recognition, and the power consumption of the whole device is still relatively large.

[0094] In addition, traditional solutions to reduce the power consumption of monitoring equipment usually put the monitoring device into a sleep state to reduce power consumption when the user or monitoring application (APP) does not initiate a streaming request to the monitoring device. When the user clicks on the preview screen through the monitoring application, the monitoring device will be awakened, but this wake-up action usually takes 3-10 seconds. Currently, it is impossible to guarantee the real-time effect of users clicking on the browsing screen to quickly stream under low power consumption.

[0095] In view of this, in order to solve the problem that the overall power consumption of low-power devices based on PIR perception is high and the remote APP randomly wakes up the monitoring device and consumes time to output the stream, the present application provides a data stream push method, which specifically includes: first obtaining the probability value of the target object viewing the target monitoring within a preset time period, and then determining the working mode of the target monitoring based on the numerical interval to which the probability value belongs, and finally pushing the video stream to the target object according to the video stream push rules corresponding to the working mode of the target monitoring.

[0096] This application obtains perception information related to the user or monitoring APP, and performs real-time AI analysis in combination with the monitoring screen information. It can accurately predict the probability value of the target object viewing the target monitoring within a preset time period, and then select different low-power modes according to the probability value, thereby reducing the hardware modules powered by the monitoring device and the number of wake-up times, thereby achieving the purpose of reducing device power consumption; by waking up the device in advance when the probability value of the user using the monitoring APP to view the target monitoring within the preset time period is greater than 99%, the user can open the monitoring APP and immediately start the flow, and the wake-up delay can be reduced to 0 seconds; when the probability value is lower than 99%, the heartbeat keep-alive interval and paging cycle are dynamically adjusted to follow the change of the probability value to achieve dual optimization of the power consumption and flow timeliness of the monitoring device; in case of AI training platform prediction errors, a remedial mechanism is designed (timed wake-up of the monitoring device) to avoid the long-term inability to pull the APP when the device is in low-power mode, and at the same time, the training results are adjusted based on the latest perception data to update the working mode of the monitoring device.

[0097] Reference Figure 1 As shown, it is a flow chart of a data stream pushing method provided in an embodiment of the present application, the method comprising:

[0098] S1, obtaining a probability value of a target object viewing a target monitoring object within a preset time period.

[0099] First of all, the method provided in this application can be applied to Figure 2 In the system architecture shown, the system architecture includes: a target object, a monitoring device, a base station, a video monitoring platform and an AI training platform, and the method provided in the present application can be run in the monitoring device.

[0100] The following is a brief introduction to the functions of the above devices and platforms:

[0101] The target object can be a user or a monitoring application (remote APP). When the monitoring application is running, it will collect the perception data of the surrounding area (WIFI perception area) in real time and upload the perception data to the AI ​​training platform. The perception data includes but is not limited to: location information, channel status (WIFI CSI) information and behavior information.

[0102] The monitoring device is used to upload the collected video or image information to the video monitoring platform in real time. In the present application, the monitoring device can upload the video stream to the video monitoring platform through the base station. The base station can be a fourth-generation mobile communication base station, a fifth-generation mobile communication base station or a sixth-generation mobile communication base station. The present application does not impose specific restrictions on the number of monitoring devices and the type of base stations, and the target monitoring can be any one of the monitoring devices.

[0103] The video surveillance platform is used to record the video stream uploaded by the monitoring device and forward the video stream to the remote APP.

[0104] The AI ​​training platform is used to train the user behavior prediction model based on the perception data collected by the remote APP. During the training process, through AI analysis of the user's location, WIFI CSI information, and the screen information previewed by the user, the exact environment in which the user opens the remote APP (for example, the number of people around, viewing posture, whether there are elderly people or children, etc.) can be identified, and the monitoring screen displayed on the APP side combined with AI analysis can identify the information that the user is concerned about (such as pets, elderly people, children or orchards, etc.). The AI ​​training platform combines the AI ​​analysis perception of user behavior to predict the probability value of the user opening the APP to remotely view the monitoring device in the future time period (T).

[0105] In the embodiments of the present application, as shown in the attached Figure 3 As shown, the monitoring equipment can be specifically divided into the following components: chip-level SOC system, image sensor, low-power microcontroller unit, power management and cellular module. The chip-level SOC system is the main processor of the monitoring equipment. The system integrates multiple software modules and is used to control peripheral hardware entities. The multiple software modules include: image processing module, network transmission module and power management module. The image processing module realizes the image acquisition, encoding and identification of whether there is an object moving in the current warning area of ​​the monitoring information by controlling the image sensor in the hardware entity; the network transmission module is responsible for initializing the cellular module in the hardware entity, establishing a network connection between the cellular module and the base station and the monitoring platform, and then responsible for transmitting the collected video stream to the corresponding monitoring platform; the power management module is responsible for powering on or off other hardware modules in the monitoring equipment to control the power consumption of the equipment.

[0106] In the embodiments of this application, see the attached Figure 4 As shown, before the monitoring device obtains the probability value of the target object viewing the target monitoring within the preset time period, it is first necessary to power on and initialize the monitoring device, and then establish a communication connection between the cellular module of the monitoring device and the base station. After the connection is established, the monitoring device can initiate device registration to the video monitoring platform through the base station and obtain the current working mode. After completing the registration of the monitoring device, the video monitoring platform can initiate a prediction request to the AI ​​training platform to obtain the probability value of the user viewing the target monitoring within the preset time period at the current moment, and return a registration message to the monitoring device, wherein the registration message includes the probability value of the user viewing the target monitoring within the preset time period at the current moment.

[0107] Based on the received registration message, the monitoring device can determine the initial working state of the target monitoring (default low-power working mode). In this application, because the prediction accuracy of the AI ​​training platform is low during the early installation and use stage of the monitoring device, the probability value of the initial output of the AI ​​training platform can be set to 0, and the working mode corresponding to the probability value can be set to the default initial working mode. The initial working mode can be that the target monitoring and the monitoring platform operate according to the preset heartbeat keep-alive interval, and the target monitoring and the base station operate according to the preset paging cycle (EDRX). The heartbeat keep-alive interval indicates the time interval for the target monitoring to periodically send heartbeat packets to the video monitoring platform; the paging cycle indicates the periodic time for the target monitoring to periodically check whether there is a paging message in idle mode.

[0108] Through the above method, the monitoring device can be initialized, so that the monitoring device can operate in a more moderate power consumption mode, thereby reducing the power consumption of the monitoring device.

[0109] In the embodiments of this application, see the attached Figure 5 As shown, the monitoring device can specifically obtain the probability value of the target object viewing the target monitoring within the preset time period in the following manner:

[0110] When the user opens the APP to view the real-time monitoring screen, the AI ​​training platform can obtain the perception data associated with the target object (WIFI CSI perception data, user location, time and GNSS location information), and then divide the perception data into training set, validation set and test set. The training set is used to train the preset user behavior prediction model (based on the perception data regularly reported by the APP, the AI ​​training set is continuously updated and the accuracy of the APP preview time is updated). The validation set is used to adjust the model parameters and the test set is used to evaluate the performance of the model to obtain a target user behavior prediction model whose prediction accuracy meets the current needs; then the perception data of the user currently viewing the target monitoring is input into the target user behavior prediction model to obtain the probability value of the target object viewing the target monitoring within the preset time period.

[0111] Based on the above method, different probability values ​​of users in different locations viewing the target monitoring within a preset time period can be obtained, which facilitates the subsequent selection of different low-power operation strategies based on different probability values.

[0112] S2, based on the numerical interval to which the probability value belongs, determines the working mode of target monitoring.

[0113] In an embodiment of the present application, in order to achieve the real-time effect of users clicking on a remote APP to stream data while reducing the power consumption of the monitoring device, the monitoring terminal can determine the working mode of the target monitoring based on the numerical interval to which the probability value belongs.

[0114] Specifically, when the probability value output by the AI ​​training platform belongs to the first numerical interval (e.g., 0-0.01), the value indicates that the possibility (probability) of the user opening the remote APP to view the target monitoring within the preset time period T is less than 1%. Therefore, the working mode of the target monitoring can be switched from the initial working mode to the first low-power working mode, and run based on the first low-power working mode. The first low-power working mode is a deep low-power working mode of the monitoring device, which can reduce the power consumption of the device as much as possible. In addition, in order to prevent the monitoring device from excessive sleep, in the first low-power working mode, the low-power microcontroller unit MCU of the monitoring device will set the RTC timed wake-up (timed power-on) function. This application does not impose specific restrictions on the size of the timing period.

[0115] When the probability value output by the AI ​​training platform belongs to the second numerical range (e.g., 0.99-1), the value indicates that the possibility (probability) of the user opening the remote APP to view the target monitoring within the preset time period T is greater than 99%. Therefore, the working mode of the target monitoring can be switched from the initial working mode to the second low-power working mode, and run based on the second low-power working mode. In the non-sleep stage of the second low-power working mode, it can ensure that when the user opens the remote APP, the monitoring device can push the video stream to the user in time without waking up the device.

[0116] When the probability value output by the AI ​​training platform belongs to the third numerical range (e.g., 0.01-0.99), the value indicates that the possibility (probability) of the user opening the remote APP to view the target monitoring within the preset time period T is higher than 1%, but lower than 99%. Therefore, the working mode of the target monitoring can be switched from the initial working mode to the third low-power working mode, and run based on the third low-power working mode. The third low-power working mode is an optimized version of the initial working mode. In the third low-power working mode, the heartbeat interval S1 between the target monitoring and the monitoring platform and the paging cycle EDRX between the cellular module and the base station will be dynamically adjusted based on the probability value K predicted by the current AI training platform that the user will view the target monitoring based on the APP in the time period T. Obviously, the larger the K value, the smaller the S1 value and the smaller the EDRX, which can ensure that the APP or user's pull request is transmitted to the device with the lowest delay. The smaller the K value, the less likely the user is to open the APP within the set time period T. In order to ensure that the power consumption of the monitoring equipment can be reduced to a lower level, this application will increase the values ​​of S1 and EDRX to reduce the number of wake-ups of the cellular module AP core and the number of wake-ups of the cellular module baseband, thereby ultimately reducing the power consumption of the cellular module.

[0117] Through the above method, different low-power operation modes are selected based on the numerical interval to which the probability value belongs, so that the streaming request sent by the APP or user can be transmitted to the device end with the lowest delay while reducing the power consumption of the monitoring device.

[0118] S3, pushing the video stream to the target object according to the video stream pushing rule corresponding to the target monitoring working mode.

[0119] In the embodiment of the present application, the specific process of pushing the video stream to the target object according to the video stream push rule corresponding to the target monitoring working mode is as follows:

[0120] The first low power mode, see Appendix Figure 6 As shown, the monitoring platform will receive and update the prediction results output by the AI ​​training platform in real time; if the AI ​​training platform predicts that the probability of the user opening the remote APP to view the target monitoring within the preset time period T is less than 1%, the monitoring platform can notify the monitoring device to enter the first low-power working mode. When the monitoring device is running in the first low-power mode, the target monitoring can be set to a sleep state, wherein the sleep state means waking up the main control hardware of the target monitoring according to a preset first duration, and powering off other hardware of the target monitoring; exemplarily, when the monitoring device is in a sleep state, the low-power microcontroller unit (MCU) in the monitoring device can be awakened once according to a preset timed power-on duration (first duration T2), and during this period, all other hardware peripherals of the monitoring device are powered off to maintain minimum power consumption.

[0121] To prevent the target monitoring from being in a dormant state for a long time and unable to respond to the user's video stream acquisition request, the target monitoring can determine whether the current duration has reached the first duration; if it has reached, the target monitoring can re-power on the main control hardware and other hardware, and then query whether there is a missed wake-up message within the first duration (that is, determine whether the video stream acquisition request sent by the target object is received within the first duration). When it is determined that the video stream acquisition request is received, the target monitoring can push the target video stream associated with the video stream acquisition request to the user, and adjust the working mode of the target monitoring based on the first updated value of the probability value (the latest prediction result of the AI ​​training platform); if the T2 time arrives, the target monitoring and the monitoring platform establish a connection, and the monitoring platform has no missed wake-up event for the device, the device will continue to be in the first low-power working mode until the T1 time arrives to exit the mode. If it is not reached, the target monitoring can remain in a dormant state.

[0122] In one possible implementation, if the target monitoring is running in the first low power consumption mode and is in a dormant state, but at this time the user initiates a video stream acquisition request for the target monitoring (i.e., a prediction error), then the monitoring platform will notify the remote APP to obtain an indication message of the target video stream abnormality, cache the video stream acquisition request, wait until T2 time is reached, and then reconnect to the monitoring platform to supplement the video stream; in addition, the AI ​​training platform will also synchronize the current APP perception data, determine that the current prediction is invalid based on the synchronization time point, update the prediction accuracy and retrain the user behavior prediction model.

[0123] In response to the situation where the AI ​​training platform makes prediction errors, this application designs a remedial mechanism (periodic wake-up of the monitoring device) to avoid the long-term inability to pull APP traffic when the device is in the first low-power mode. At the same time, the training results are adjusted based on the latest perception data to update the working mode of the monitoring device.

[0124] The second low-power working mode: if the AI ​​training platform predicts that the probability that the user opens the remote APP to view the target monitoring within the preset time period T is greater than or equal to 99%, the target monitoring can be set to the awake state in advance, and the awake state means that all the hardware of the target monitoring is powered on; then determine whether the video stream acquisition request sent by the user is received within the preset time period. If so, the target monitoring can directly push the target video stream associated with the video stream acquisition request to the user; if not, the target monitoring can obtain a second updated value of the probability value, and adjust the working mode of the target monitoring based on the second updated value.

[0125] For example, see the attached Figure 7 As shown in the figure, if the AI ​​training platform predicts that the probability that the user opens the remote APP to view the target monitoring within the preset time period T is greater than or equal to 99%, the monitoring platform will wake up the target monitoring in advance before time T is reached, so that the target monitoring enters the video acquisition and encoding stage in advance, waiting for the arrival of the APP streaming request sent by the user. When the user opens the APP within the time period T, the target monitoring will directly transmit the code stream without any video delay. If the APP streaming request sent by the user is not received within the time period T, the low-power working mode of the device can be adjusted to sleep, and the AI ​​training platform will recalculate based on the latest data and update the probability value of the time period.

[0126] By waking up the target monitoring in advance when the probability value of the user using the monitoring APP to view the target monitoring within the preset time period is greater than 99%, the user can push the video stream immediately when opening the monitoring APP, which will greatly reduce the wake-up delay of the target monitoring.

[0127] The third low power mode, see Appendix Figure 8As shown, if the AI ​​training platform predicts that the probability that the user opens the remote APP to view the target monitoring within the preset time period T is greater than 1% and less than 99%, the heartbeat keep-alive interval and paging cycle of the target monitoring within the current time period can be adjusted based on the current probability value of the target object viewing the target monitoring within the current time period, where the heartbeat keep-alive interval represents the time interval for the target monitoring to periodically send heartbeat packets to the monitoring platform; the paging cycle represents the cycle time for the target monitoring to regularly check whether there is a paging message in idle mode, and at this time the target monitoring runs based on the new heartbeat keep-alive interval and the new paging cycle; when the target monitoring determines that a video stream acquisition request has been received, the target monitoring device is awakened according to the heartbeat keep-alive interval and the paging cycle, and the target video stream associated with the video stream acquisition request is pushed to the target object.

[0128] To summarize, the method provided in the present application, by acquiring perception information related to the user or monitoring APP and performing real-time AI analysis in combination with the monitoring screen information, can accurately predict the probability value of the target object viewing the target monitoring within a preset time period, and then select different low-power modes according to the probability value, thereby reducing the hardware modules and wake-up times powered by the monitoring device, thereby achieving the purpose of reducing the power consumption of the device; by waking up the device in advance when the probability value is greater than 99%, the user can open the monitoring APP and immediately start streaming, greatly reducing the wake-up delay; when the probability value is higher than 1% and lower than 99%, the heartbeat keep-alive interval and paging cycle are dynamically adjusted to achieve dual optimization of the power consumption and timeliness of the monitoring device by following the changes in the probability value; in response to the prediction errors of the AI ​​training platform, a remedial mechanism is designed (timed wake-up of the monitoring device) to avoid the long-term inability to pull APP traffic when the device is in low-power mode, and at the same time, the training results are adjusted based on the latest perception data to update the working mode of the monitoring device.

[0129] Based on the method provided in the above embodiment, the embodiment of the present application also provides a video stream pushing device, such as Fig. 9 The figure is a schematic diagram of the structure of a video stream pushing device in an embodiment of the present application, the device comprising:

[0130] The video stream processing module 901 is used to collect the video stream of the monitoring area;

[0131] The video stream transmission module 902 is used to obtain the probability value of the target object viewing the target monitoring within a preset time period;

[0132] The power consumption management module 903 is used to determine the working mode of the target monitoring based on the numerical interval to which the probability value belongs;

[0133] The video stream is pushed to the target object according to the video stream pushing rule corresponding to the working mode of the target monitoring.

[0134] In a possible implementation, the power consumption management module 903 is specifically used to:

[0135] If the probability value belongs to the first numerical interval, switching the target monitoring working mode from the initial working mode to the first low power consumption working mode, and operating based on the first low power consumption working mode;

[0136] If the probability value belongs to the second numerical interval, switching the target monitoring working mode from the initial working mode to the second low power consumption working mode, and operating based on the second low power consumption working mode;

[0137] If the probability value belongs to the third numerical interval, the target monitoring working mode is switched from the initial working mode to the third low power consumption working mode, and operates based on the third low power consumption working mode.

[0138] In a possible implementation, the power consumption management module 903 is specifically used to:

[0139] Setting the target monitoring to a dormant state, wherein the dormant state means waking up the main control hardware of the target monitoring periodically according to a preset first duration, and powering off other hardware of the target monitoring;

[0140] Determining whether the current duration reaches the first duration;

[0141] If so, the master control hardware and the other hardware are powered on, and it is determined whether a video stream acquisition request initiated by the target object is received within the first time period. When it is determined that the video stream acquisition request is received, the target video stream associated with the video stream acquisition request is pushed to the target object, and the working mode of the target monitoring is adjusted based on the first updated value of the probability value;

[0142] If not, the sleep state is maintained.

[0143] In a possible implementation, the power consumption management module 903 is specifically used to:

[0144] Setting the target monitoring to a wake-up state, wherein the wake-up state means powering on all hardware of the target monitoring;

[0145] Determining whether a video stream acquisition request initiated by the target object is received within a preset time period;

[0146] If yes, push the target video stream associated with the video stream acquisition request to the target object;

[0147] If not, a second updated value of the probability value is obtained, and the working mode of the target monitoring is adjusted based on the second updated value.

[0148] In a possible implementation, the power consumption management module 903 is specifically used to:

[0149] Based on the current probability value of the target object viewing the target monitoring in the current time period, adjust the heartbeat keep-alive interval and paging cycle of the target monitoring in the current time period, wherein the heartbeat keep-alive interval represents the time interval for the target monitoring to periodically send heartbeat packets to the monitoring platform; the paging cycle represents the periodic time for the target monitoring to regularly check whether there is a paging message in idle mode;

[0150] When it is determined that the video stream acquisition request initiated by the target object is received, the target monitoring device is woken up according to the heartbeat keep-alive interval and the paging cycle, and the target video stream associated with the video stream acquisition request is pushed to the target object.

[0151] In a possible implementation, the video stream transmission module 902 is specifically used for:

[0152] Acquire perception data associated with the target object, wherein the perception data includes at least location information, channel state information, and behavior information;

[0153] Using the perception data to train a preset user behavior prediction model to obtain an updated target user behavior prediction model;

[0154] The perception data of the target object when currently viewing the target monitoring is input into the target user behavior prediction model to obtain the probability value of the target object viewing the target monitoring within a preset time period.

[0155] In a possible implementation, the video stream processing module 901 is further used for:

[0156] Establishing a communication connection between the target monitoring and a base station, initiating device registration to the monitoring platform through the base station, and receiving a registration message returned by the monitoring platform after successful device registration;

[0157] Based on the registration message, the initial working mode of the target monitoring is determined, wherein the initial working mode indicates that the target monitoring and the monitoring platform operate according to a preset heartbeat keep-alive interval, and the target monitoring and the base station operate according to a preset paging cycle.

[0158] Based on the same inventive concept, an electronic device is also provided in the embodiment of the present application, and the electronic device can realize the function of the aforementioned data stream pushing device, referring to Fig.10 , the electronic device comprises:

[0159] At least one processor 1001, and a memory 1002 connected to the at least one processor 1001. The specific connection medium between the processor 1001 and the memory 1002 is not limited in the embodiment of the present application. Fig.10 In the example, the processor 1001 and the memory 1002 are connected via the bus 1000. Fig.10 The connections between other components are shown in bold lines, and are not intended to be limiting. The bus 1000 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.10 Only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. Alternatively, the processor 1001 can also be called a controller, and there is no limitation on the name.

[0160] In the embodiment of the present application, the memory 1002 stores instructions that can be executed by at least one processor 1001. The at least one processor 1001 can execute the data stream push method discussed above by executing the instructions stored in the memory 1002. The processor 1001 can implement Fig. 9 The functions of each module in the device shown.

[0161] Among them, processor 1001 is the control center of the device, and can use various interfaces and lines to connect various parts of the entire control device. By running or executing instructions stored in memory 1002 and calling data stored in memory 1002, various functions of the device and processing data, the device can be monitored as a whole.

[0162] In one possible design, the processor 1001 may include one or more processing units, and the processor 1001 may integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the modem processor may not be integrated into the processor 1001. In some embodiments, the processor 1001 and the memory 1002 may be implemented on the same chip, and in some embodiments, they may also be implemented separately on separate chips.

[0163] Processor 1001 can be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, and can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the data stream push method disclosed in the embodiments of the present application can be directly embodied as a hardware processor execution, or a combination of hardware and software modules in the processor.

[0164] The memory 1002 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory 1002 may include at least one type of storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory, a random access memory (Random Access Memory, RAM), a static random access memory (Static Random Access Memory, SRAM), a programmable read-only memory (Programmable Read Only Memory, PROM), a read-only memory (Read Only Memory, ROM), an electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), a magnetic memory, a disk, an optical disk, etc. The memory 1002 is any other medium that can be used to carry or store a desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 1002 in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.

[0165] By designing and programming the processor 1001, the code corresponding to the data stream push method described in the above embodiment can be fixed into the chip, so that the chip can execute the code when running. Figure 1 The steps of the data stream pushing method of the embodiment shown are as follows: How to design and program the processor 1001 is a technology well known to those skilled in the art and will not be described in detail here.

[0166] Based on the same inventive concept, an embodiment of the present application further provides a storage medium, which stores computer instructions. When the computer instructions are executed on a computer, the computer executes the data stream pushing method discussed above.

[0167] In some possible implementations, various aspects of the data stream push method provided by the present application may also be implemented in the form of a program product, which includes program code. When the program product is run on an apparatus, the program code is used to enable the control device to execute the steps of the data stream push method according to various exemplary implementations of the present application described above in this specification.

[0168] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0169] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0170] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0171] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0172] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A video stream pushing method, characterized in that: include: Obtain the probability value of the target object viewing the target monitoring within a preset time period; Determine the target monitoring working mode based on the numerical interval to which the probability value belongs, and push the video stream to the target object according to the video stream pushing rule corresponding to the target monitoring working mode; If the working mode of the target monitoring is the first low power consumption mode, the target monitoring is set to a sleep state, wherein the sleep state means that the main control hardware of the target monitoring is awakened regularly according to a preset first duration, and other hardware of the target monitoring is powered off; Determining whether the current duration reaches the first duration; If so, the master control hardware and the other hardware are powered on, and it is determined whether a video stream acquisition request initiated by the target object is received within the first time period. When it is determined that the video stream acquisition request is received, the target video stream associated with the video stream acquisition request is pushed to the target object, and the working mode of the target monitoring is adjusted based on the first updated value of the probability value; If not, the sleep state is maintained.

2. The method according to claim 1, characterized in that The determining the target monitoring working mode based on the numerical interval to which the probability value belongs includes: If the probability value belongs to the first numerical interval, switching the target monitoring working mode from the initial working mode to the first low power consumption working mode, and operating based on the first low power consumption working mode; If the probability value belongs to the second numerical interval, switching the target monitoring working mode from the initial working mode to the second low power consumption working mode, and operating based on the second low power consumption working mode; If the probability value belongs to the third numerical interval, the target monitoring working mode is switched from the initial working mode to the third low power consumption working mode, and operates based on the third low power consumption working mode.

3. The method according to claim 2, characterized in that If the target monitoring is operated in the second low power consumption working mode, the step of pushing the video stream to the target object according to the video stream pushing rule corresponding to the working mode of the target monitoring includes: Setting the target monitoring to a wake-up state, wherein the wake-up state means powering on all hardware of the target monitoring; Determining whether a video stream acquisition request initiated by the target object is received within a preset time period; If yes, push the target video stream associated with the video stream acquisition request to the target object; If not, a second updated value of the probability value is obtained, and the working mode of the target monitoring is adjusted based on the second updated value.

4. The method according to claim 2, characterized in that If the target monitoring is operated in the third low power consumption working mode, the step of pushing the video stream to the target object according to the video stream pushing rule corresponding to the working mode of the target monitoring includes: Based on the current probability value of the target object viewing the target monitoring in the current time period, adjust the heartbeat keep-alive interval and paging cycle of the target monitoring in the current time period, wherein the heartbeat keep-alive interval represents the time interval for the target monitoring to periodically send heartbeat packets to the monitoring platform; the paging cycle represents the periodic time for the target monitoring to regularly check whether there is a paging message in idle mode; When it is determined that the video stream acquisition request initiated by the target object is received, the target monitoring device is woken up according to the heartbeat keep-alive interval and the paging cycle, and the target video stream associated with the video stream acquisition request is pushed to the target object.

5. The method according to claim 1, characterized in that The obtaining of the probability value of the target object viewing the target monitoring within a preset time period includes: Acquire perception data associated with the target object, wherein the perception data includes at least location information, channel state information, and behavior information; Using the perception data to train a preset user behavior prediction model to obtain an updated target user behavior prediction model; The perception data of the target object when currently viewing the target monitoring is input into the target user behavior prediction model to obtain the probability value of the target object viewing the target monitoring within a preset time period.

6. The method according to any one of claims 1 to 5, characterized in that: Before obtaining the probability value of the target object viewing the target monitoring within a preset time period, the method further includes: Establishing a communication connection between the target monitoring and a base station, initiating device registration to the monitoring platform through the base station, and receiving a registration message returned by the monitoring platform after successful device registration; Based on the registration message, the initial working mode of the target monitoring is determined, wherein the initial working mode indicates that the target monitoring and the monitoring platform operate according to a preset heartbeat keep-alive interval, and the target monitoring and the base station operate according to a preset paging cycle.

7. A video stream pushing device, characterized in that: include: Video stream processing module, used to collect video streams of the monitoring area; The video stream transmission module is used to obtain the probability value of the target object viewing the target monitoring within a preset time period; A power consumption management module, configured to determine a working mode of the target monitoring based on the numerical interval to which the probability value belongs, and push a video stream to the target object according to a video stream push rule corresponding to the working mode of the target monitoring; If the working mode of the target monitoring is the first low power consumption mode, the target monitoring is set to a sleep state, wherein the sleep state means that the main control hardware of the target monitoring is awakened regularly according to a preset first duration, and other hardware of the target monitoring is powered off; Determining whether the current duration reaches the first duration; If so, the master control hardware and the other hardware are powered on, and it is determined whether a video stream acquisition request initiated by the target object is received within the first time period. When it is determined that the video stream acquisition request is received, the target video stream associated with the video stream acquisition request is pushed to the target object, and the working mode of the target monitoring is adjusted based on the first updated value of the probability value; If not, the sleep state is maintained.

8. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to implement the method steps of any one of claims 1 to 6 when executing the computer program stored in the memory.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of claims 1 to 6 are implemented.

Images