Low-power-consumption control method and equipment for monitoring equipment and storage medium

Abstract

The invention discloses a low-power-consumption control method and equipment for monitoring equipment and a storage medium, and belongs to the technical field of video monitoring. The method comprises the following steps: processing a received image based on an RTOS system to obtain image processing data; the image processing data is recognized based on an image recognition algorithm, system scheduling of a data processing end is adjusted according to an image recognition result, and the system scheduling comprises dormancy of RTOS or operation of a Linux system; and if the system scheduling is to run Linux, encoding the image processing data to obtain target image data. According to the method, the image data collected by the RTOS is further accurately recognized through an image recognition algorithm, and if the target event is the target event, Linux is operated and a 4G network is opened to operate a video storage program, so that more complex intelligent applications are operated; otherwise, stopping the RTOS system and enabling the RTOS system to enter dormancy, so as to reduce the occurrence of mistaken touch conditions, realize accurate task scheduling and improve the validity of the collected data.

Description

Technical Field

[0001] The present application relates to the field of video surveillance technology, and in particular to a low power consumption control method, device and storage medium for a monitoring device. Background Art

[0002] At present, the human or vehicle recognition equipment on the market requires the camera to be turned on all the time, and recognition begins when a person or vehicle enters the screen. It consumes high power, occupies high storage space, and occupies high bandwidth traffic. Low-power cameras have become an important tool for real-time image monitoring. In related technologies, the camera enters a low-power sleep state, suspends image data acquisition and transmission, and wakes up the camera for recording only after the arranged sensor detects the motion information of the object. However, no matter what object motion information is detected, the camera is awakened, and the data collected after awakening cannot guarantee whether there is personal information. Therefore, in the process of video image monitoring, the validity of the data cannot be guaranteed, resulting in low validity of the acquired data.

[0003] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0004] The main purpose of this application is to provide a low-power control method, device and storage medium for monitoring equipment, aiming to solve the technical problem that after detecting the motion information of an object, the cameras are awakened, and the data collected after awakening cannot be guaranteed whether there is personnel information or the validity of the data, resulting in low validity of the acquired data.

[0005] To achieve the above purpose, the present application provides a low-power control method for monitoring equipment, the method comprising the following steps: quickly starting and processing the received image based on the RTOS system to obtain image processing data; identifying the image processing data based on the image recognition algorithm, and adjusting the system scheduling of the data processing end according to the image recognition result, the system scheduling includes hibernating the RTOS system or running the Linux system; if the system scheduling is to run the Linux system, encoding the image processing data to obtain the target image data. The above RTOS operation cycle is completed within 150 milliseconds.

[0006] In one embodiment, the image processing data is identified based on an image recognition algorithm, and the system scheduling of the data processing end is adjusted according to the image recognition result, and the system scheduling includes the steps of hibernating the RTOS system or running the Linux system, including: Determine the configuration of the RTOS system based on functional requirements, and determine the image recognition algorithm based on the configuration; Performing image recognition on the image processing data according to the image recognition algorithm to determine the image recognition result; If the image recognition result does not include a preset target, determining that the system is scheduled to hibernate the RTOS system; If the image recognition result includes a preset target, the system is determined to be scheduled to run a Linux system.

[0007] In one embodiment, if the image recognition result does not include a preset target, the step of determining that the system is scheduled to hibernate the RTOS system includes: If the image recognition result does not include the preset target, release the resources occupied by the RTOS system and enter a low power consumption mode, wherein the resources include memory, cache and hardware resources; A wake-up condition is set for the RTOS system in the low power consumption mode to put the RTOS system into hibernation.

[0008] In one embodiment, if the system is scheduled to run the Linux system, the step of encoding the image processing data to obtain the target image data includes: Initializing an encoder to encode the image processing data and determine first image data; Encoding the received real-time image data through the Linux system to determine the second image data; The first image data and the second image data are combined to obtain target image data.

[0009] In one embodiment, the step of processing the received image based on the RTOS system to obtain image processing data includes: Initialize the image signal processor; The image is processed based on the initialized image signal processor to obtain image processing data.

[0010] In one embodiment, before the step of processing the received image based on the RTOS system to obtain image processing data, the step further includes: Receive a trigger signal generated by a preset sensor, decode the trigger signal, and determine valid information; The valid information is identified and analyzed, and the wake-up instruction is sent to the RTOS system to run the RTOS system at the data processing end.

[0011] In one embodiment, if the system is scheduled to run the Linux system, after the step of encoding the image processing data to obtain the target image data, the method further includes: Based on the collected target image data, using the image recognition algorithm to identify key information in the image; Detecting a moving path of a monitored target based on the key information; The moving path is uploaded to the host computer through 4G cloud technology, so that the host computer can wake up the Linux systems of other monitoring devices on the moving path to achieve joint control and joint defense of at least two target monitoring devices.

[0012] In one embodiment, after the step of using the image recognition algorithm to identify key information in the image based on the collected target image data, the method further includes: Detecting the key information to determine abnormal conditions in the target image data; Based on the abnormal situation, the abnormal information is sent to the detection personnel.

[0013] In addition, to achieve the above-mentioned purpose, the present application also provides a low-power consumption control device for a monitoring device, the device comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor, the computer program being configured to implement the steps of the low-power consumption control method for the monitoring device as described above.

[0014] In addition, to achieve the above-mentioned purpose, the present application also provides a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the low power consumption control method of the monitoring device as described above are implemented.

[0015] One or more technical solutions proposed in the present application have at least the following technical effects: the present application processes the received image based on the RTOS system to obtain image processing data, then identifies the image processing data according to the image recognition algorithm, and then adjusts the system scheduling of the data processing end according to the image recognition result; if the system is scheduled to run the Linux system, the image processing data is encoded to obtain the target image data, and there is a judgment mechanism for the high power consumption detection service, which reduces the occurrence of false touches and effectively improves the validity of the collected data. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings herein are incorporated in and constitute a part of the specification, illustrate embodiments consistent with the present application, and together with the description, are used to explain the principles of the present application.

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0018] Figure 1 A schematic diagram of a flow chart of a first embodiment of a low power consumption control method for a monitoring device of the present application; Figure 2 A flowchart of a second embodiment of a low power consumption control method for a monitoring device of the present application; Figure 3 A schematic flow chart of a third embodiment of a low power consumption control method for a monitoring device of the present application; Figure 4 A flowchart of a fourth embodiment of a low power consumption control method for a monitoring device of the present application; Figure 5 A flowchart of a fifth embodiment of a low power consumption control method for a monitoring device of the present application; Figure 6 A flowchart of a sixth embodiment of a low power consumption control method for a monitoring device of the present application; Figure 7 The resource scheduling diagram corresponding to the low power consumption control method of the monitoring device of this application; Figure 8 It is a structural diagram of a low-power control device of a monitoring device of a hardware operating environment involved in an embodiment of the present application.

[0019] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0020] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0021] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0022] The fastest and most convenient way to install network surveillance cameras in inconvenient places is to use battery power and mobile networks, which requires the device to have low power consumption capabilities. Although related technologies already exist to wake up the device through sensors, turn on recording and 4G networks to reduce power consumption.

[0023] However, this situation is easily disturbed. Generally, the chip takes a long time to start up, and it takes 6 seconds for the image to be normal. Often, the recording starts but the event has already passed. At the same time, false triggers will also record. For example, the infrared sensor detects the presence of a dynamic object, and the temperature sensor determines whether it is a biological organism. On this basis, it is impossible to determine whether it is human movement or an animal false trigger. After the trigger, the recording has already started and cannot be interrupted and adjusted in time. This will result in additional storage or network bandwidth usage, causing unnecessary energy loss and less effective data collected.

[0024] In order to solve the above problems, the present application proposes a low-power control method for monitoring equipment. When the microwave radar detects that an object has entered its detection range, a trigger signal is transmitted to the low-power control system of the monitoring equipment. After the first processor (CPU1) in the system receives the trigger signal, it immediately loads a first-stage real-time operating system (Real TimeOperate The system receives the first-stage RTOS from the CPU1, initializes the image signal processor (ISP), starts collecting images, and stores the collected images in the designated running memory space. At the same time, after determining that the radar wakes up the system, CPU1 starts loading the second-stage RTOS system, and then CPU1 hands the second-stage RTOS system to CPU2 while loading the Uboot and Linux systems. If the neural network processor (NPU) of CPU2 recognizes that there are people or vehicles in the first-stage photo stored in the designated running memory space according to the recognition algorithm, the RTOS system exits and actively releases the control of CPU2. At the same time, CPU2 loads the Linux system. After CPU2 starts the Linux system, it will initialize the encoder, read the video data from the designated running memory space for encoding, and then encode the real-time received video data, and customize the alarm information to provide accurate monitoring reminders. If CPU2 does not recognize a person or vehicle from the photo, the system will enter a dormant state to reduce unnecessary power consumption.

[0025] Optionally, when awakened by 4G, CPU2 directly runs the Linux system, cooperates with other monitoring devices for joint control and joint defense, and starts high-power network service applications at the same time. This method runs the human recognition algorithm, and after detecting human information, it starts encoding and recording, loads Linux and wakes up the 4G network to push alarm information according to the settings. Behaviors that appear on the screen for 0.2 seconds can be captured, filtering 50% of false awakenings without people and vehicles, and improving the effectiveness of the collected data.

[0026] In order to better understand the above technical solution, exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0027] Based on this, the first embodiment of the present application provides a low power consumption control method for a monitoring device, referring to Figure 1 , Figure 1 This is a flow chart of Embodiment 1 of the low power consumption control method for monitoring equipment of the present application.

[0028] In this embodiment, the low power consumption control method of the monitoring device includes steps S100 to S300: Step S100: Process the received image based on the RTOS system to obtain image processing data.

[0029] It should be noted that the execution subject of this embodiment can be a low-power control device of a monitoring device, or a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, etc., and this embodiment does not specifically limit this. The following takes the low-power control device of a monitoring device as an example to illustrate this embodiment and the following embodiments.

[0030] When passively awakened by the human body radar or PIR device, the RTOS system starts quickly, receives the wake-up command and image transmitted by the sensor, and stores the image in the specified space of the running memory. CPU1 immediately loads the first-stage RTOS system, and then passes the loaded first-stage RTOS system to CPU2. The started RTOS system will first initialize ISP, including fill light control, buffer construction, etc., and then process the received image and store the processed image in the specified running memory space.

[0031] It should be noted that RTOS (real-time operating system) refers to an operating system that can accept and process external events or data at a sufficiently fast speed when they are generated, and the processing results can control the production process or respond quickly to the processing system within the specified time, dispatch all available resources to complete real-time tasks, and control all real-time tasks to run in a coordinated manner. The wake-up instruction refers to the process of switching a device or system from a sleep or low-power state to a normal working state. The designated space of the running memory can be a cache space located in the system's random access memory (RAM), which can meet the real-time requirements of image processing.

[0032] As an optional solution for transmitting wake-up commands for microwave radar sensors, targets are detected by emitting microwave signals and receiving the signals reflected back. When the microwave radar detects a target object, such as a person or a vehicle, entering its detection range, a wake-up command is sent to the system and the detection device is woken up.

[0033] As another optional solution for transmitting wake-up commands through sensors, infrared sensors and temperature sensors are used for detection. Heat sources such as humans or animals emit infrared radiation of specific wavelengths, and the PIR sensor can detect these radiations and convert them into electrical signals. At the same time, motion information can also be detected through temperature, so the wake-up command is sent to the system to wake up the detection device.

[0034] As an optional solution for receiving wake-up instructions and running the RTOS system, when the microwave radar detects that a moving target has entered its detection range, the microwave radar sensor sends a wake-up instruction to the data processing end. After receiving the wake-up instruction, the data processing end first starts the first stage of the RTOS system by CPU1 and hands it over to CPU2 for processing. The image signal processor of CPU2 receives and processes the first frame of valid image sent by the camera and saves it to the specified space of the running memory.

[0035] For example, a microwave radar detects a moving target by emitting a microwave signal and receiving the reflected signal. When a target is detected, it sends a wake-up command to the data processing end. After receiving the wake-up command, the data processing end first starts the first-stage RTOS system, initializes the ISP to obtain the first frame of valid images and the first-stage video data, builds a stream buffer, and saves the video data and the first frame of valid images to the specified space of the running memory of the stream buffer.

[0036] Step S200, identifying the image processing data based on an image recognition algorithm, and adjusting the system scheduling of the data processing end according to the image recognition result, wherein the system scheduling includes hibernating the RTOS system or running the Linux system.

[0037] After ISP processes the video data and the first valid frame of the image, CPU1 loads the second-stage RTOS system and hands it over to CPU2 for processing. CPU2 starts RTOS and uses NPU to read the first valid frame of the image from the specified space of the running memory. The first valid frame of the image is used as the input of the image recognition algorithm. The image recognition algorithm is used to extract features of the input image, such as edge detection, texture analysis, shape recognition, etc., to extract key information from the image, and then the extracted features are used for target object recognition, and target objects such as human figures and vehicles in the image are recognized, and finally the recognition results are output. According to the image recognition results, the system decides whether to adjust the system scheduling according to the preset judgment conditions.

[0038] Specifically, if the target identified is a person or a vehicle that requires key monitoring, CPU2 runs the Linux system to encode the real-time video data, and reads the video data collected by the first-stage RTOS in the specified space of the running memory from the specified space of the running memory, and starts running high-power services, which can be opening the network to complex video encoding, storage, application and customized alarm information services; if an irrelevant target or background noise is identified, the system enters a sleep state to reduce the consumption of resources such as power.

[0039] As an optional solution for extracting features from input images using an image recognition algorithm, a deep learning algorithm, which can be a convolutional neural network, is used to extract features from images processed by ISP, learn the key features in the image, and convert them into vector and matrix forms for subsequent user recognition. The extracted features are then post-processed, such as feature selection and feature dimensionality reduction.

[0040] Optionally, a method that does not rely on explicit feature point or region detection is selected, which may be a convolutional neural network, which can relatively more directly perform global and local analysis on the image, automatically learn and extract features.

[0041] As an optional solution for target recognition and Linux system scheduling, the deep learning algorithm running in the NPU recognizes the extracted features and determines whether there are specific targets such as people and vehicles in the image. If the target is recognized, the system scheduling is triggered to run the Linux system to obtain video data in the specified running memory space and collect target video data in real time. If the target is not recognized, the system goes into sleep mode to reduce the consumption of high-power applications.

[0042] Step S300: If the system is scheduled to run the Linux system, the image processing data is encoded to obtain target image data.

[0043] As a case of scheduling, after the second-stage RTOS system recognizes human or vehicle information, it runs the Linux system and opens high-power services such as network, complex video encoding, storage and applications. The Linux system obtains the image saved by the RTOS from the specified space of the running memory, encodes the obtained image, including image compression, frame rate adjustment and other processing, and collects it into video data.

[0044] As an optional solution for encoding images and collecting them into video data, H.264 or H.265 encoding compression can be used; finally, the video data is collected by saving the video file to a specified storage medium, such as a TF card.

[0045] The present embodiment provides a low-power control method for a monitoring device. By receiving a wake-up instruction issued by a sensor control unit, a first-stage RTOS system is first run at the data processing end. After initializing the image signal processor, the first-stage video data is quickly processed to obtain the first frame of valid image, and the first-stage video data is saved to a specified space of the running memory. The first frame of valid image is recognized by an image recognition algorithm, and the system scheduling is intelligently adjusted according to the recognition result, that is, a dormant RTOS system is selected to save power consumption, or a Linux system is run to further process the first-stage video data and collect target video data, thereby reducing the collection of false touch data and improving the validity of the data.

[0046] Based on the first embodiment, the second embodiment of the present application provides a low power consumption control method for a monitoring device, referring to Figure 2 , Figure 2 This is a flow chart of Embodiment 2 of the low power consumption control method for monitoring equipment of the present application, wherein step S200 includes steps S210 to S240: Step S210: determining the configuration of the RTOS system based on functional requirements, and determining the image recognition algorithm based on the configuration.

[0047] Determine the recognition algorithm that the RTOS system needs to configure based on specific functional requirements. First, you need to configure the deep learning framework, as well as the corresponding libraries and dependencies. Use the Convolutional Neural Networks (CNN) structure, add Conv2D, MaxPooling2D, Dropout, and Dense layers, and then compile the model using the classification cross entropy loss function and Adam optimizer. Finally, train the model and use the trained model to predict the test image.

[0048] Step S220: performing image recognition on the image according to the image recognition algorithm to determine the image recognition result.

[0049] It should be noted that, first of all, it is necessary to determine whether the RTOS is awakened by the microwave radar or the 4G remote awakening. If the RTOS is awakened by the microwave radar, after awakening, the NPU built into the RTOS system can run the image recognition algorithm to efficiently process the first frame of valid images received. When the video data is received and preliminarily processed by the ISP after the RTOS system is initialized, the algorithm will immediately perform a detailed recognition analysis on the first frame of valid images, select a suitable feature extraction method according to the requirements of the image recognition task, input the preprocessed first frame of valid images into the image recognition algorithm, extract image features, and determine the required recognition results in the image.

[0050] As an optional solution for extracting human features, the HOG algorithm is used to extract the gradient direction histogram features in the first valid frame of the image. These features can well describe the shape and edge information of the human body. The extracted features are trained using machine learning algorithms such as SVM to obtain a classifier that can distinguish between human and non-human objects. A fixed-size window is slid on the image to be detected, and the HOG features are extracted from each window. The trained classifier is used for judgment. If the classifier believes that a window contains a human body, a detection box is marked at that position.

[0051] Exemplarily, first, the input image is preprocessed to reduce the amount of calculation and improve the accuracy of feature extraction. The preprocessing includes grayscale, denoising, etc., and then the HOG algorithm is used to calculate the gradient direction and size of each pixel in the image, and the histogram of these gradient directions in the local area is statistically analyzed to obtain a feature vector that describes the local shape and edge information of the image. At the same time, a fixed-size window is slid on the image to be detected. This window will traverse each position of the image. After extracting the feature vector, each window is classified using a trained SVM classifier. If the classifier believes that a window contains a human body (that is, the classification result is a positive sample), a detection box is marked at that position, so that the positions of all detected human bodies can be marked on the image.

[0052] As an optional solution for extracting vehicle features, we first need to collect and annotate a large number of vehicle images, perform preprocessing operations such as normalization and scaling on the images, and then design a CNN architecture, including convolutional layers, pooling layers, fully connected layers, etc., and then use the convolutional layers to automatically extract features from the images, map the features to the segmentation map of the image through the pooling layers and fully connected layers, train the network with annotated data, and optimize it using the cross entropy loss function and optimization algorithm. The optimization algorithm can be Adam or SGD, and finally perform threshold processing on the segmentation map output by the network to extract the vehicle contour.

[0053] Exemplarily, 1,000 historical vehicle images and 1,000 historical non-vehicle images are collected. The images are first normalized and scaled to a unified size. Image features are then extracted by designing a convolutional neural network to generate a feature map. The features are mapped to a segmentation map, and the output segmentation map is thresholded to extract vehicle contours. Optionally, when the output probability is greater than 0.5, the pixel is considered to belong to a vehicle, thereby identifying the vehicle.

[0054] Step S230: if the image recognition result does not include a preset target, determining that the system is scheduled to hibernate the RTOS system.

[0055] It should be noted that when no target is detected in the first frame of valid image, the RTOS system is scheduled to enter a sleep state. It should be noted that the preset target can be a human figure or vehicle information. If there is no human activity or vehicle passing by, there is no need to wake up the Linux system and put the RTOS system into sleep mode.

[0056] Furthermore, if no preset target is identified in the image, it is determined that there is no need to start the Linux system, and the video data of the first stage of sensor transmission processed by the RTOS system is removed from the designated space of the running memory of the RTOS system and the Linux system, and the RTOS system is put into sleep mode.

[0057] It should be noted that the specified running memory space means that the RTOS system and the Linux system cannot be run at the same time, but a specified running memory space can be set to achieve data interaction.

[0058] Further, step S230 includes steps S231-S232: Step S231, if the image recognition result does not include the preset target, release the resources occupied by the RTOS system and enter a low power consumption mode, wherein the resources include memory, cache and hardware resources.

[0059] Step S232, setting a wake-up condition for the RTOS system in the low power consumption mode to hibernate the RTOS system.

[0060] When the image recognition result does not contain the preset target, the video data of the first stage of the RTOS processing is released, and then the RTOS system is notified to power off, the motion detection sensor starts working, and the 4G and MCU enter a sleep state.

[0061] Specifically, if the image recognition result does not contain the preset target, the RTOS system in the first stage of release will trigger the power management operation. First, the RTOS system will send an instruction to the power management module, requesting to power off the data processing end. After the power management module receives the instruction from the RTOS system, it will power off the SOC. Then, after the SOC is powered off, the RTOS will enter a sleep state, and at the same time detect the motion sensor and then detect the picture. At the same time, a wake-up condition is set for the detection result to match the wake-up signal without opening the 4G high-power application service.

[0062] Step S240: If the image recognition result includes a preset target, determine that the system is scheduled to run the Linux system.

[0063] The comparison result is analyzed, and if the RTOS system awakened by the sensor detects and recognizes the presence of a preset target in the image, subsequent system operations, such as a human figure or a vehicle, are triggered, thereby determining that the system is scheduled to run the Linux system.

[0064] As a scheduling method, it checks whether the image recognition result meets the preset conditions and whether there is a preset target. If there is a preset target in the recognition result, the Linux system is started, and the RTOS system is stopped when the Linux system is started, thereby running the Linux system.

[0065] Specifically, when the RTOS system awakened by the sensor recognizes the existence of a preset target, it starts the Linux system in turn. During the startup process, the RTOS system continues to collect video data transmitted by the sensor and stores the video data in the specified space of the running memory until the Linux system is loaded. The RTOS system actively transfers resources to the Linux system to run network, complex video encoding, storage and application high-power services.

[0066] This embodiment provides a low power consumption control method for monitoring equipment. When a motion sensor is awakened, an image recognition algorithm is run to analyze the input image to determine whether the image contains a preset target. If the recognition result shows that there are no preset targets in the image, the system will automatically shut down the data processing end to save energy and reduce unnecessary processing until a new motion sensor is awakened or a 4G wake-up signal is detected.

[0067] Based on the first embodiment, the third embodiment of the present application provides a low power consumption control method for a monitoring device, referring to Figure 3 , Figure 3 This is a flow chart of Embodiment 3 of the low power consumption control method for monitoring equipment of the present application, wherein step S300 includes steps S310 to S330: Step S310, initializing the encoder, encoding the image processing data, and determining first image data.

[0068] It should be noted that after the RTOS system initializes the ISP, it captures the image transmitted by the sensor to obtain a stage of video data and the first frame of valid image, and stores the captured image data in a specified running memory space shared between the RTOS system and the Linux system. After recognizing that there is a preset target in the first frame of valid image, the Linux system is started to encode the data captured by the RTOS system. The Linux system first obtains a stage of video data in the specified running memory space, and at the same time the Linux system initializes the encoder, including configuring the encoder parameters such as resolution, frame rate and bit rate; after obtaining the first stage of video data cached by the RTOS system, the image processing data is encoded according to the preset encoding parameters, involving image compression, color space conversion, bit rate control, etc., thereby obtaining the first image data.

[0069] Step S320: Encode the received real-time image data through the Linux system to determine the second image data.

[0070] It should be noted that after the RTOS system recognizes a person or a vehicle, it starts the Linux system to collect the real-time monitoring images. In addition to encoding the video data of the first stage captured by the RTOS system, it is also necessary to receive the real-time video data transmitted by the sensor. Similarly, the received real-time video data is encoded using the same encoding parameters as the first image data, thereby obtaining the second image data.

[0071] Step S330: merge the first image data and the second image data to obtain target image data.

[0072] Based on the first image data and the second image data processed by the same encoder, when the format and frame rate of the first image data and the second image data are the same, the end timestamp of the first image data and the start timestamp of the second image data are obtained and aligned and spliced, and then the encoder is used to re-encode the spliced ​​image data into a video file, and finally the encoded video stream is encapsulated back into the container format to determine the target video data.

[0073] It should be noted that after the Linux system is successfully started and runs stably, the encoder will be initialized and the video data cached by the RTOS system will be efficiently loaded from the specified space of the running memory for encoding processing. At the same time, the network service and background management service will be loaded to ensure that the device can seamlessly access the network environment and establish a stable and efficient communication link with the management background. Among them, the loading of the network service module includes the initialization of the network protocol stack, the activation of the network interface driver, and the configuration of the network management tool. These components work together, follow network protocols such as TCP / IP, and realize data sending and receiving through the network interface, thereby realizing information sharing and communication in the network environment; the loading of the background management service focuses on the monitoring and management of device hardware resources, including the status monitoring of key resources such as CPU, memory and disk, as well as the optimization of task scheduling and the maintenance of log records.

[0074] This embodiment provides a low-power consumption control method for monitoring equipment. The Linux system compresses and encodes the first-stage video data saved by the RTOS system by setting encoding parameters, converts the video data into a format, and obtains first image data. At the same time, the real-time video data transmitted by the sensor is collected, and the real-time video data is processed using the same encoding parameters to obtain second image data. The first image data and the second image data are then spliced ​​and re-encoded according to the timestamp sequence to obtain a complete video sequence. The video data can be collected on the premise of ensuring the existence of a preset target, thereby further ensuring the validity of the collected image data.

[0075] Based on the first embodiment, the fourth embodiment of the present application provides a low power consumption control method for a monitoring device, referring to Figure 4 , Figure 4 This is a flow chart of Embodiment 4 of the low power consumption control method for monitoring equipment of the present application, wherein step S100 includes steps S110 to S120: Step S110, initializing the image signal processor.

[0076] Step S120: Process the image based on the initialized image signal processor to obtain image processing data.

[0077] It should be noted that when the data processing end receives the wake-up command and starts the first-stage RTOS system, the image captured by the camera is transmitted to the first-stage RTOS system for processing. The RTOS system first pre-processes the ISP, and then uses the ISP to perform necessary signal conversion, correction and optimization on the first-stage video data to obtain image processing data.

[0078] Optionally, after using a first-stage RTOS system to process the video data transmitted by the sensor and obtaining the first-stage video data, the first-stage video data needs to be stored. It is feasible to store the first-stage video data in the designated space of the running memory of the RTOS system and the Linux system to facilitate the subsequent Linux system to call the first-stage video data.

[0079] As an optional solution for ISP initialization, first check whether the ISP and its components are powered normally, and configure the internal registers of the ISP according to specific needs, such as exposure time, white balance, gain and other parameters.

[0080] As an optional solution for using ISP to process the received raw image signal, the image is converted into a high-quality, high-visual-experience image through lens correction, noise removal, automatic exposure, etc.

[0081] Exemplarily, the ISP first receives the original image signals from the camera, which may be analog signals including image brightness, color and other information. The ISP then converts the received analog signals into digital signals. The ISP then performs various corrections on the image, such as white balance correction, brightness correction, contrast correction, etc. The ISP then performs optimization operations such as noise reduction and sharpening on the image. After completing this series of steps, the image processed by the ISP will be stored in the specified space of the running memory for use by subsequent image recognition algorithms.

[0082] This embodiment provides a low power consumption control method for monitoring equipment, which uses an initialized image signal processor (ISP) to convert raw image data received from a camera or sensor into a desired target format, stores the converted image data in a predetermined target storage location, and uses an RTOS system to perform low power consumption collection on images.

[0083] Based on the first embodiment, the fifth embodiment of the present application provides a low power consumption control method for a monitoring device, referring to Figure 5 , Figure 5 This is a flowchart of Embodiment 5 of the low power consumption control method for monitoring equipment of the present application, before step S100, including steps S80 to S90: Step S80, receiving a trigger signal generated by a preset sensor, decoding the trigger signal, and determining valid information.

[0084] Before turning on the camera to collect video data, the pre-configured sensor performs a preliminary detection of the monitoring area. If an action that meets the detection requirements is detected, a trigger signal is generated. The system receives the trigger signal and decodes it to determine the valid information.

[0085] As an optional solution for generating a trigger signal, a microwave radar is used as a preset sensor. When the radar initially detects the presence of a moving object, a trigger signal is generated and sent to CPU1. After identifying the trigger signal, CPU1 starts loading a first-stage RTOS system and hands the first-stage RTOS to CPU2 for processing, which completes more accurate image recognition.

[0086] As an optional solution for decoding to determine valid information, a preset decoding algorithm is used to decode the signal to extract valid information. For example, the algorithm identifies the type of moving object corresponding to the trigger signal, which may be a person, vehicle or other animal, and the distance between the object and the radar.

[0087] Step S90, identifying and analyzing the valid information, and sending the wake-up instruction to the RTOS system to run the RTOS system at the data processing end.

[0088] After decoding the trigger signal, it is necessary to confirm the valid information in the trigger signal. If the valid information is irrelevant information, for example, the valid information in the trigger signal is pre-identified human information, vehicle information or other animal activity information, then the valid information needs to be further identified and analyzed. Only when it is determined that relevant information appears will a wake-up command be sent to the real-time operating system to perform the next identification operation.

[0089] This embodiment provides a low-power control method for a monitoring device, which monitors and receives trigger signals sent by preset sensors, decodes these signals to extract valid information, identifies and analyzes the information, and then sends a wake-up instruction to the RTOS system based on the analysis results, so that the system can respond to external events or changes in conditions in a timely manner.

[0090] Based on the first embodiment, the sixth embodiment of the present application provides a low power consumption control method for a monitoring device, referring to Figure 6 , Figure 6 This is a flow chart of Embodiment 6 of the low power consumption control method for monitoring equipment of the present application. After step S400, steps S500 to S700 are also included: Step S500: Based on the collected target image data, the image recognition algorithm is used to identify key information in the image.

[0091] The key information is determined by the detection requirements, and the target video data is identified through the image recognition algorithm to detect the key information of the relevant detection requirements, which may be human information and vehicle information, etc. The corresponding image recognition algorithm may be a human recognition algorithm or a vehicle recognition algorithm, etc.

[0092] Optionally, after step S500, steps S510 to S520 are included: Step S510: Detect the key information to determine abnormal conditions in the target video data.

[0093] Step S520: Based on the abnormal situation, abnormal information is sent to the detection personnel.

[0094] As an optional solution to determine abnormal situations and notify detection personnel, key features such as object motion patterns, speed changes, etc. are extracted from video data, and then machine learning algorithms such as support vector machines or Bayesian networks are used to train anomaly detection models, so that abnormal behaviors that are significantly different from normal behavior patterns can be identified. When the model identifies an abnormal situation, the system will generate an abnormal report and notify the detection personnel through appropriate channels.

[0095] For example, in the field of intelligent transportation, applications can use algorithms to conduct real-time analysis of data such as traffic flow, vehicle speed, and pedestrian behavior, and customize warning information services based on the analysis results. For example, when a traffic jam or traffic accident occurs on a certain road section, the system can immediately send warning information to the relevant vehicles to remind them to choose other routes to detour, which helps to ease traffic pressure and improve road traffic efficiency.

[0096] Step S600: Detecting the moving path of the monitored target based on the key information.

[0097] Through the detected key information, the monitoring target in the target video data is determined, and the moving path of the monitoring target is recorded in real time.

[0098] As an alternative to recording the movement path of the surveillance target, the OpenCV library for machine learning is used to read the video and let the user select the initial position of the surveillance target and define the region of interest. The color histogram of the region of interest is calculated for target matching in subsequent frames. The CamShift algorithm is used to track the target in the video frame.

[0099] Step S700, uploading the moving path to the host computer through 4G cloud technology, so that the host computer wakes up the Linux systems of other monitoring devices on the moving path to achieve joint control and joint defense of at least two target monitoring devices.

[0100] It should be noted that when awakened by the 4G cloud, the data processing end will not be powered off and will not respond to power off. The system does not run the first-stage RTOS system, but directly runs the second-stage Linux. When the monitoring device receives the 4G wake-up command, there is no need for the RTOS system to recognize people or vehicles in the image. The Linux system is directly started to collect video data, and high-power services such as network and background management services are opened at the same time.

[0101] On the cloud monitoring platform, the monitoring devices and their moving paths that need to be jointly controlled and jointly defended are pre-set. When a device detects an abnormality (such as a human figure, a vehicle, etc.), the linkage mechanism will be triggered. When the linkage mechanism is triggered, the cloud monitoring platform sends a wake-up command to the next monitoring device on the moving path through the 4G network. The receiving end monitoring device receives the wake-up command through the 4G module and immediately executes the wake-up operation of the Linux system. There is no need to start the RTOS system to perform human or vehicle recognition on the image, and it directly enters the working state. The awakened device continues to monitor the moving path and forms a linkage with the previous device to jointly monitor the moving target.

[0102] For example, there are two monitoring devices A and B on the same street, installed at both ends of the street. When device A detects a humanoid target entering from one end of the street, the cloud monitoring platform receives the alarm information from device A and immediately sends a wake-up command to device B through the 4G network. After receiving the command, device B quickly wakes up the Linux system and enters the working state, jointly monitoring the movement trajectory of the humanoid target with device A.

[0103] Optionally, the device receives a wake-up command and detection target information sent from the cloud, and wakes up the Linux system to detect the detection target.

[0104] As an optional implementation, the built-in 4G module of the device maintains a communication connection with the cloud, waiting to receive a wake-up command. After receiving the wake-up command from the cloud, the MCU will be activated first, and then the power management unit will be controlled to ensure that the device wakes up quickly from a low-power sleep state and provide the necessary power for the system. At the same time, the device starts to execute the initialization process of the Linux system, including hardware device detection and system configuration loading, and then directly starts the Linux system, takes over the intelligent detection of the detection target, outputs the video stream, realizes linkage with the original monitoring device, and takes over the monitoring task.

[0105] Exemplarily, monitoring device D receives the wake-up command and detection target information transmitted from the cloud, directly runs the Linux system, and activates background management functions and AI functions, such as human recognition and vehicle recognition, performs intelligent analysis and processing on image data, identifies the detection targets, and outputs video streams. Furthermore, it forms a monitoring network with the original monitoring device C to achieve joint control and defense of the detection targets in the detection area.

[0106] This embodiment provides a low-power control method for monitoring equipment, which analyzes the collected target video data by applying an image recognition algorithm, identifies key information therefrom, and detects the moving path of the monitoring target based on this information. Then, the Linux system of the relevant monitoring equipment on the moving path is remotely and directly awakened by using 4G cloud technology. After the relevant equipment is awakened, the Linux system is directly run, the 4G network and other high-power consumption services and applications are turned on, and the linkage prevention and control between at least two monitoring devices is realized to enhance the synergy and response capability of the monitoring system.

[0107] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the low power consumption control method of the monitoring device of the present application. More simple transformations based on this technical concept are all within the protection scope of the present application.

[0108] It should be noted that Figure 7The resource scheduling diagram can show the resource scheduling of the present application. First, the system is in a low-power standby state, waiting for microwave radar wake-up or 4G network remote wake-up. If it is a passive wake-up by the human body radar or PIR device end, the RTOS system starts quickly at this time, and then stores it in the specified space of the running memory, and immediately starts image preprocessing, including initializing the image signal processor after the fill light control, and then building a stream buffer to temporarily store the video stream collected by the RTOS, and save the specified running memory space, and quickly load the RTOS image recognition algorithm at the same time, identify the image processing data based on the image recognition algorithm, and adjust the system scheduling of the data processing end according to the image recognition result, that is, Determine whether there is a human figure or a vehicle in the image. The system scheduling includes hibernating the RTOS system or further running the Linux system. If the system scheduling is to run the Linux system, the 4G network is turned on, the image processing data is video-encoded, the target image data is obtained, and complex high-energy consumption applications are run. After the task is completed, the device returns to a dormant low-power state. If the 4G network is actively awakened remotely, after the 4G remote power-on is normal, the RTOS system immediately turns on the recording function, loads and runs the Linux system, and after the Linux system works, it takes over the recording function of the RTOS system and continues to run high-energy consumption applications. After the task is completed, it goes back to sleep.

[0109] The present application provides a low power consumption control device for a monitoring device, the device comprising: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the low power consumption control method for the monitoring device in the above-mentioned embodiment 1.

[0110] Reference below Figure 8 , which shows a schematic diagram of the structure of a low-power control device for a monitoring device suitable for implementing an embodiment of the present application. The low-power control device for the monitoring device in the embodiment of the present application may include, but is not limited to, mobile terminals such as laptop computers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 8 The low power consumption control device of the monitoring device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0111] like Figure 8As shown, the low power consumption control device of the monitoring device may include a processing device 1001 (e.g., a core processor, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the low power consumption control device of the monitoring device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, and a microphone; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display); a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the low-power control device of the monitoring device to communicate wirelessly or wired with other devices to exchange data. Although the figure shows a low-power control device of the monitoring device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or provided alternatively.

[0112] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0113] The low-power control device of the monitoring device provided by the present application adopts the low-power control method of the monitoring device in the above embodiment, which can solve the technical problem of low-power control of the monitoring device. Compared with the prior art, the beneficial effects of the low-power control device of the monitoring device provided by the present application are the same as the beneficial effects of the low-power control method of the monitoring device provided by the above embodiment, and other technical features in the low-power control device of the monitoring device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0114] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0115] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0116] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the low power consumption control method of the monitoring device in the above-mentioned embodiment.

[0117] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0118] The computer-readable storage medium may be included in the low-power consumption control device of the monitoring device; or may exist independently without being assembled into the low-power consumption control device of the monitoring device.

[0119] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the low-power control device of the monitoring device, the low-power control device of the monitoring device: processes the received image based on the RTOS system to obtain image processing data; Identify the image processing data based on an image recognition algorithm, and adjust the system scheduling of the data processing end according to the image recognition result, wherein the system scheduling includes hibernating the RTOS system or running the Linux system; If the system is scheduled to run the Linux system, the image processing data is encoded to obtain target image data.

[0120] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0121] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0122] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0123] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the low-power control method of the above-mentioned monitoring device, and can solve the technical problem that after detecting the motion information of the object, the cameras are all awakened, and the data collected after awakening cannot guarantee whether there is personnel information, and the validity of the data cannot be guaranteed, resulting in low validity of the acquired data. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the low-power control method of the monitoring device provided by the above-mentioned embodiment, and will not be repeated here.

[0124] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A low power consumption control method for a monitoring device, characterized in that: The low power consumption control method of the monitoring device comprises the following steps: Process the received image based on the RTOS system to obtain image processing data; Identify the image processing data based on an image recognition algorithm, and adjust the system scheduling of the data processing end according to the image recognition result, wherein the system scheduling includes hibernating the RTOS system or running the Linux system; If the system is scheduled to run the Linux system, the image processing data is encoded to obtain target image data.

2. The low power consumption control method of the monitoring device according to claim 1, characterized in that: The image processing data is identified based on an image recognition algorithm, and the system scheduling of the data processing end is adjusted according to the image recognition result, wherein the system scheduling includes the steps of hibernating the RTOS system or running the Linux system, including: Determine the configuration of the RTOS system based on functional requirements, and determine the image recognition algorithm based on the configuration; Performing image recognition on the image processing data according to the image recognition algorithm to determine the image recognition result; If the image recognition result does not include a preset target, determining that the system is scheduled to hibernate the RTOS system; If the image recognition result includes a preset target, the system is determined to be scheduled to run a Linux system.

3. The low power consumption control method of the monitoring device according to claim 2, characterized in that: If the image recognition result does not include a preset target, the step of determining that the system is scheduled to hibernate the RTOS system comprises: If the image recognition result does not include the preset target, release the resources occupied by the RTOS system and enter a low power consumption mode, wherein the resources include memory, cache and hardware resources; A wake-up condition is set for the RTOS system in the low power consumption mode to put the RTOS system into hibernation.

4. The low power consumption control method of the monitoring device according to claim 1, characterized in that: If the system is scheduled to run the Linux system, the step of encoding the image processing data to obtain target image data includes: Initializing an encoder to encode the image processing data and determine first image data; Encoding the received real-time image data through the Linux system to determine the second image data; The first image data and the second image data are combined to obtain target image data.

5. The low power consumption control method of the monitoring device according to claim 1, characterized in that: The step of processing the received image based on the RTOS system to obtain image processing data includes: Initialize the image signal processor; The image is processed based on the initialized image signal processor to obtain image processing data.

6. The low power consumption control method of the monitoring device according to claim 1, characterized in that: Before the step of processing the received image based on the RTOS system to obtain image processing data, the method further includes: Receive a trigger signal generated by a preset sensor, decode the trigger signal, and determine valid information; The valid information is identified and analyzed, and a wake-up instruction is sent to the RTOS system to run the RTOS system at the data processing end.

7. The low power consumption control method of the monitoring device according to claim 1, characterized in that: If the system is scheduled to run the Linux system, after the step of encoding the image processing data to obtain the target image data, the method further includes: Based on the collected target image data, using the image recognition algorithm to identify key information in the image; Detecting a moving path of a monitored target based on the key information; The moving path is uploaded to the host computer through 4G cloud technology, so that the host computer can wake up the Linux systems of other monitoring devices on the moving path to achieve joint control and joint defense of at least two target monitoring devices.

8. The low power consumption control method of the monitoring device according to claim 7, characterized in that: After the step of using the image recognition algorithm to identify key information in the image based on the collected target image data, the method further includes: Detecting the key information to determine abnormal conditions in the target image data; Based on the abnormal situation, the abnormal information is sent to the detection personnel.

9. A low power consumption control device for a monitoring device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the low power consumption control method for the monitoring device according to any one of claims 1 to 8.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the low power consumption control method of the monitoring device according to any one of claims 1 to 8 are implemented.

Images