Control method and device of security and protection equipment, security and protection equipment and medium

Abstract

The invention relates to a control method and device of security and protection equipment, the security and protection equipment and a medium. The security equipment comprises a first processor, a second processor and an ISP, the power consumption of the second processor is smaller than that of the first processor, and the method comprises the steps that the second processor obtains first image data of a target area, and the first image data is original image data, collected by the ISP, of the target area; the second processor performs target detection on the first image data to obtain a first detection result; the second processor instructs the first processor to enter a dormant state or instructs the first processor to perform image processing on second image data output by the image signal processor according to the first detection result, the second image data is obtained by performing image quality processing on the first image data by the image signal processor. By adopting the scheme, the power consumption of the security equipment can be reduced.

Description

Technical Field

[0001] This application relates to the field of security technology, and in particular to a control method, device, security equipment, and medium for security equipment. Background Technology

[0002] With the rapid development of image technology, it has been widely used in fields such as security monitoring, video conferencing, and smart homes.

[0003] In the image / video acquisition phase, taking a camera as an example, PIR (Passive Infrared Radiation) or radar is generally used as a trigger source to directly wake up the camera. However, this control method suffers from the problem of high power consumption of the camera. Summary of the Invention

[0004] Therefore, it is necessary to provide a control method, device, security equipment, and medium for security equipment that can reduce the power consumption of security equipment, in response to the above-mentioned technical problems.

[0005] In a first aspect, this application provides a control method for a security device. The method is used in a security device, which includes a first processor, a second processor, and an image signal processor. The power consumption of the second processor is less than that of the first processor. The method includes:

[0006] The second processor acquires first image data of the target area, wherein the first image data is the original image data of the target area acquired by the image signal processor;

[0007] The second processor performs target detection on the first image data to obtain a first detection result;

[0008] Based on the first detection result, the second processor instructs the first processor to enter a sleep state or instructs the first processor to perform image processing on the second image data output by the image signal processor, wherein the second image data is obtained by the image signal processor through image quality processing of the first image data.

[0009] In one embodiment, the second processor, based on the first detection result, instructs the first processor to enter a sleep state or instructs the first processor to perform image processing on the second image data output by the image signal processor, including:

[0010] If the first detection result indicates that there is no target in the target area, the second processor instructs the first processor to enter a sleep state;

[0011] If the first detection result indicates that a target exists in the target area, the second processor instructs the first processor to perform image processing on the second image data.

[0012] In one embodiment, the method further includes:

[0013] The first processor enters a sleep state or performs image processing on the second image data according to the instructions of the second processor.

[0014] In one embodiment, after the second processor instructs the first processor to enter a sleep state or instructs the first processor to perform image processing on the second image data output by the image signal processor based on the first detection result, the method further includes:

[0015] The second processor enters a sleep state.

[0016] In one embodiment, the method further includes:

[0017] The second processor determines whether the first detection result is reliable;

[0018] Based on the first detection result, the second processor instructs the first processor to enter a sleep state or instructs the first processor to perform image processing on the second image data output by the image signal processor, including:

[0019] If the first detection result is reliable, the second processor will instruct the first processor to enter a sleep state or instruct the first processor to perform image processing on the second image data based on the first detection result.

[0020] In one embodiment, the method further includes:

[0021] If the first detection result is unreliable, the second processor enters a sleep state, and the first processor acquires the second image data of the target area. The first processor performs target detection on the second image data to obtain the second detection result.

[0022] If the second detection result indicates that there is no target in the target area, the first processor enters a sleep state.

[0023] If the second detection result indicates that a target exists in the target area, the first processor performs image processing on the second image data.

[0024] In one embodiment, the second processor determines whether the first detection result is reliable, including:

[0025] The second processor acquires a visual capability model, which is used to characterize the visual capability of the second processor;

[0026] The second processor determines whether the first detection result is reliable based on the visual ability model.

[0027] In one embodiment, the second processor determines whether the first detection result is reliable based on the visual ability model, including:

[0028] If the capability field indicated by the visual capability model is a first value, the second processor determines that the first detection result is reliable;

[0029] If the capability field indicated by the visual capability model is the second value, the second processor determines that the first detection result is unreliable, and that the first value and the second value are different.

[0030] In one embodiment, the second processor acquires the visual capability model, including:

[0031] The second processor reads the visual capability model from a preset storage location. The visual capability model is established and stored in the preset storage location by the first processor after verifying the visual capability of the second processor.

[0032] In one embodiment, the method further includes:

[0033] The second processor performs target detection on the first sample image data of the sample region, obtains a third detection result, and sends the first sample image data and the third detection result to the first processor. The first sample image data is the original image data of the sample region acquired by the image signal processor.

[0034] The first processor obtains second sample image data based on the first sample image data, and performs target detection on the second sample image data to obtain a fourth detection result. The second sample image data is obtained by performing image quality processing on the first sample image data.

[0035] The first processor uses the fourth detection result to verify the third detection result, and establishes the visual ability model based on the verification result.

[0036] In one embodiment, the security device further includes a neural network processor, wherein the second processor performs target detection on the first image data to obtain a first detection result, including:

[0037] The second processor sends the first image data to the neural network processor;

[0038] The neural network processor performs target detection on the first image data, obtains the first detection result, and returns the first detection result to the second processor.

[0039] In one embodiment, the security device further includes a low-power communication module and a sensor, and before the second processor acquires the first image data of the target area, the method further includes:

[0040] The first processor receives a first wake-up signal sent by the low-power communication module and enters the wake-up state according to the indication of the first wake-up signal;

[0041] The second processor receives the second wake-up signal sent by the low-power communication module and enters the wake-up state according to the instruction of the second wake-up signal;

[0042] The first wake-up signal and the second wake-up signal are generated by the low-power communication module based on the sensor's perception results of the target area.

[0043] Secondly, this application also provides a control device for a security device. The device is used in a security device, which includes a first processor, a second processor, and an image signal processor. The power consumption of the second processor is less than that of the first processor. The device includes:

[0044] The acquisition module is used to acquire first image data of the target area through the second processor, wherein the first image data is the original image data of the target area acquired by the image signal processor;

[0045] The detection module is used to perform target detection on the first image data through the second processor to obtain a first detection result;

[0046] The control module is configured to, based on the first detection result, instruct the first processor to enter a sleep state or instruct the first processor to perform image processing on the second image data output by the image signal processor, wherein the second image data is obtained by the image signal processor performing image quality processing on the first image data.

[0047] Thirdly, this application also provides a security device. The security device includes a memory, a first processor, and a second processor, wherein the power consumption of the second processor is less than that of the first processor, and the memory stores a computer program; wherein,

[0048] When the first processor executes the computer program, it implements the steps performed by the first processor in the method described in the first aspect.

[0049] When the second processor executes the computer program, it implements the steps performed by the second processor in the method described in the first aspect.

[0050] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the method described in the first aspect.

[0051] The aforementioned security equipment control method, device, security equipment, and medium include a first processor, a second processor, and an image signal processor. The power consumption of the second processor is lower than that of the first processor. The second processor acquires first image data of the target area, which is the original image data of the target area collected by the image signal processor. Then, the second processor performs target detection on the first image data to obtain a first detection result. Based on the first detection result, the second processor instructs the first processor to enter a sleep state or instructs the first processor to perform image processing on the second image data output by the image signal processor. The second image data is obtained by the image signal processor through image quality processing of the first image data. In this way, the second processor, with its lower power consumption, performs false trigger filtering. The second processor, based on the target detection... The first detection result determines whether the first processor needs to be triggered. If the first processor does not need to be triggered, it can directly enter a sleep state, effectively avoiding the situation where the first processor is falsely triggered due to poor perception accuracy of PIR or radar, thus saving the overall power consumption of the security equipment. In addition, the object of the target detection by the second processor (i.e., the first image data) is the raw image data collected by the image signal processor. This raw image data refers to image data without image quality processing. Compared with the method of target detection by the main control CPU on YUV domain image data (i.e., image data after image quality processing) in related technologies, this application does not need to perform complicated image quality processing on the first image data, saving the power consumption of the image quality processing process, thereby further saving the overall power consumption of the security equipment. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 This is a schematic diagram of the ISP Pipeline processing procedure;

[0054] Figure 2 A flowchart illustrating how the main control CPU in a camera uses visual algorithms to detect targets.

[0055] Figure 3 This is a flowchart illustrating a control method for a security device in one embodiment;

[0056] Figure 4 This is a flowchart illustrating the control method for a security device in another embodiment;

[0057] Figure 5 This is a flowchart illustrating the control method for a security device in another embodiment;

[0058] Figure 6 This is a flowchart illustrating step 501 in another embodiment;

[0059] Figure 7 A flowchart illustrating the process of establishing a visual capability model for the first processor in another embodiment;

[0060] Figure 8 This is a flowchart illustrating the control method for a security device in another embodiment;

[0061] Figure 9 This is a schematic diagram comparing the duration of determining a false trigger in this embodiment with the duration of determining a false trigger in the conventional technology;

[0062] Figure 10 This is a structural block diagram of the control device of a security equipment in one embodiment;

[0063] Figure 11 This is an internal structural diagram of a security device in one embodiment. Detailed Implementation

[0064] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0065] With the rapid development of image technology, it has been widely used in fields such as security monitoring, video conferencing, and smart homes.

[0066] In the image / video acquisition stage, taking battery-powered smart camera products as an example, PIR (Passive Infrared Radiation) or radar is generally used as a trigger source to wake up the camera, allowing it to acquire video data and then save the recording.

[0067] During the image acquisition process of the camera, the entire ISP (Image Signal Processing) pipeline, such as... Figure 1 As shown, the ISP Pipeline is responsible for a series of image processing steps. Light passes through the lens and is projected onto the sensor, where it is converted from photoelectric to analog electrical signals. These signals are then converted into digital signals by an A / D (Analog to Digital Converter) and sent to the ISP for further processing.

[0068] The ISP Pipeline mainly includes the processing procedures shown in Table 1:

[0069] Table 1

[0070]

[0071] However, both PIR and radar have shortcomings in target detection. For example, PIR has weak target detection performance in the longitudinal direction, radar is more sensitive to target speed, and radar has shortcomings in analyzing large background noise. As a result, the perception accuracy of PIR or radar is poor. Therefore, there may be a situation where the sensor (PIR or radar) wakes up the camera after sensing the target, but the camera does not find the target when using visual algorithms to detect the target. This will cause the camera to be falsely triggered.

[0072] Please combine Figure 2 , Figure 2 A flowchart illustrating the process of waking up the main control CPU (Central Processing Unit) in the camera and using visual algorithms to detect targets.

[0073] Figure 2 In the process, after the sensor detects the target, it triggers the low-power Wi-Fi (Wireless Fidelity) module to wake up the main control chip. After the main control CPU system initializes, it executes... Figure 1 Each processing step in the process yields YUV (Y represents luminance, U and V represent chrominance) domain data, which is image data after image quality processing. The main control CPU sends the YUV domain data to the main control NPU (Neural Processing Unit) for target detection. The main control NPU returns the detection result. If the detection result is that no target such as a person, vehicle or pet is found, the main control CPU will be triggered erroneously.

[0074] As can be seen, the camera's visual algorithm is based on the YUV domain data output by the camera module. As shown in Table 1, the camera module needs the main control CPU to go through a considerable number of pipelines to obtain the YUV domain data. As a high-performance CPU, the main control CPU consumes a lot of power, resulting in high power consumption of the camera.

[0075] In view of this, this application provides a control method, device, security equipment, and medium for security devices, which can reduce the power consumption of security devices.

[0076] It should be noted that the beneficial effects or technical problems solved by the embodiments of this application are not limited to this one, but may also be other implicit or related problems. For details, please refer to the description of the embodiments below.

[0077] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0078] In one exemplary embodiment, such as Figure 3 As shown, a control method for security equipment is provided. The method is illustrated by taking the application of this method to security equipment as an example. Security equipment can be cameras, video doorbells, etc.

[0079] In this embodiment, the security device includes a first processor, a second processor, and an image signal processor. The power consumption of the second processor is less than that of the first processor; that is, for the same running time, the power consumption of the first processor is greater than that of the second processor. The first processor can be a high-performance processor, and the second processor can be a low-power processor. That is, the processing power of the first processor is higher than that of the second processor (processing power such as calculation precision, calculation speed, etc.). For example, the first processor can be a CPU, and the second processor can be an MCU (Microcontroller Unit).

[0080] Please see Figure 3 The method includes the following steps 301 to 303:

[0081] Step 301: The second processor acquires the first image data of the target area.

[0082] The target area can be the coverage area of ​​security equipment. Sensors can be installed in the security equipment. The sensors can be PIR and / or radar as mentioned above. The sensors perceive the target area. After the sensors perceive the target, they wake up the first processor and the second processor.

[0083] Optionally, the security device may also include a low-power communication module (e.g., a low-power WIFI module). The sensor detects the target area, and after the sensor detects the target, it triggers the low-power communication module. The low-power communication module then sends a wake-up signal to the first processor and the second processor, respectively, to wake up the first processor and the second processor.

[0084] For example, the low-power communication module sends a first wake-up signal to the first processor and a second wake-up signal to the second processor. It can be understood that the first wake-up signal and the second wake-up signal are generated by the low-power communication module based on the sensor's perception result of the target area (if the perception result is that the target is perceived, the low-power communication module generates the first wake-up signal and the second wake-up signal).

[0085] After receiving the first wake-up signal sent by the low-power communication module, the first processor enters the wake-up state according to the instruction of the first wake-up signal. After receiving the second wake-up signal sent by the low-power communication module, the second processor enters the wake-up state according to the instruction of the second wake-up signal.

[0086] In this way, the first and second processors enter the wake-up state. After the first and second processors are awakened, the security device acquires images of the target area through the image signal processor, and the second processor can obtain the first image data of the target area.

[0087] In this embodiment, the first image data is the raw image data of the target area acquired by the image signal processor. Raw image data refers to image data acquired by the image signal processor before image quality processing, i.e., image data before image quality processing, also known as RAW domain image data. During the acquisition of the first image data, light is projected onto the sensor through the lens, converted into an analog electrical signal by photoelectric conversion, and then converted into a digital signal by an A / D converter. The image signal processor processes this digital signal in the RAW domain to obtain the first image data.

[0088] Step 302: The second processor performs target detection on the first image data to obtain the first detection result.

[0089] The second processor acquires the first image data of the target area, and uses a visual algorithm to perform target detection on the first image data to obtain the first detection result.

[0090] In one possible implementation of step 302, the second processor can input the first image data into a pre-trained target detection model to obtain the first detection result output by the target detection model.

[0091] In another possible implementation of step 302, the security device may further include a neural network processor. When the first processor and the second processor are awakened, the neural network processor is also awakened. In this way, after the second processor obtains the first image data of the target area, the second processor sends the first image data to the neural network processor. The neural network processor performs target detection on the first image data. For example, the neural network processor can input the first image data into a pre-trained target detection model to obtain the first detection result output by the target detection model. Then, the neural network processor returns the first detection result to the second processor.

[0092] Step 303: Based on the first detection result, the second processor instructs the first processor to enter a sleep state or instructs the first processor to perform image processing on the second image data output by the image signal processor.

[0093] The second image data is obtained by the image signal processor through image quality processing of the first image data.

[0094] Taking the second processor performing target detection on the first image data via a neural network processor as an example, if the second processor can successfully receive the first detection result returned by the neural network processor, optionally, the second processor can directly determine the subsequent operation of the first processor (entering sleep mode or performing image processing on the second image data output by the image signal processor) based on whether a target exists in the target area indicated by the first detection result. Optionally, after obtaining the first image data, the second processor can also perform target detection on the first image data itself to obtain a target detection result, and use the target detection result to verify the first detection result. If the target detection result is the same as the first detection result (i.e., verification passes), the second processor then determines the subsequent operation of the first processor based on the first detection result. If the verification fails, the second processor can instruct the neural network processor to re-perform target detection on the first image data to obtain an accurate first detection result, which can improve the control accuracy of the control method in this embodiment.

[0095] Optionally, if the second processor fails to receive the first detection result returned by the neural network processor (for example, after the second processor sends the first image data to the neural network processor, the second processor does not receive the first detection result returned by the neural network processor for a preset time), the second processor can perform target detection on the first image data to obtain the target detection result, and determine the subsequent operation of the first processor based on whether there is a target in the target area indicated by the target detection result.

[0096] If the second processor instructs the first processor to enter a sleep state based on the first detection result, the first processor will enter a sleep state according to the instruction of the second processor, effectively avoiding the situation where the first processor is falsely triggered due to poor perception accuracy of PIR or radar.

[0097] If the second processor instructs the first processor to perform image processing on the second image data output by the image signal processor based on the first detection result, the image signal processor will first continue to perform image quality processing on the first image data, that is, according to... Figure 1 As shown in Table 1, the first image data is sequentially processed in the RGB (Red Green Blue) domain and the YUV domain to obtain the second image data, which is image data in the YUV domain. Then, the first processor performs further image processing on the second image data output by the image signal processor. For example, the first processor performs target detection on the second image data, saves the target detection results, saves the second image data to obtain a video stream, and so on.

[0098] Overall, the security device described in the above embodiment includes a first processor, a second processor, and an image signal processor. The power consumption of the second processor is lower than that of the first processor. The second processor acquires first image data of the target area, which is the original image data of the target area collected by the image signal processor. Then, the second processor performs target detection on the first image data to obtain a first detection result. Based on the first detection result, the second processor instructs the first processor to enter a sleep state or instructs the first processor to perform image processing on the second image data output by the image signal processor. The second image data is obtained by the image signal processor through image quality processing of the first image data. In this way, the second processor, with its lower power consumption, performs false trigger filtering. The second processor determines whether the first processor needs to be triggered based on the first detection result obtained from the target detection. If the first processor does not need to be triggered, it can directly enter a sleep state, effectively avoiding the situation where the first processor is falsely triggered due to poor perception accuracy of PIR or radar. Especially in scenarios with many false triggers (such as when the sensor quality is poor or the environment is complex), the second processor effectively uses visual detection to judge the first image data, effectively filtering false triggers against the first processor, and greatly saving the overall power consumption of the security device.

[0099] Furthermore, it is understood that the power consumption of the false triggering process is directly proportional to the working duration of the false triggering; that is, the longer the working duration of the false triggering, the greater the power consumption of the false triggering process. In this embodiment, the object of target detection by the second processor (i.e., the first image data) is the raw image data collected by the image signal processor. This raw image data refers to image data that has not undergone image quality processing. In contrast, in related technologies, the main control CPU performs target detection on image data in the YUV domain (i.e., image data that has undergone image quality processing). Figure 1 It is known that the time required for the second processor to acquire the raw image data collected by the image signal processor is less than the time required for the second processor to acquire the YUV domain image data output by the image signal processor. Therefore, the embodiments of this application reduce the working time of a false trigger, thereby saving the power consumption of the security equipment in case of false trigger.

[0100] During the research and development process, the inventors of this application discovered through extensive experimental verification that the second processor can call the neural network processor to detect and recognize image data before image quality algorithm processing within the startup time of the first processor (assuming 500ms) to filter false triggers. The second processor decides whether to put the entire main control into sleep mode based on the first detection result, thereby reducing unnecessary overhead caused by false triggers and further reducing the overall power consumption of the security equipment.

[0101] If the visual capabilities of the first processor do not meet the requirements (or are not up to standard), the entire system startup process will not be interrupted. That is, after the first processor starts up, it will use the image data in the YUV domain to re-detect targets such as people, vehicles or pets to ensure the normal and reliable operation of the security equipment.

[0102] In one embodiment, based on Figure 3 The illustrated embodiment can be found in [reference]. Figure 4 This embodiment provides an exemplary description of step 303 and possible implementations thereafter. For example... Figure 4 As shown, step 303 may include Figure 4 Steps 3031 and 3032 are shown below:

[0103] Step 3031: If the first detection result indicates that there is no target in the target area, the second processor instructs the first processor to enter a sleep state.

[0104] After the second processor obtains the first detection result for the target area, if it determines that there is no target in the target area, the second processor instructs the first processor to enter a sleep state, and the first processor enters a sleep state according to the instruction of the second processor.

[0105] For example, if the second processor determines that there is no target in the target area, the second processor can send a sleep command to the first processor. After receiving the sleep command sent by the second processor, the first processor enters a sleep state. This can avoid the problem of high power consumption caused by the first processor mistakenly triggering image processing of the second image data output by the image signal processor when there is no target in the target area.

[0106] Step 3032: If the first detection result indicates that a target exists in the target area, the second processor instructs the first processor to perform image processing on the second image data output by the image signal processor.

[0107] If the second processor determines that a target exists in the target area, it instructs the first processor to perform image processing on the second image data output by the image signal processor. The first processor then performs image processing on the second image data according to the instructions of the second processor. For example, the first processor performs target detection on the second image data, saves the target detection result, saves the second image data to obtain a video stream, and so on.

[0108] For example, if the second processor determines that a target exists in the target area, the second processor can send an image processing instruction to the first processor. After receiving the image processing instruction sent by the second processor, the first processor will start to process the second image data output by the image signal processor, since the second processor has already confirmed that a target exists in the target area. This can avoid the problem of high power consumption caused by the first processor mistakenly triggering image processing of the second image data when there is no target in the target area.

[0109] Furthermore, in this embodiment of the application, steps 3031 and 3032 are followed by... Figure 4 Step 401 shown:

[0110] Step 401: The second processor enters sleep mode.

[0111] In this way, after the second processor instructs the first processor to enter a sleep state based on the first detection result or instructs the first processor to perform image processing on the second image data output by the image signal processor, the second processor enters a sleep state and waits for the next wake-up, thereby further reducing the power consumption of the second processor and reducing the overall power consumption.

[0112] The above embodiments describe the process by which the second processor performs target detection on the first image data of the target area, and, based on the obtained first detection result, instructs the first processor to enter a sleep state or instructs the first processor to perform image processing on the second image data output by the image signal processor.

[0113] To further ensure the accuracy of the aforementioned control processes of security equipment, based on Figure 3 The illustrated embodiment can be found in [reference]. Figure 5 After step 302, this embodiment also includes Figure 5 Step 501 shown:

[0114] Step 501: The second processor determines whether the first detection result is reliable.

[0115] In one possible implementation, as described above, the second processor can perform target detection on the first image data through the neural network processor, and the second processor can obtain a target detection result by performing target detection on the first image data on its own. After receiving the first detection result returned by the neural network processor, the second processor uses the target detection result obtained by its own target detection to verify the first detection result. If the target detection result is the same as the first detection result, it indicates that the first detection result is credible; otherwise, it indicates that the first detection result is unreliable.

[0116] Other possible implementations for the second processor to determine whether the first detection result is credible will be described in the following embodiments.

[0117] The following sections will discuss the cases where the first test result is reliable and the cases where the first test result is unreliable.

[0118] Please continue reading Figure 5 Step 303 includes Figure 5 Step 3033 shown:

[0119] Step 3033: If the first detection result is reliable, the second processor instructs the first processor to enter a sleep state or instructs the first processor to perform image processing on the second image data based on the first detection result.

[0120] If the second processor determines that the first detection result is reliable, the second processor then instructs the first processor to enter a sleep state or instructs the first processor to perform image processing on the second image data based on the first detection result. The process of the second processor instructing the first processor to enter a sleep state or instructing the first processor to perform image processing on the second image data based on the first detection result can be referred to the relevant description in the above embodiments, and will not be repeated here.

[0121] In this way, the problem of poor control accuracy of security equipment due to the unreliability of the first detection result can be avoided. If the first detection result is reliable, the second processor can then instruct the first processor to perform corresponding operations based on the first detection result, thereby improving the control accuracy of the security equipment.

[0122] Please continue reading Figure 5The embodiments of this application also include Figure 5 Steps 502-504 are shown below:

[0123] Step 502: If the first detection result is unreliable, the second processor enters a sleep state, and the first processor acquires the second image data, performs target detection on the second image data, and obtains the second detection result.

[0124] If the first detection result is unreliable, in order to avoid the problem of poor control accuracy of security equipment, the second processor will enter a sleep state, and the first processor will verify whether the target exists.

[0125] The first processor then acquires the second image data of the target area, performs target detection on the second image data, and obtains the second detection result. The second image data can be image data in the YUV domain or image data that has undergone image quality processing to ensure the accuracy of the second detection result.

[0126] Step 503: If the second detection result indicates that there is no target in the target area, the first processor enters a sleep state.

[0127] Step 504: If the second detection result indicates that a target exists in the target area, the first processor performs image processing on the second image data.

[0128] After the first processor confirms that there is no target in the target area, it indicates that the first processor has been mistakenly triggered and enters a sleep state. After the first processor confirms that there is a target in the target area, the first processor enters a normal working state and performs image processing on the second image data, such as performing target detection on the second image data, saving the target detection results, saving the second image data to obtain a video stream, etc.

[0129] In this way, by checking whether the first detection result is reliable, the second processor will instruct the first processor to enter a sleep state or instruct the first processor to perform image processing on the second image data output by the image signal processor only if the first detection result is reliable. If the first detection result is unreliable, the first processor will confirm whether there is a target in the target area, thus ensuring the control accuracy of the security equipment.

[0130] In one embodiment, based on Figure 5 The embodiment shown below describes another possible implementation of how the second processor determines whether the first detection result is reliable.

[0131] Please see Figure 6 Step 501 may include Figure 6 Steps 601 and 602 are shown below:

[0132] Step 601: The second processor acquires a visual capability model, which is used to characterize the visual capabilities of the second processor.

[0133] The second processor can acquire a visual capability model, which characterizes the visual capabilities of the second processor. In one possible implementation of step 601, the second processor can receive a visual capability model sent by another device via a communication module.

[0134] In another possible implementation of step 601, the second processor may also actively read the visual capability model from a preset storage location. The visual capability model may be established and stored in the preset storage location by the first processor after verifying the visual capability of the second processor in advance. The process of the first processor establishing the visual capability model will be described in the following embodiments.

[0135] Step 602: The second processor determines whether the first detection result is credible based on the visual capability model.

[0136] After the second processor obtains the visual capability model of the second processor, if the visual capability of the second processor meets the requirements, it indicates that the first detection result obtained by the second processor from the first image data is reliable; otherwise, it indicates that the first detection result obtained by the second processor from the first image data is unreliable.

[0137] As one implementation of step 602, the visual capability model may include a capability field, the different values ​​of which indicate different visual capabilities of the second processor.

[0138] If the capability field indicated by the visual capability model is the first value, it indicates that the visual capability of the second processor meets the requirements, and the second processor then determines that the first detection result is reliable.

[0139] If the capability field indicated by the visual capability model is the second value, it indicates that the visual capability of the second processor does not meet the requirements, and the second processor determines that the first detection result is unreliable.

[0140] The first value and the second value are different. For example, the first value is 1 and the second value is 0, or the first value is Y and the second value is N, and so on.

[0141] In this way, the second processor can conveniently, quickly and accurately determine its visual capabilities by acquiring the visual capability model, thus avoiding the problem of poor control accuracy of security equipment caused by the second processor's visual capabilities not meeting the requirements.

[0142] The following section describes the process by which the first processor establishes a visual capability model.

[0143] See Figure 7 The process by which the first processor establishes or updates the visual capability model may include Figure 7 Steps 701-703 are shown below:

[0144] Step 701: The second processor performs target detection on the first sample image data of the sample region, obtains a third detection result, and sends the first sample image data and the third detection result to the first processor.

[0145] During the establishment phase of the visual capability model of the second processor, the second processor acquires the first sample image of the sample area. The so-called sample area refers to the coverage area of ​​the security equipment. The sample area may be the same as the target area or it may be different from the target area, depending on the specific installation location of the security equipment.

[0146] The first sample image data can be the original image data of the sample area collected by the image signal processor. The second processor performs target detection on the first sample image data to obtain a third detection result. The method by which the second processor performs target detection on the first sample image data to obtain the third detection result is similar to the method by which the second processor performs target detection on the first image data to obtain the first detection result, and will not be described again here.

[0147] After obtaining the third detection result, the second processor sends the first sample image data and the third detection result to the first processor.

[0148] Step 702: The first processor obtains the second sample image data based on the first sample image data, and performs target detection on the second sample image data to obtain the fourth detection result.

[0149] The second sample image data is obtained by performing image quality processing on the first sample image data. After receiving the first sample image data and the third detection result, the first processor performs image quality processing on the first sample image data in the RGB domain and the YUV domain sequentially through the image signal processor to obtain the second sample image data, which is image data in the YUV domain.

[0150] The first processor uses a visual algorithm in the YUV domain to perform target detection on the second sample image data, and obtains the fourth detection result.

[0151] Step 703: The first processor uses the fourth detection result to verify the third detection result, and establishes a visual capability model based on the verification result.

[0152] The first processor uses the fourth detection result to verify the third detection result, thereby establishing the visual capability model of the second processor.

[0153] In one possible implementation, there can be multiple first sample image data sets, with each first sample image data set corresponding to a third detection result. Accordingly, there can also be multiple second sample image data sets and multiple fourth detection results.

[0154] For each third detection result, the first processor can check whether the third detection result is the same as the corresponding fourth detection result. If they are the same, the third detection result is verified as passed; otherwise, the third detection result is not verified.

[0155] If the percentage of verified third detection results in each third detection result is greater than a preset threshold (including cases where all verification results corresponding to each third detection result are verified), the first processor establishes a visual capability model. For example, it sets the capability field indicated by the visual capability model to a first value. This visual capability model is used to characterize that the visual capability of the second processor meets the requirements. If the percentage of verified third detection results in each third detection result is less than a preset threshold (including cases where all verification results corresponding to each third detection result are verified as unsuccessful), the first processor establishes a visual capability model. For example, it sets the capability field indicated by the visual capability model to a second value. This visual capability model is used to characterize that the visual capability of the second processor does not meet the requirements.

[0156] Then, the first processor stores the visual capability model in a preset storage location that the second processor can read. During the control process of the security device, when the second processor needs to confirm whether the first detection result is reliable, it can read the visual capability model from the preset storage location and then determine whether the first detection result is reliable based on the visual capability model.

[0157] In this embodiment, after the first processor establishes the visual capability model, it can also update the visual capability model during application. For example, the second processor can periodically send first image data (the original image data of the target area acquired by the image signal processor) and a first detection result to the first processor. Then, similar to the process of the first processor establishing the visual capability model, the first processor obtains the corresponding YUV domain image data (let's call it the third image data) based on the first image data, performs target detection on the third image data, and obtains a fifth detection result. The first processor then verifies the first detection result based on the fifth detection result. If the verification fails, the visual capability model is updated; otherwise, the visual capability model remains unchanged.

[0158] In this way, the visual capability model can be flexibly updated during application, which can ensure the reliability of the visual capability of the second processor. The second processor can then more accurately detect whether there is a target in the target area, thereby more accurately instructing the first processor to enter a sleep state or to perform image processing on the second image data output by the image signal processor.

[0159] In one embodiment, a control method for a security device is provided. The security device includes a sensor, a first processor, and a second processor. The power consumption of the second processor is less than that of the first processor. See [link to relevant documentation]. Figure 8 The method includes the following steps:

[0160] Step 801: The sensor detects the target in the target area and wakes up the first processor and the second processor.

[0161] The security equipment may also include a low-power WIFI module. In step 801, the sensor senses the target area and sends a trigger signal to the low-power WIFI module after sensing the target. The low-power WIFI module then generates a first wake-up signal and a second wake-up signal. The first processor receives the first wake-up signal sent by the low-power WIFI module and enters the wake-up state according to the indication of the first wake-up signal. The second processor receives the second wake-up signal sent by the low-power WIFI module and enters the wake-up state according to the indication of the second wake-up signal.

[0162] Step 802: The second processor acquires the first image data of the target area.

[0163] The first image data is the original image data of the target area acquired by the image signal processor.

[0164] Step 803: The second processor sends the first image data to the neural network processor.

[0165] Step 804: The neural network processor performs target detection on the first image data, obtains a first detection result, and returns the first detection result to the second processor.

[0166] Step 805: The second processor reads the visual capability model from the preset storage location.

[0167] Step 806: The second processor determines whether the capability field indicated by the visual capability model is the first value.

[0168] Step 807: If the capability field indicated by the visual capability model is the first value, the second processor determines that the first detection result is reliable.

[0169] Step 808: The second processor checks whether the first detection result indicates that a target exists in the target area.

[0170] Step 809: If the first detection result indicates that there is no target in the target area, the second processor instructs the first processor to enter a sleep state, and the second processor enters a sleep state.

[0171] Step 810: If the first detection result indicates that a target exists in the target area, the second processor instructs the first processor to perform image processing on the second image data, and the second processor enters a sleep state.

[0172] The second image data is obtained by the image signal processor through image quality processing of the first image data.

[0173] Step 811: If the capability field indicated by the visual capability model is the second value, the second processor determines that the first detection result is unreliable, the second processor enters a sleep state, and the first processor acquires the second image data of the target area. The first processor performs target detection on the second image data to obtain the second detection result.

[0174] Step 812: If the second detection result indicates that there is no target in the target area, the first processor enters a sleep state; if the second detection result indicates that there is a target in the target area, the first processor performs image processing on the second image data output by the image signal processor.

[0175] The visual capability model is pre-built by the first processor and stored in a preset storage location. The construction process of the visual capability model includes:

[0176] 1) The second processor performs target detection on the first sample image data of the sample area, obtains the third detection result, and sends the first sample image data and the third detection result to the first processor. The first sample image data is the original image data collected by the image signal processor.

[0177] 2) The first processor obtains the second sample image data based on the first sample image data, and performs target detection on the second sample image data to obtain the fourth detection result. The second sample image data is obtained by image quality processing of the first sample image data.

[0178] 3) The first processor uses the fourth detection result to verify the third detection result, and establishes a visual ability model based on the verification result.

[0179] In the above embodiments, the second processor detects and identifies the first image data before image quality processing to control whether the main controller needs to enter a sleep state. The second processor does not need to have the same high performance as the first processor. A low-power processor can be used for the second processor, thereby saving unnecessary power consumption caused by false triggering.

[0180] For example, see Figure 9 , Figure 9 This is a schematic diagram comparing the duration of determining a false trigger in this embodiment with the duration of determining a false trigger in conventional technology.

[0181] Depend on Figure 9 It can be seen that the time required to determine a false trigger in the traditional technology is about 5 times that required to determine a false trigger in the present application embodiment. Therefore, the present application embodiment significantly reduces the working time of a false trigger and saves the power consumption of the camera during false triggering.

[0182] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0183] Based on the same inventive concept, this application also provides a control device for security equipment to implement the control method for the security equipment described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more embodiments of the control device for security equipment provided below can be found in the limitations of the control method for security equipment described above, and will not be repeated here.

[0184] In one exemplary embodiment, such as Figure 10 As shown, a control device for a security device is provided. The security device includes a first processor, a second processor, and an image signal processor. The power consumption of the second processor is less than that of the first processor. The device includes:

[0185] The acquisition module 1001 is used to acquire first image data of the target area through the second processor, wherein the first image data is the original image data of the target area collected by the image signal processor;

[0186] Detection module 1002 is used to perform target detection on the first image data through the second processor to obtain a first detection result;

[0187] The control module 1003 is used to instruct the first processor to enter a sleep state or to instruct the first processor to perform image processing on the second image data output by the image signal processor based on the first detection result, wherein the second image data is obtained by the image signal processor performing image quality processing on the first image data.

[0188] In one embodiment, the control module 1003 is specifically configured to, if the first detection result indicates that there is no target in the target area, instruct the first processor to enter a sleep state via the second processor; and if the first detection result indicates that there is a target in the target area, instruct the first processor to perform image processing on the second image data via the second processor.

[0189] In one embodiment, the control module 1003 is further configured to control the first processor to enter a sleep state or to perform image processing on the second image data according to the instructions of the second processor.

[0190] In one embodiment, the control module 1003 is further configured to control the second processor to enter a sleep state.

[0191] In one embodiment, the device further includes:

[0192] The determination module is used to determine whether the first detection result is reliable through the second processor;

[0193] The control module 1003 is specifically used to, if the first detection result is reliable, instruct the first processor to enter a sleep state or instruct the first processor to perform image processing on the second image data based on the first detection result.

[0194] In one embodiment, the control module 1003 is further configured to, if the first detection result is unreliable, control the second processor to enter a sleep state, and obtain second image data of the target area through the first processor, and perform target detection on the second image data to obtain a second detection result; if the second detection result indicates that there is no target in the target area, control the first processor to enter a sleep state; if the second detection result indicates that there is a target in the target area, control the first processor to perform image processing on the second image data.

[0195] In one embodiment, the determining module is specifically used to obtain a visual capability model through the second processor, the visual capability model being used to characterize the visual capability of the second processor; and to determine whether the first detection result is reliable based on the visual capability model by the second processor.

[0196] In one embodiment, the determining module is specifically used to determine the first detection result as reliable by the second processor if the capability field indicated by the visual capability model is a first value; and to determine the first detection result as unreliable by the second processor if the capability field indicated by the visual capability model is a second value, wherein the first value and the second value are different.

[0197] In one embodiment, the determining module is specifically used to read the visual capability model from a preset storage location by the second processor. The visual capability model is established by the first processor after verifying the visual capability of the second processor and stored in the preset storage location.

[0198] In one embodiment, the control module 1003 is further configured to perform target detection on the first sample image data of the sample region through the second processor to obtain a third detection result, and send the first sample image data and the third detection result to the first processor, wherein the first sample image data is the original image data of the sample region acquired by the image signal processor; the first processor obtains second sample image data based on the first sample image data, and performs target detection on the second sample image data to obtain a fourth detection result, wherein the second sample image data is obtained by image quality processing of the first sample image data; the first processor verifies the third detection result using the fourth detection result, and establishes the visual capability model based on the verification result.

[0199] In one embodiment, the security device further includes a neural network processor, and the detection module 1002 is specifically used to send the first image data to the neural network processor through the second processor; to perform target detection on the first image data through the neural network processor to obtain the first detection result, and to return the first detection result to the second processor.

[0200] In one embodiment, the device further includes:

[0201] The first receiving module is configured to receive a first wake-up signal sent by the low-power communication module through the first processor, and enter the wake-up state according to the indication of the first wake-up signal;

[0202] The second receiving module is used to receive the second wake-up signal sent by the low-power communication module through the second processor, and enter the wake-up state according to the instruction of the second wake-up signal;

[0203] The first wake-up signal and the second wake-up signal are generated by the low-power communication module based on the sensor's perception results of the target area.

[0204] Each module in the control device of the aforementioned security equipment can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of the security equipment in hardware form or independent of it, or stored in the memory of the security equipment in software form, so that the processor can call and execute the corresponding operations of each module.

[0205] In one exemplary embodiment, a security device is provided, the internal structure of which can be shown in the following diagram. Figure 11 As shown, the security device includes a first processor and a second processor, a memory, an input / output interface (I / O), and a communication interface. The first and second processors, the memory, and the I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The first and second processors provide computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage medium. The database stores control data for the security device. The I / O interfaces allow the first and second processors to exchange information with external devices. The communication interface allows communication with external terminals via a network connection. When the computer program is executed by the first and second processors, it implements a control method for the security device.

[0206] Those skilled in the art will understand that Figure 10 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the security equipment to which the present application is applied. Specific security equipment may include more or fewer components than those shown in the figure, or may combine certain components, or may have different component arrangements.

[0207] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the control method for the security device in any of the above embodiments.

[0208] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the control method for the security device in any of the above embodiments.

[0209] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to images, data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0210] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0211] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0212] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A control method of a security device, characterized by, The security device comprises a first processor, a second processor and an image signal processor, the power consumption of the second processor is less than that of the first processor, and the security device comprises: The second processor acquires first image data of a target area, the first image data being original image data of the target area collected by the image signal processor; The second processor performs target detection on the first image data to obtain a first detection result; The second processor instructs the first processor to enter a sleep state or instructs the first processor to perform image processing on second image data output by the image signal processor according to the first detection result, the second image data being obtained by performing quality processing on the first image data by the image signal processor.

2. The method of claim 1, wherein, The second processor instructs the first processor to enter a sleep state or instructs the first processor to perform image processing on second image data output by the image signal processor according to the first detection result, comprising: If the first detection result is that there is no target in the target area, the second processor instructs the first processor to enter a sleep state; If the first detection result is that there is a target in the target area, the second processor instructs the first processor to perform image processing on the second image data.

3. The method of claim 1, wherein, The method further comprises: The second processor determines whether the first detection result is reliable; The second processor instructs the first processor to enter a sleep state or instructs the first processor to perform image processing on second image data output by the image signal processor according to the first detection result, comprising: If the first detection result is reliable, the second processor instructs the first processor to enter a sleep state or instructs the first processor to perform image processing on the second image data according to the first detection result.

4. The method of claim 3, wherein, The method further comprises: If the first detection result is not reliable, the second processor enters a sleep state, and the first processor acquires the second image data, performs target detection on the second image data to obtain a second detection result; If the second detection result is that there is no target in the target area, the first processor enters a sleep state; If the second detection result is that there is a target in the target area, the first processor performs image processing on the second image data.

5. The method of claim 3, wherein, The second processor determines whether the first detection result is reliable, comprising: The second processor acquires a visual capability model, the visual capability model being used to represent the visual capability of the second processor; The second processor determines whether the first detection result is reliable according to the visual capability model.

6. The method of claim 5, wherein, The second processor determines whether the first detection result is reliable according to the visual capability model, comprising: If the capability field indicated by the visual capability model is a first value, the second processor determines that the first detection result is reliable; If the capability field indicated by the visual capability model is a second value, the second processor determines that the first detection result is not reliable, the first value and the second value being different.

7. The method of claim 5, wherein, The second processor acquires a visual capability model, comprising: The second processor reads the visual capability model in a preset storage location, the visual capability model being established and stored in the preset storage location by the first processor after verifying the visual capability of the second processor.

8. The method of claim 5, wherein, The method further comprises: The second processor performs target detection on first sample image data of a sample region to obtain third detection results, and sends the first sample image data and the third detection results to the first processor, the first sample image data being original image data of the sample region collected by the image signal processor; The first processor acquires second sample image data from the first sample image data, and performs target detection on the second sample image data to obtain fourth detection results, the second sample image data being obtained by performing quality processing on the first sample image data; The first processor verifies the third detection results using the fourth detection results, and establishes the visual capability model according to the verification results.

9. The method of claim 1, wherein, The security device further comprises a neural network processor, and the second processor performs target detection on the first image data to obtain first detection results, comprising: The second processor sends the first image data to the neural network processor; The neural network processor performs target detection on the first image data to obtain the first detection results, and returns the first detection results to the second processor.

10. The method of claim 1, wherein, The security device further comprises a low-power communication module and a sensor, and before the second processor acquires the first image data of the target region, the method further comprises: The first processor receives a first wake-up signal sent by the low-power communication module, and enters a wake-up state according to the indication of the first wake-up signal; The second processor receives a second wake-up signal sent by the low-power communication module, and enters a wake-up state according to the indication of the second wake-up signal; The first wake-up signal and the second wake-up signal are generated by the low-power communication module according to the sensing results of the sensor for the target region.

11. A control device of a security device, characterized by comprising: For a security device, the security device comprises a first processor, a second processor and an image signal processor, the power consumption of the second processor being less than that of the first processor, comprising: An acquisition module is configured to acquire, by the second processor, first image data of a target region, the first image data being original image data of the target region collected by the image signal processor; A detection module is configured to perform, by the second processor, target detection on the first image data to obtain first detection results; A control module is configured to instruct the first processor to enter a sleep state or to instruct the first processor to perform image processing on second image data output by the image signal processor according to the first detection result, the second image data being obtained by performing quality processing on the first image data by the image signal processor.

12. A security device, characterized in that The device comprises a memory, a first processor and a second processor, the power consumption of the second processor being less than that of the first processor, and the memory stores a computer program; wherein, The first processor, when executing the computer program, implements the steps performed by the first processor in the method of any one of claims 1 to 10. The second processor, when executing the computer program, implements the steps performed by the second processor in the method of any one of claims 1 to 10.

13. A computer readable storage medium having stored thereon a computer program, characterized in that The computer program, when executed by a processor, implements the steps of the method of any one of claims 1 to 10.

Images