Image processing method based on AOV camera, main controller and AOV camera

Abstract

The invention relates to the technical field of video recording, and discloses an image processing method based on an AOV camera, a main controller and the AOV camera, and the method comprises the steps: enabling the camera to receive a first wake-up signal; the camera enters a second state from a first state according to the first wake-up signal, and the camera power in the second state is not lower than the camera power in the first state; when the camera is in the second state, the following steps are synchronously executed once: step A, an original nth frame image is collected, and n is an integer greater than or equal to 2; b, performing post-processing on the original (n-1) th frame of image to obtain a processed (n-1) th frame of image; and the camera enters the first state again. In this way, the power consumption of the system can be reduced to a greater extent.

Description

Technical Field

[0001] This application relates to the field of video recording technology, specifically to an image processing method, a main controller, and an AOV camera based on an AOV camera. Background Technology

[0002] AOV, or Always On Video, is a low-power camera technology. The common practice is to keep the system in a low-power mode most of the time, switch to a high-power mode when recording video, and then switch back to a low-power mode after recording is complete.

[0003] In AOV (Area of ​​Effect) video recording, the recording of a single frame requires multiple processing steps. Consequently, the main control system needs to maintain high power consumption for a relatively long time to successfully output a single frame. Therefore, although AOV video recording technology can reduce system power consumption to some extent, the degree of reduction is limited. Summary of the Invention

[0004] In view of the above problems, this application provides an image processing method, main controller and AOV camera based on an AOV camera, which can reduce system power consumption to a greater extent. According to one aspect of the embodiments of this application, an image processing method based on an AOV camera is provided, comprising: the camera receiving a first wake-up signal; the camera entering a second state from a first state according to the first wake-up signal, wherein the camera power in the second state is not lower than the camera power in the first state; the camera synchronously executing the following steps once in the second state: step A: acquiring the original nth frame image, wherein n≥2 and n is an integer; step B: performing post-processing on the original (n-1)th frame image to obtain the processed (n-1)th frame image; the camera entering the first state again.

[0005] In one alternative approach, in the second state, while performing steps A and B, the camera also simultaneously performs step C: encoding and AI processing the processed (n-2)th frame image to obtain the final (n-2)th frame image and storing it in the video buffer, where n≥3.

[0006] In one alternative approach, the image processing method further includes: when a second wake-up signal is received, the camera enters a second state to synchronously execute steps A, B, and C multiple times at a preset frequency; the camera then enters the first state again.

[0007] In one alternative approach, the second wake-up signal is generated when the passive infrared PIR sensor alarms; when the second wake-up signal is received, the camera enters a second state to synchronously execute steps A, B, and C multiple times at a preset frequency, including: when the second wake-up signal is received, the camera enters a second state to continuously and synchronously execute steps A, B, and C at a first preset frequency until the PIR sensor alarm ends.

[0008] In one alternative approach, when a second wake-up signal is received, the camera enters a second state to continuously and synchronously execute steps A, B, and C at a first preset frequency until the PIR sensor alarm ends. This includes: when the second wake-up signal is received, the camera enters the second state; reading data from the PIR sensor to confirm whether actual motion has occurred; if so, continuously and synchronously executing steps A, B, and C at the first preset frequency until the PIR sensor alarm ends; if not, executing the step of the camera re-entering the first state.

[0009] In one alternative approach, the second wake-up signal is generated when there is a network connection; when the second wake-up signal is received, the camera enters a second state to synchronously execute steps A, B, and C multiple times at a preset frequency, including: when the second wake-up signal is received, the camera enters a second state to continuously and synchronously execute steps A, B, and C at a second preset frequency until the network connection ends.

[0010] In one alternative approach, the image processing method further includes: when a network connection is available, responding to a request from the network requesting end by uploading video stream data stored in the video buffer to the network requesting end in real time.

[0011] In one optional approach, step C includes: performing AI processing on the encoded (n-2)th frame image to detect whether a target object exists in the encoded (n-2)th frame image, and storing the final (n-2)th frame image obtained after AI processing in the video buffer; if a target object is detected in the encoded (n-2)th frame image, an alarm message is generated; the image processing method further includes: in response to the alarm message, the camera synchronously executes steps A, B and C multiple times at a third preset frequency; the camera re-enters the first state.

[0012] In one alternative approach, in response to an alarm message, the camera synchronously executes steps A, B, and C multiple times at a third preset frequency, including: in response to an alarm message, the camera continuously and synchronously executes steps A, B, and C at a third preset frequency until no more alarm messages are generated.

[0013] In one alternative approach, step B includes: performing effect processing and scaling processing on the original (n-1)th frame image to obtain the processed (n-1)th frame image.

[0014] According to another aspect of the embodiments of this application, an image processing method based on an AOV camera is provided, comprising: the camera receiving a first wake-up signal; the camera entering a second state from a first state according to the first wake-up signal, wherein the camera power in the second state is not lower than the camera power in the first state; the camera synchronously executing the following steps once in the second state: step A: acquiring the original nth frame image, wherein n≥4 and n is an integer; step B: performing post-processing on the original (n-1)th frame image to obtain the processed (n-1)th frame image; step C1: encoding the processed (n-2)th frame image to obtain the encoded (n-2)th frame image; step C2: performing AI processing on the encoded (n-3)th frame image to obtain the final (n-3)th frame image and storing it in the video buffer; the camera re-entering the first state.

[0015] According to another aspect of the embodiments of this application, an image processing method based on an AOV camera is provided, comprising: the camera receiving a first wake-up signal; the camera entering a second state from a first state according to the first wake-up signal, wherein the camera power in the second state is not lower than the camera power in the first state; the camera synchronously executing the following steps once in the second state: step A: acquiring the original nth frame image, wherein n≥4 and n is an integer; step B1: performing effect processing on the original (n-1)th frame image to obtain the effect-processed (n-1)th frame image; step B2: performing scaling processing on the effect-processed (n-2)th frame image to obtain a scaled-down (n-2)th frame image; step C: encoding and AI processing on the scaled-down (n-3)th frame image to obtain the final (n-3)th frame image and storing it in the video buffer; the camera re-entering the first state.

[0016] According to another aspect of the embodiments of this application, an image processing method based on an AOV camera is provided, comprising: the camera receiving a first wake-up signal; the camera entering a second state from a first state according to the first wake-up signal, wherein the camera power in the second state is not lower than the camera power in the first state; the camera synchronously executing the following steps once in the second state: Step A: acquiring the original nth frame image, wherein n≥5 and n is an integer; Step B1: performing effect processing on the original (n-1)th frame image to obtain the effect-processed (n-1)th frame image; Step B2: performing scaling processing on the effect-processed (n-2)th frame image to obtain a scaled-down (n-2)th frame image; Step C1: encoding the scaled-down (n-3)th frame image to obtain the encoded (n-3)th frame image; Step C2: performing AI processing on the encoded (n-4)th frame image to obtain the final (n-4)th frame image and storing it in the video buffer; the camera re-entering the first state.

[0017] According to another aspect of the embodiments of this application, a main controller is provided, which stores a computer program and is used to execute the computer program to implement the image processing method based on an AOV camera as described above.

[0018] According to another aspect of the embodiments of this application, an AOV camera is provided, characterized in that it includes: a sensor, an auxiliary controller and the aforementioned main controller, wherein the main controller is electrically connected to the sensor and the auxiliary controller respectively; when the main controller is in a first state, the auxiliary controller is used to send a first wake-up signal to the main controller to wake up the main controller to enter a second state; the main controller is used to control the sensor exposure in the second state.

[0019] In one alternative embodiment, the AOV camera further includes a PIR sensor electrically connected to an auxiliary controller; the PIR sensor is used to send an alarm signal to the auxiliary controller when a target is detected; the auxiliary controller is used to send a second wake-up signal to the main controller after receiving the alarm signal; the main controller is used to enter a second state after receiving the second wake-up signal.

[0020] In one alternative approach, the auxiliary controller includes a network chip electrically connected to the main controller; the network chip is also used to send a second wake-up signal to the main controller when a network connection is available, so as to wake the main controller into a second state.

[0021] In one alternative embodiment, the auxiliary controller further includes a microcontroller electrically connected to the main controller; the microcontroller is used to send a first wake-up signal to the main controller; the network chip is in a first state when there is no network connection, and exits the first state when there is a network connection, in order to send a second wake-up signal to the main controller.

[0022] In the image processing method provided in this application embodiment, when the system enters the second state to acquire and process images, in order to minimize the duration of high power consumption, a parallel processing approach is adopted for each step. In the image recording and processing process, the earlier step processes the image of the previous frame, and the later step processes the image of the next frame. Specifically, step A acquires the original nth frame image, and step B performs post-processing on the original (n-1)th frame image. This cleverly avoids the situation where the later processing step needs to wait for the output of the previous processing step, and realizes the synchronous execution of each step. By using them to process the corresponding frame images at the same time, the time required for the system to enter high power consumption and perform one image acquisition and processing is effectively shortened, thereby reducing the duration of power consumption peaks and reducing the total power consumption of the system.

[0023] The above description is merely an overview of the technical solutions of the embodiments of this application. In order to better understand the technical means of the embodiments of this application and to implement them in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the embodiments of this application more obvious and understandable, specific implementation methods of this application are described below. Attached Figure Description

[0024] The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 This is a schematic diagram illustrating the relationship between current and time in a traditional AOV camera system with a frame / s output. Figure 2 A schematic flowchart of the image processing method provided in the first embodiment of this application; Figure 3 A schematic flowchart illustrating the image processing method provided in the second embodiment of this application; Figure 4 A tabular illustration of the image processing method provided in the second embodiment of this application; Figure 5 A schematic flowchart illustrating the image processing method provided in the third embodiment of this application; Figure 6 A schematic flowchart of the image processing method provided in the fourth embodiment of this application; Figure 7 A schematic flowchart of the image processing method provided in the fifth embodiment of this application; Figure 8 This is a schematic diagram of the modular structure of the AOV camera provided in an embodiment of this application. Detailed Implementation

[0025] Exemplary embodiments of the present application will now be described in more detail with reference to the accompanying drawings. Although exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application may be implemented in various forms and should not be limited to the embodiments set forth herein.

[0026] Taking an AOV camera solution with 1 frame / s as an example, the system's power consumption peaks every 1 second. The specific power consumption changes are as follows: Figure 1 As shown in the figure, the horizontal axis represents time, the vertical axis represents current, and the current is proportional to the system power consumption.

[0027] from Figure 1 As can be seen, the system power consumption changes periodically, specifically increasing to a peak at predetermined intervals and then decreasing. Specifically, for the recording method of 1 frame / s, the time interval between the two peaks highlighted in the figure is 1 second. During this 1 second, the system is in a low-power state, while during the process from the formation of the peak to its disappearance, the system is in a high-power state for recording images.

[0028] Therefore, the main factors determining the system's power consumption are the duration and magnitude of the power spikes. This application primarily focuses on optimizing the duration of power spikes.

[0029] Currently, when the system enters a high-power state and records, the process of outputting a single frame of image is relatively long, requiring sequential steps such as sensor exposure, post-processing, video encoding, and AI processing. Since the data processed in each subsequent step is the result of the previous step, it needs to be processed step by step. This undoubtedly causes the system to remain in a high-power state for a relatively long time, resulting in a large total power consumption.

[0030] In response, this application proposes an image processing method based on an AOV camera, which processes images of consecutive frames in parallel to minimize the duration of high power consumption in the system, i.e., reduce the duration of peak power consumption, thereby effectively reducing the power consumption of the camera system.

[0031] Specifically, please refer to Figure 2 The figure shows a flowchart of an image processing method based on an AOV camera provided in an embodiment of this application. The image processing method includes the following steps: Step 110: The camera receives the first wake-up signal.

[0032] Step 130: The camera enters the second state from the first state according to the first wake-up signal, wherein the camera power in the second state is not lower than the camera power in the first state.

[0033] The first state, also known as the low-power state mentioned earlier, can be achieved using the Linux system's suspend mechanism. Linux's suspend mechanism is a system-level energy-saving feature that allows the computer to enter a low-power state to conserve energy without completely shutting down. When the system enters suspend mode, it saves the current program and register information to memory, then shuts down the CPU and other unnecessary hardware components, putting the system into a "frozen" state. At this time, memory enters self-refresh mode to maintain its contents, while other hardware components cease operation, thus significantly reducing power consumption.

[0034] The second state is the high-power state, where the camera's power consumption is no less than that of the camera in the first state. Preferably, the camera's power consumption in the second state should be higher than that of the camera in the first state.

[0035] In the second state, the camera synchronously executes steps A and B once.

[0036] Step A: Acquire the original nth frame image, where n≥2 and n is an integer.

[0037] Among them, image acquisition can be done using a sensor, including CCD (Charge-Coupled Device) and CMOS (Complementary Metal-Oxide-Semiconductor). Considering speed and power consumption factors, CMOS is usually used for sensors.

[0038] The sensor primarily converts light signals into electrical signals, then captures these electrical signals to obtain the original nth frame image. This original nth frame image can be a RAW image, which records the raw light signal data captured by the digital camera sensor. This RAW image can be a Bayer array. A Bayer array, by placing one of the red, green, or blue filters in front of each pixel, allows the camera to capture color images at a relatively low cost and high resolution. However, it also requires de-Bayer processing in post-processing to reconstruct the complete color image.

[0039] Step B: Perform post-processing on the original (n-1)th frame image to obtain the processed (n-1)th frame image.

[0040] Since the original (n-1)th frame image is obtained based on mechanical light perception, which does not conform to the light perception of the human eye, it needs to be post-processed. Specifically, post-processing can include effects processing, scaling, etc.

[0041] Effects processing can include gamma correction, deBayer (for schemes where the original image is a Bayer array), noise reduction, sharpening, color correction, etc.

[0042] Gamma correction is used to adjust image brightness and contrast, improving the visual effect of the image on the display device through nonlinear transformation. DeBayer reconstruction reconstructs a complete color image from the raw data obtained from the Bayer array. Through interpolation and color reconstruction, the deBayer algorithm can remove the mosaic effect in the raw data and obtain a high-quality color image. Noise reduction is mainly used to reduce noise components in the image and improve image quality. Specifically, it can be achieved through spatial domain filtering, frequency domain filtering, wavelet transform, deep learning methods, etc. Sharpening is used to enhance the edges and details of the image, making the image look clearer and more vivid. Specifically, it can be achieved through spatial domain sharpening algorithms, frequency domain sharpening algorithms, etc. Color adjustment mainly adjusts the color, contrast, saturation, and hue of the image to obtain the desired visual effect. Specifically, it can be achieved through color correction, color grading, and color matching techniques. Color adjustment can effectively improve image quality.

[0043] As the name suggests, scaling is used to change the size of an image. By reducing the size of the image, the time and power consumption of subsequent processing steps can be reduced. Specifically, image scaling can be achieved through methods such as nearest neighbor interpolation, bilinear interpolation, bicubic interpolation, and Lanzos interpolation.

[0044] Further, please refer to Figure 3 In some embodiments, when the camera performs steps A and B in the second state, it can also simultaneously perform step C. It should be noted that for the scheme of simultaneously performing steps A, B and C, n≥3.

[0045] Step C: Encode and perform AI processing on the processed (n-2)th frame image to obtain the final (n-2)th frame image and store it in the video buffer.

[0046] The processed (n-2)th frame image is relatively large, which makes it difficult to transmit and store it over the network. Encoding the processed (n-2)th frame image can effectively reduce its size. Specifically, the processed (n-2)th frame image can be encoded using the H.264 or H.265 encoding standard.

[0047] AI processing refers to using artificial intelligence technology, especially AI learning and deep learning algorithms, to analyze the (n-2)th frame of an image, detect and identify specific targets in the image, such as people, vehicles, and animals. Furthermore, it can analyze the behavior of the targets based on the recognition results, such as the actions of people or animals, and the driving trajectory of vehicles. Of course, it can also determine whether there are any abnormal situations based on the recognition results, such as intrusion detection and fire alarm lights.

[0048] By combining AI processing with image encoding, more efficient and intelligent video surveillance and recording functions can be achieved, providing richer information analysis and more accurate event response to meet the needs of various application scenarios.

[0049] A video buffer is a memory area in the system used to temporarily store video data. Its main function is to provide temporary storage space during video data acquisition, processing, and playback to address the mismatch between the data generation rate and the data processing or usage rate. The video buffer ensures smooth playback and processing of video data, avoiding stuttering or frame drops caused by data rate mismatch.

[0050] Based on the above description of step C, it can be seen that in some embodiments, step C is not necessary, and in this case, only steps A and B need to be executed simultaneously.

[0051] It is important to emphasize that in the image processing method provided in this application embodiment, step A is to acquire the original nth frame image, step B is to perform post-processing on the original (n-1)th frame image, and step C is to encode and perform AI processing on the processed (n-2)th frame image. Each step processes images of different frames. Therefore, steps A, B, and C are executed synchronously, that is, they are processed in parallel. This can effectively shorten the duration of the system processing one frame image in the second state.

[0052] Step 150: The camera returns to the first state.

[0053] Specifically, in step 110, the first wake-up signal can be received periodically, and the second state can be entered periodically to synchronously execute steps A, B and C, so as to achieve continuous and uninterrupted recording. For example, the first wake-up signal can be received once every 1 second, so as to record video at 1 frame / s.

[0054] In the first state, the system cannot recover on its own and requires an externally triggered interrupt to wake it up; that is, it needs to receive a first wake-up signal to exit the first state. Therefore, in terms of hardware, a low-power network chip or microcontroller can be connected to the main chip, and the network chip or microcontroller can periodically send a first wake-up signal to the main control chip to make the main control chip exit the first state for recording.

[0055] For a more intuitive understanding, please refer to [link / reference]. Figure 4 In the diagram, each vertical column represents the steps executed after receiving the first wake-up signal, and each horizontal row represents the frame of the image processed in that step. As shown in the diagram, after receiving the first wake-up signal for the first time, only step A is executed, acquiring the original first frame image, and then the system returns to the first state. After a predetermined interval, the first wake-up signal is received for the second time, and steps A and B are executed simultaneously. Step A acquires the original second frame image, while step B processes the previously acquired original first frame image, and then the system returns to the first state. After another predetermined interval, the first wake-up signal is received for the third time, and steps A, B, and C are executed simultaneously. Step A acquires the original third frame image, step B processes the previously acquired original second frame image, and step C processes the processed first frame image obtained from step B. Step C encodes and performs AI processing on the processed first frame image to generate the first frame of the video. This process continues in this manner, forming a continuous video recording.

[0056] It should be noted that before entering the first state by executing step 150, it is necessary to wait for steps A, B and C to complete their work. Otherwise, if the modules executing steps A, B or C lose power directly after entering the first state, it will cause image abnormalities or module abnormalities.

[0057] In the image processing method provided in this application embodiment, when the system enters the second state to acquire and process images, in order to minimize the duration of high power consumption, a parallel processing approach is adopted for each step. In the image recording and processing process, the earlier step processes the image of the previous frame, and the later step processes the image of the next frame. Specifically, step A acquires the original nth frame image, and step B performs post-processing on the original (n-1)th frame image. This cleverly avoids the situation where the later processing step needs to wait for the output of the previous processing step, and realizes the synchronous execution of each step. By using them to process the corresponding frame images at the same time, the time required for the system to enter high power consumption and perform one image acquisition and processing is effectively shortened, thereby reducing the duration of power consumption peaks and reducing the total power consumption of the system.

[0058] Furthermore, since the image processing method provided in this application embodiment shortens the processing time of a video frame, correspondingly, when the system needs to remain in the second state and perform high-frequency video recording, the shorter processing time of a video frame can provide higher frequency setting support.

[0059] In embodiments where graphics encoding and AI processing are required, step C can be further executed synchronously during the execution of steps A and B to encode and perform AI processing on the processed (n-2)th frame image.

[0060] Under normal conditions, in order to reduce power consumption, the recording frequency is set to be relatively low, usually 1 frame / s. This is obviously insufficient to meet the detailed analysis of alarm events in case of emergencies, and it is easy to miss fast-moving objects or events that occur in an instant.

[0061] Therefore, the image processing method provided in this application embodiment may further include the following steps: Step 170: When the second wake-up signal is received, the camera enters the second state.

[0062] The second wake-up signal can be triggered in the event of an emergency, such as unauthorized personnel entering the monitored area, the appearance of abnormally moving objects, personnel exhibiting unusual behavior, or a fire. Alternatively, the second wake-up signal can also be generated upon a network request, such as a user terminal requesting to view real-time monitoring footage.

[0063] Step 190: The camera synchronously executes steps A, B, and C multiple times at a preset frequency.

[0064] When the main control chip receives the second wake-up signal in step 180 and enters the second state, it indicates that high-frequency recording is required. The main control chip will remain in the second state for a relatively long time. Therefore, in step 190, steps A, B, and C will be executed synchronously multiple times at a predetermined frequency. Between two adjacent executions, the main control chip will not return to the first state. The predetermined frequency can be, for example, 15 frames / s, etc., and the specific frequency is not limited here.

[0065] In addition, it should be noted that the number of times or duration of image capture after receiving the second wake-up signal can be preset. That is, in step 190, steps A, B and C are executed synchronously for a preset number of times or for a preset duration at a preset frequency. Of course, the number of times or duration of image capture can also be not preset, but can end based on the actual situation. For example, steps A, B and C can be executed continuously and synchronously at a predetermined frequency. When the emergency ends or the network request ends, the synchronous execution of steps A, B and C will also stop accordingly.

[0066] Step 210: The camera returns to the first state.

[0067] In this embodiment, when the main control chip is woken up from the first state by the second wake-up signal, unlike the first wake-up signal, it will remain in the second state for a longer period of time and synchronously execute steps A, B, and C multiple times at a predetermined frequency to achieve high-frequency recording. This design is mainly to capture the process and details of events in more detail in case of abnormal situations, providing more complete video evidence for subsequent analysis, and to capture fast-moving objects or instantaneous events. For real-time viewing of monitoring screens or review of historical videos, it can improve video smoothness and make frame-by-frame video analysis more coherent.

[0068] In security monitoring applications, the determination of whether there are any anomalies is generally based on AI processing results. Figure 3 As can be seen from the embodiments of this application, the image processing method only generates the first frame of the video when the first wake-up signal is received for the third time, that is, it only outputs the AI ​​processing result of the first frame. Taking a recording of 1 frame / s as an example, the AI ​​processing result of the first frame will only be generated at the 3rd second, which means there is a long delay in the AI ​​processing result. If the AI ​​processing result of the first frame indicates an abnormal situation, then the abnormal situation has actually passed 2 seconds. If the main control chip is woken up at this time for high-frequency recording or alarm, it may not be possible to capture the occurrence of the critical event. If the duration of the abnormal situation is less than 2 seconds, then by the time the main control chip is woken up to respond, the abnormal situation has already ended.

[0069] To address this and ensure timely response to abnormal situations, a PIR sensor can be configured on the AOV camera to detect the monitored area. The second wake-up signal is generated when the passive infrared PIR sensor alarms. Since the active chip does not affect the real-time detection of the PIR sensor in the first state, there is no delay in triggering the second wake-up signal through the PIR sensor alarm.

[0070] For the scheme where the second wake-up signal is generated when the PIR sensor alarms, step 190 includes: Step 193: Execute steps A, B and C continuously and synchronously at the first preset frequency until the PIR sensor alarm ends.

[0071] In this embodiment, the alarm is detected by a PIR sensor to initiate high-frequency recording, which can respond to abnormal situations in a timely manner. During the PIR alarm process, high-frequency recording is maintained continuously until the alarm ends, ensuring that the abnormal event can be recorded completely, providing data support for subsequent analysis and evidence collection.

[0072] PIR sensors primarily detect the presence and movement of objects by detecting the infrared radiation they emit. Changes in an object's temperature affect the intensity of its infrared radiation; consequently, the detection accuracy of PIR sensors is significantly influenced by ambient temperature. For example, when the ambient temperature is close to the temperature of the object being detected (i.e., the temperature difference is small), detection errors may occur. Furthermore, direct sunlight or strong light shining on the PIR sensor, as well as the presence of objects reflecting infrared radiation within the monitored area, such as mirrors or reflective glass lamps, can also cause false alarms from the PIR sensor.

[0073] To avoid increased power consumption due to false alarms from the PIR sensor, this application further proposes an implementation method, specifically, prior to step 193, which includes: Step 191: Read the data from the PIR sensor to confirm whether actual motion has occurred.

[0074] Specifically, a time window can be set to analyze sensor data over a period of time. If abnormal motion signals are detected multiple times within a short period, it is confirmed as genuine motion; if it is only detected once or twice occasionally, it is confirmed as a false alarm. Alternatively, the intensity of the signal detected by the PIR sensor can be analyzed. If the signal intensity is higher than or equal to an intensity threshold, it is confirmed as genuine motion; conversely, if the signal intensity is lower than the intensity threshold, it is confirmed as a false alarm.

[0075] If step 191 determines that it is true, it indicates that it is not a false alarm, and step 193 is executed for high-frequency continuous recording. If step 191 determines that it is false, it indicates that the PIR sensor is false, and step 210 is executed accordingly: the camera re-enters the first state to avoid excessive power consumption caused by false alarms from the PIR sensor.

[0076] The second wake-up signal can also be generated when there is a network connection. In this case, step 190 may include: 192: Execute steps A, B and C continuously and synchronously at the second preset frequency until the network connection ends.

[0077] The second preset frequency may be equal to or different from the first preset frequency mentioned above; no specific limitation is made here.

[0078] When there is a network connection, such as when a user terminal accesses and views a real-time monitoring screen, the main control chip continuously and synchronously executes steps A, B, and C at a second preset frequency in the second state to ensure smooth video playback and improve the user experience. When the network connection ends, it automatically exits high-frequency recording and re-enters the first state to ensure low power consumption.

[0079] When a network connection is available, in response to a request from the network requester (i.e., the user terminal), the video stream data stored in the video buffer can be uploaded to the network requester in real time, so that the user terminal can view the monitoring screen in real time.

[0080] Of course, the image processing method provided in this application embodiment can also achieve assisted alarm through AI processing results. Specifically, step C may include: Step CI: Perform AI processing on the encoded (n-2)th frame image to detect whether there is a target object in the encoded (n-2)th frame image, and store the final (n-2)th frame image obtained after AI processing into the video buffer.

[0081] Step CII: If a target object is detected in the encoded (n-2)th frame image, an alarm message is generated.

[0082] This image processing method also includes: Step 230: In response to the alarm information, execute steps A, B and C synchronously multiple times according to the third preset frequency.

[0083] Among them, the third preset frequency, the first preset frequency and the second preset frequency mentioned above can all be equal, any two of them can be equal, or none of them can be equal.

[0084] 250: The camera returns to the first state.

[0085] Similar to the processing method mentioned above for receiving the second wake-up signal, specifically, when generating alarm information, steps A, B, and C can be executed synchronously for a preset number of times or a preset duration according to the third preset frequency.

[0086] Similar to the further processing method after receiving the second wake-up signal, high-frequency recording can be continuously performed before the alarm ends. Specifically, step 230 can further include step 231: in response to the alarm information, steps A, B and C are executed continuously and synchronously at a third preset frequency until no more alarm information is generated.

[0087] In this embodiment, AI processing is used to assist in alarm detection, which can more comprehensively detect abnormal situations in the monitored area and prevent missed detections that could cause property damage or other adverse consequences to users.

[0088] Based on the design concept of parallel processing of different processing nodes for different frames of images, according to another aspect of the embodiments of this application, an image processing method based on an AOV camera is also provided, for details please refer to Figure 5 The figure shows the flow of this image processing method. As shown in the figure, the image processing method includes the following steps: Step 310: The camera receives the first wake-up signal; Step 330: The camera enters the second state from the first state according to the first wake-up signal, wherein the camera power in the second state is not lower than the camera power in the first state.

[0089] After entering the second state, execute the following steps A, B, C1 and C2 simultaneously once.

[0090] Step A: Acquire the original nth frame image, where n≥4 and n is an integer; Step B: Perform post-processing on the original (n-1)th frame image to obtain the processed (n-1)th frame image; Step C1: Encode the processed (n-2)th frame image to obtain the encoded (n-2)th frame image; Step C2: Perform AI processing on the encoded (n-3)th frame image to obtain the final (n-3)th frame image and store it in the video buffer; Step 350: The camera returns to the first state.

[0091] for Figure 3 In the illustrated embodiment, for the case where step C takes longer than steps A and B, the total time for simultaneously executing steps A, B, and C is determined by the time taken for step C. To further shorten the total time for simultaneously executing steps A, B, and C, and to significantly reduce the duration of the main control chip in the second state, the image processing method provided in this embodiment is relatively... Figure 3 In the embodiment shown, step C is split into step C1, which is the encoding process of the previous frame image, and step C2, which is the AI ​​process of the next frame image. Since the total time for executing steps A, B, C1, and C2 simultaneously is less than the total time for executing steps A, B, and C simultaneously, the duration of power consumption spikes can be further shortened, thereby reducing the total power consumption of the system.

[0092] Similarly, for Figure 3 In the illustrated embodiment, for cases where step B takes longer than steps A and C, this application also provides an image processing method based on an AOV camera. Please refer to the following for details. Figure 6 The figure illustrates the flow of this image processing method, which includes the following steps: Step 410: The camera receives the first wake-up signal; Step 430: The camera enters the second state from the first state according to the first wake-up signal, wherein the camera power in the second state is not lower than the camera power in the first state.

[0093] After entering the second state, execute the following steps A, B1, B2 and C simultaneously once.

[0094] Step A: Acquire the original nth frame image, where n≥4 and n is an integer; Step B1: Perform effects processing on the original (n-1)th frame image to obtain the (n-1)th frame image after effects processing; Step B2: Scale the (n-2)th frame image after effect processing to obtain the scaled-down (n-2)th frame image; Step C: Encode and perform AI processing on the reduced (n-3)th frame image to obtain the final (n-3)th frame image and store it in the video buffer; Step 450: The camera re-enters the first state. The image processing method provided in this application embodiment is relative to... Figure 3 In the embodiment shown, the post-processing of step B is split into step B1, which is the effect processing of the previous frame image, and step B2, which is the scaling processing of the next frame image. The time taken to execute step A, step B1, step B2 and step C simultaneously is shorter than that taken to execute step A, step B and step C simultaneously, so the duration of the second state can be reduced.

[0095] To significantly shorten the duration of the system in the second state and minimize system power consumption, step B can be split into synchronously executed steps B1 and B2, and step C can be split into synchronously executed steps C1 and C2. Please refer to [link to relevant documentation] for details. Figure 7 The flowchart of another embodiment of the image processing method based on AOV camera provided shows the image processing method including: Step 510: The camera receives the first wake-up signal.

[0096] Step 530: The camera enters the second state from the first state according to the first wake-up signal, wherein the camera power in the second state is not lower than the camera power in the first state.

[0097] In the second state, the following steps A, B1, B2, C1, and C2 are executed synchronously once.

[0098] Step A: Acquire the original nth frame image, where n≥5 and n is an integer; Step B1: Perform effects processing on the original (n-1)th frame image to obtain the (n-1)th frame image after effects processing; Step B2: Scale the (n-2)th frame image after effect processing to obtain the scaled-down (n-2)th frame image; Step C1: Encode the reduced (n-3)th frame image to obtain the encoded (n-3)th frame image; Step C2: Perform AI processing on the encoded (n-4)th frame image to obtain the final (n-4)th frame image and store it in the video buffer; Step 550: The camera returns to the first state.

[0099] This embodiment maximizes the reduction of the time required to record one frame of image in the second state by simultaneously splitting step B into step B1 and step B2, and step C into step C1 and step C2, thereby significantly reducing system power consumption.

[0100] According to another aspect of the embodiments of this application, a main controller is provided, such as a main control chip, which stores a computer program and is used to execute the computer program to implement the image processing method based on an AOV camera provided in any of the above embodiments.

[0101] According to another aspect of the embodiments of this application, an AOV camera is also provided, please refer to [link / reference needed]. Figure 8 The figure shows the modular structure of the AOV camera. As shown in the figure, the AOV camera 600 includes a sensor 610, an auxiliary controller 620, and a main controller 630 in the above embodiment. The main controller 630 is electrically connected to the sensor 610 and the auxiliary controller 620, respectively.

[0102] When the main controller 630 is in the first state, the auxiliary controller 620 sends a first wake-up signal to the main controller 630 to wake it up and enter the second state. In the second state, the main controller 630 controls the exposure of the sensor 610 for video frame acquisition.

[0103] like Figure 8 As shown, in some embodiments, the AOV camera 600 further includes a PIR sensor 640, which is electrically connected to the auxiliary controller 620. The PIR sensor 640 is used to send an alarm signal to the auxiliary controller 620 when a target is detected. After receiving the alarm signal, the auxiliary controller 620 is used to send a second wake-up signal to the main controller 630, and the main controller 630 enters a second state after receiving the second wake-up signal.

[0104] Furthermore, such as Figure 8 As shown, the auxiliary controller 620 includes a network chip 621, which is electrically connected to the main controller 630. The network chip 621 is also used to send a second wake-up signal to the main controller 630 when there is a network connection, so as to wake up the main controller 630 and enter the second state.

[0105] To further reduce the power consumption of the network chip 621, thereby significantly reducing the overall system power consumption, such as Figure 7 As shown, the auxiliary controller 620 may further include a microcontroller 622, which is electrically connected to the main controller 630. The microcontroller 622 is used to send a first wake-up signal to the main controller 630. The network chip 621 is used to be in a first state when there is no network connection, so as to send a second wake-up signal to the main controller 630.

[0106] This application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements any of the above-described image processing method embodiments.

[0107] This application provides a computer program that can be executed by a processor to implement any of the above-described image processing method embodiments.

[0108] This application provides a computer program product, which includes a computer program that, when executed by a processor, implements any of the above-described image processing method embodiments.

[0109] In the several embodiments provided in this application, any function, if implemented as a software functional module / unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, all or part of the technical solution of this application can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or other electronic device) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing computer program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0110] The algorithms or displays provided herein are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used in conjunction with the teachings herein. The required structure for constructing such systems is apparent from the above description. Furthermore, the embodiments of this application are not directed to any particular programming language. It should be understood that the content of this application described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing the best mode of implementation of this application.

[0111] It should be noted that the above embodiments are illustrative of this application and not restrictive, and those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. This application can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In claims enumerating several means, several units or modules of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be construed as limiting the order of execution.

[0112] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An image processing method based on an AOV camera, characterized in that, include: The camera receives the first wake-up signal; The camera enters a second state from a first state according to the first wake-up signal, wherein the camera power in the second state is not lower than the camera power in the first state; In the second state, the camera synchronously performs the following steps once: Step A: Acquire the original nth frame image, where n≥2 and n is an integer; Step B: Perform post-processing on the original (n-1)th frame image to obtain the processed (n-1)th frame image; The camera then re-enters the first state.

2. The image processing method according to claim 1, characterized in that, In the second state, while performing steps A and B, the camera also simultaneously performs step C: encoding and AI processing on the processed (n-2)th frame image to obtain the final (n-2)th frame image and storing it in the video buffer, where n≥3.

3. The image processing method according to claim 2, characterized in that, The image processing method further includes: When the second wake-up signal is received, the camera enters the second state to synchronously execute steps A, B, and C multiple times at a preset frequency; The camera then re-enters the first state.

4. The image processing method according to claim 3, characterized in that, The second wake-up signal is generated when the passive infrared PIR sensor alarms; When the second wake-up signal is received, the camera enters the second state to synchronously execute steps A, B, and C multiple times at a preset frequency, including: When the second wake-up signal is received, the camera enters the second state and continuously and synchronously executes steps A, B and C at a first preset frequency until the PIR sensor alarm ends.

5. The image processing method according to claim 4, characterized in that, When a second wake-up signal is received, the camera enters the second state and continuously and synchronously executes steps A, B, and C at a first preset frequency until the PIR sensor alarm ends, including: When the second wake-up signal is received, the camera enters the second state; Read the data from the PIR sensor to confirm whether actual motion has occurred; If so, then steps A, B, and C are executed continuously and synchronously at the first preset frequency until the PIR sensor alarm ends; If not, then proceed with the step of having the camera re-enter the first state.

6. The image processing method according to claim 3, characterized in that, The second wake-up signal is generated when there is a network connection; When the second wake-up signal is received, the camera enters the second state to synchronously execute steps A, B, and C multiple times at a preset frequency, including: When the second wake-up signal is received, the camera enters the second state and continuously and synchronously executes steps A, B and C at a second preset frequency until the network connection ends.

7. The image processing method according to claim 6, characterized in that, The image processing method further includes: When a network connection is available, in response to a request from the network requesting end, the video stream data stored in the video buffer is uploaded to the network requesting end in real time.

8. The image processing method according to any one of claims 2-7, characterized in that, Step C includes: AI processing is performed on the encoded (n-2)th frame image to detect whether there is a target object in the encoded (n-2)th frame image, and the final (n-2)th frame image obtained after AI processing is stored in the video buffer. If a target object is detected in the encoded (n-2)th frame image, an alarm message is generated; The image processing method further includes: In response to the alarm information, the camera synchronously executes steps A, B, and C multiple times at a third preset frequency; The camera then re-enters the first state.

9. The image processing method according to claim 8, characterized in that, In response to the alarm information, the camera synchronously executes steps A, B, and C multiple times at a third preset frequency, including: In response to the alarm information, the camera continuously and synchronously executes steps A, B, and C at a third preset frequency until no more alarm information is generated.

10. The image processing method according to any one of claims 1-7, characterized in that, Step B includes: The original (n-1)th frame image is processed by effects and scaling to obtain the processed (n-1)th frame image.

11. An image processing method based on an AOV camera, characterized in that, include: The camera receives the first wake-up signal; The camera enters a second state from a first state according to the first wake-up signal, wherein the camera power in the second state is not lower than the camera power in the first state; In the second state, the camera synchronously performs the following steps once: Step A: Acquire the original nth frame image, where n≥4 and n is an integer; Step B: Perform post-processing on the original (n-1)th frame image to obtain the processed (n-1)th frame image; Step C1: Encode the processed (n-2)th frame image to obtain the encoded (n-2)th frame image; Step C2: Perform AI processing on the encoded (n-3)th frame image to obtain the final (n-3)th frame image and store it in the video buffer; The camera then re-enters the first state.

12. An image processing method based on an AOV camera, characterized in that, include: The camera receives the first wake-up signal; The camera enters a second state from a first state according to the first wake-up signal, wherein the camera power in the second state is not lower than the camera power in the first state; In the second state, the camera synchronously performs the following steps once: Step A: Acquire the original nth frame image, where n≥4 and n is an integer; Step B1: Perform effects processing on the original (n-1)th frame image to obtain the (n-1)th frame image after effects processing; Step B2: Scale the (n-2)th frame image after effect processing to obtain the scaled-down (n-2)th frame image; Step C: Encode and perform AI processing on the reduced (n-3)th frame image to obtain the final (n-3)th frame image and store it in the video buffer; The camera then re-enters the first state.

13. An image processing method based on an AOV camera, characterized in that, include: The camera receives the first wake-up signal; The camera enters a second state from a first state according to the first wake-up signal, wherein the camera power in the second state is not lower than the camera power in the first state; In the second state, the camera synchronously performs the following steps once: Step A: Acquire the original nth frame image, where n≥5 and n is an integer; Step B1: Perform effects processing on the original (n-1)th frame image to obtain the (n-1)th frame image after effects processing; Step B2: Scale the (n-2)th frame image after effect processing to obtain the scaled-down (n-2)th frame image; Step C1: Encode the reduced (n-3)th frame image to obtain the encoded (n-3)th frame image; Step C2: Perform AI processing on the encoded (n-4)th frame image to obtain the final (n-4)th frame image and store it in the video buffer; The camera then re-enters the first state.

14. A main controller, characterized in that, The main controller stores a computer program, which is used to execute the computer program to implement the image processing method based on an AOV camera as described in any one of claims 1-13.

15. An AOV camera, characterized in that, include: The sensor, the auxiliary controller, and the main controller as described in claim 14, wherein the main controller is electrically connected to the sensor and the auxiliary controller, respectively. When the main controller is in the first state, the auxiliary controller is used to send the first wake-up signal to the main controller to wake up the main controller and enter the second state; The main controller is used to control the exposure of the sensor in the second state.

16. The AOV camera according to claim 15, characterized in that, The AOV camera also includes a PIR sensor, which is electrically connected to the auxiliary controller. The PIR sensor is used to send an alarm signal to the auxiliary controller when a target object is detected; The auxiliary controller is used to send a second wake-up signal to the main controller after receiving the alarm signal; The main controller is used to enter the second state after receiving the second wake-up signal.

17. The AOV camera according to claim 15, characterized in that, The auxiliary controller includes a network chip, which is electrically connected to the main controller. The network chip is also used to send a second wake-up signal to the main controller when there is a network connection, so as to wake up the main controller and enter the second state.

18. The AOV camera according to claim 17, characterized in that, The auxiliary controller further includes a microcontroller, which is electrically connected to the main controller. The microcontroller is used to send the first wake-up signal to the main controller; When there is no network connection, the network chip is in the first state, and when there is a network connection, it exits the first state to send the second wake-up signal to the main controller.

Images