Image processing method, ai-mcu module and camera device

Abstract

This disclosure provides an image processing method, an AI-MCU module, and a camera device. The image processing method includes: waking up from sleep mode in response to a first wake-up command from an infrared monitoring module; acquiring a first image through an image acquisition module; performing target region detection on the first image, and cropping the first image to obtain a target image block if a target region is detected; performing target recognition on the target image block to obtain a target recognition result; and, if the target recognition result indicates that the potential target is a preset target, sending a second wake-up command to the SoC module to wake it up from sleep mode and acquire a second image or video stream through the image acquisition module. This disclosure can effectively reduce the false alarm / missed alarm rate and avoid the power waste caused by frequent wake-ups of high-power SoC modules due to false alarms, significantly reducing the overall power consumption of the system.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and in particular to an image processing method, an AI-MCU module, and a camera device. Background Technology

[0002] Currently, home surveillance cameras are widely used in scenarios such as home security, elderly care, and pet monitoring.

[0003] Home surveillance cameras rely on PIR (Passive Infrared Sensor) / Radar and a main control SoC (System on Chip) for image processing. When the PIR / Radar detects movement, it directly wakes up the dormant main control SoC to capture and analyze video. However, this image processing technology suffers from high false alarm / missing alarm rates, and false alarms can cause the main control SoC to wake up frequently, resulting in high power consumption. Summary of the Invention

[0004] This disclosure provides an image processing method, an AI-MCU module, and a camera device.

[0005] According to a first aspect of the present disclosure, an image processing method is provided, the image processing method being applied to an AI-MCU module, the AI-MCU module being communicatively connected to an infrared monitoring module, an image acquisition module, and a SoC module, the image processing method comprising: In response to a first wake-up command from the infrared monitoring module, the device wakes up from sleep mode. The first image is acquired through the image acquisition module; Target region detection is performed on the first image. If a target region is detected, the first image is cropped to obtain a target image block. The target image block contains the target region, which is a region containing a potential target. Target recognition is performed on the target image block to obtain the target recognition result; If the target recognition result indicates that the potential target is a preset target, a second wake-up command is sent to the SoC module to wake up the SoC module from hibernation and acquire a second image or video stream through the image acquisition module.

[0006] In some embodiments, performing target region detection on the first image and cropping the first image to obtain a target image patch when a target region is detected includes: The target region is detected in the first image based on a preset ROI detection algorithm; If the target region is detected, the ROI coordinates corresponding to the target region in the first image are determined; Based on the ROI coordinates, the corresponding target image patch is cropped from the first image.

[0007] In some embodiments, the target region detection of the first image based on a preset ROI detection algorithm includes: The first image is downsampled and converted to grayscale according to a preset downsampling ratio to generate a first grayscale image; A pixel-level difference operation is performed between the first grayscale image and the second grayscale image to obtain a difference image; wherein, the second grayscale image is generated by downsampling and grayscale converting the background frame image according to the downsampling ratio; An adaptive threshold for the difference image is calculated using a preset adaptive threshold segmentation algorithm, and the difference image is binarized based on the calculation result to obtain a binary image. When the binary image representation contains a moving foreground, a morphological opening operation is performed on the binary image to remove isolated noise points and form connected regions, resulting in a processed binary image. If a connected region that meets the preset requirements exists in the processed binary image, the target region is determined to be detected, and the region defined by the outer rectangle of the connected region that meets the preset requirements is taken as the target region.

[0008] In some embodiments, the target region detection of the first image further includes: If the target area is not detected, resume hibernation.

[0009] In some embodiments, if the binary image representation does not contain a moving foreground, it is determined that the target region has not been detected; or If no connected region meeting the preset requirements exists in the processed binary image, it is determined that the target region has not been detected.

[0010] In some embodiments, the target region detection of the first image based on a preset ROI detection algorithm further includes: If the binary image representation corresponding to the first image acquired a preset number of times does not contain a moving foreground, the background frame image is updated to the most recently acquired first image.

[0011] In some embodiments, determining the ROI coordinates corresponding to the target region in the first image includes: Based on the downsampling ratio, the coordinates corresponding to the target region in the first grayscale image are mapped to the first image to obtain the ROI coordinates.

[0012] In some embodiments, the step of performing target recognition on the target image patch to obtain a target recognition result includes: The target image block is scaled to a preset size to obtain a scaled image block; Based on the scaled image patch, the trained AI model outputs the category recognition result; If the category identification result includes a preset target type, the target identification result is determined to characterize the potential target as the preset target.

[0013] In some embodiments, after performing target recognition on the target image patch and obtaining the target recognition result, the image processing method further includes: If the target identification result indicates that the potential target is not the preset target, the hibernation is resumed.

[0014] According to a second aspect of the present disclosure, an AI-MCU module is provided, the AI-MCU module comprising: One or more processors; A memory that stores one or more programs, which, when executed by one or more processors, enable the one or more processors to implement any of the above-mentioned image processing methods; One or more I / O interfaces are connected between the processor and memory and configured to enable information exchange between the processor and memory.

[0015] According to a third aspect of the present disclosure, a camera device is provided, the camera device comprising: an infrared monitoring module, an image acquisition module, an AI-MCU module, and a SoC module; the AI-MCU module is communicatively connected to the infrared monitoring module, the image acquisition module, and the SoC module respectively; The AI-MCU module is used to wake up from sleep after receiving a first wake-up command sent by the infrared monitoring module, acquire a first image through the image acquisition module, and perform target region detection on the first image. If a target region is detected, the first image is cropped to obtain a target image block. The target image block is then used for target recognition to obtain a target recognition result. The target image block contains the target region, which is a region containing a potential target. If the target recognition result indicates that the potential target is a preset target, a second wake-up command is sent to the SoC module. The SoC module is used to wake up from hibernation after receiving the second wake-up command, and to acquire a second image or video stream through the image acquisition module.

[0016] In some embodiments, the AI-MCU module and the SoC module are respectively connected to a multiplexer via an integrated circuit bus, and the multiplexer is also connected to the image acquisition module via an integrated circuit bus; The AI-MCU module is also used to, after waking up from hibernation, switch the master device on the integrated circuit bus to the AI-MCU module through the multiplexer, and send a first acquisition command to the image acquisition module; the image acquisition module is used to, after receiving the first acquisition command, acquire a first image corresponding to the first acquisition command, and send the first image to the AI-MCU module; The SoC module is also used to, after waking up from hibernation, switch the master device on the integrated circuit bus to the SoC module through the multiplexer, and send a second acquisition command to the image acquisition module; the image acquisition module is used to, after receiving the second acquisition command, acquire a second image or video stream corresponding to the second acquisition command, and send the second image or video stream to the SoC module.

[0017] The image processing method provided in this disclosure introduces a low-power AI-MCU module between the infrared monitoring module and the high-power SoC module. The infrared monitoring module first wakes up the AI-MCU module, and the SoC module is only woken up when the AI-MCU module detects a preset target. This achieves secondary filtering of the wake-up command of the infrared monitoring module by the AI-MCU module, effectively solving the problem of high false alarm / missing alarm rate caused by the infrared monitoring module being susceptible to interference from temperature changes, airflow, small animals, etc. At the same time, it avoids the power waste caused by frequent wake-up of the high-power SoC module due to false alarms, significantly reduces the overall power consumption of the system, extends the battery life of the battery-powered device, and improves the target detection accuracy. Attached Figure Description

[0018] Figure 1 This is a schematic flowchart of the image processing method provided in the embodiments of this disclosure; Figure 2 This is a structural block diagram of the AI-MCU module provided in the embodiments of this disclosure; Figure 3 A structural block diagram of a camera device provided in an embodiment of this disclosure; Figure 4 A structural block diagram of another camera device provided in this disclosure embodiment; Figure 5 A schematic diagram illustrating the working principle of another camera device provided in this embodiment of the present disclosure; Figure 6 A flowchart illustrating the operation of another camera device provided in this disclosure embodiment; Figure 7 This is a flowchart of the ROI detection algorithm provided in the embodiments of this disclosure; Figure 8 This is a schematic diagram showing the comparison before and after ROI clipping and scaling provided in an embodiment of this disclosure. Detailed Implementation

[0019] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions of this disclosure will be described in detail below with reference to the accompanying drawings.

[0020] Exemplary embodiments will be described more fully below with reference to the accompanying drawings; however, these exemplary embodiments may be embodied in different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will enable those skilled in the art to fully understand the scope of this disclosure.

[0021] Where there is no conflict, the various embodiments of this disclosure and the features thereof in the embodiments may be combined with each other.

[0022] As used herein, the term “and / or” includes any and all combinations of one or more related enumerated entries.

[0023] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. As used herein, the singular forms “a” and “the” are also intended to include the plural forms unless the context clearly indicates otherwise. It will also be understood that when the terms “comprising” and / or “made of” are used in this specification, the presence of the stated feature, integral, step, operation, element, and / or component is specified, but the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof is not excluded.

[0024] Unless otherwise specified, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and this disclosure, and will not be interpreted as having an idealized or overly formal meaning, unless expressly so defined herein.

[0025] In related technologies, to balance power consumption and performance, home surveillance cameras typically employ a "PIR + main control SoC" architecture, or a combination of PIR and Radar for moving object detection. In this architecture, when the PIR / Radar detects movement, it wakes up the dormant main control SoC to capture and analyze video. However, this approach has the following drawbacks: 1) High false alarm / missed alarm rate: PIR is easily affected by temperature changes, airflow, small animals, etc., causing false alarms. Radar has difficulty distinguishing between humans and pets, and relying solely on signal strength / distance filtering is prone to false alarms or missed alarms, and is also easily affected by metal reflection interference.

[0026] 2) Balancing power consumption and performance is difficult: PIR false alarms cause the main control SoC to wake up frequently, resulting in a significant reduction in battery life. However, if the SoC's sleep time after PIR triggering is extended, it is easy to miss the alarm. Radar itself also has the problem of high power consumption.

[0027] Therefore, to overcome the aforementioned shortcomings, this disclosure provides an image processing method based on a "PIR+AI-MCU+SoC" target recognition architecture. When the PIR reaches a trigger threshold, the AI-MCU is first activated to perform target recognition. Only when the AI-MCU recognizes a preset target (such as a family member or pet requiring care) is the SoC activated for video acquisition and analysis. This method, through a two-level wake-up mechanism, achieves low false alarm / missing rate and balances power consumption and performance, effectively addressing the deficiencies of related technologies.

[0028] Figure 1 This is a schematic flowchart of an image processing method provided in an embodiment of the present disclosure.

[0029] Firstly, referring to Figure 1 This disclosure provides an image processing method. The image processing method is applied to an AI-MCU module, which is communicatively connected to an infrared monitoring module, an image acquisition module, and a SoC module. AI (Artificial Intelligence) and MCU (Microcontroller Unit) are combined to form an AI-MCU, a novel embedded processor that integrates artificial intelligence reasoning capabilities into a traditional microcontroller. It integrates an AI model, enabling edge devices (such as home surveillance cameras) to achieve near-SoC-level intelligent processing capabilities with MCU-level power consumption.

[0030] like Figure 1 As shown, in this embodiment, the image processing method includes the following steps S1 to S5.

[0031] Step S1: In response to the first wake-up command from the infrared monitoring module, wake up from hibernation.

[0032] In this step, the first wake-up command is sent by the infrared monitoring module to the AI-MCU module when it detects that the change in infrared radiation flux in the environment exceeds a preset trigger threshold.

[0033] Step S2: Acquire the first image through the image acquisition module.

[0034] In this step, the resolution of the first image is greater than or equal to a preset first resolution threshold to ensure that the first image is a high-resolution image. This provides finer texture and contour information for target region detection and target recognition, which is beneficial for improving detection distance and accuracy and preventing false alarms. In some embodiments, the resolution of the first image is greater than or equal to the preset first resolution threshold and less than a preset second resolution threshold. This ensures the resolution of the first image while also saving on transmission and processing overhead, thus improving efficiency. In one example, the resolution of the first image is 720p or 1080p.

[0035] Step S3: Detect a target region in the first image. If a target region is detected, crop the first image to obtain a target image block. The target image block contains the target region, which is a region containing a potential target.

[0036] In this step, the target region can be detected using the ROI (Region of Interest) detection algorithm, and the target image patch containing the potential target can be cropped from the first image.

[0037] Step S4: Perform target recognition on the target image block to obtain the target recognition result.

[0038] In this step, an AI model can be used to identify the target, and the target identification result output by the AI ​​model can characterize whether the potential target is a preset target (such as a family member or pet that needs care).

[0039] Step S5: If the target recognition result indicates that the potential target is a preset target, a second wake-up command is sent to the SoC module to wake up the SoC module from hibernation and acquire a second image or video stream through the image acquisition module.

[0040] In this step, the video stream consists of multiple consecutive second images, each with a resolution greater than or equal to a second resolution threshold. In one example, the resolution of the second image is 4k. It is evident that the resolution of the second image is higher than that of the first image. Furthermore, the second image can be referred to as a high-resolution image, while the first image can be described as a relatively low-resolution image.

[0041] In this embodiment, a low-power AI-MCU module is introduced between the infrared monitoring module and the high-power SoC module. The infrared monitoring module first wakes up the AI-MCU module, and the SoC module is only woken up when the AI-MCU module detects a preset target. This achieves secondary filtering of the wake-up command of the infrared monitoring module by the AI-MCU module, effectively solving the problem of high false alarm / missing alarm rate caused by the infrared monitoring module being susceptible to interference from temperature changes, airflow, small animals, etc. At the same time, it avoids the power waste caused by frequent wake-up of the high-power SoC module due to false alarms, significantly reduces the overall power consumption of the system, extends the battery life of the battery-powered device, and improves the target detection accuracy.

[0042] In practical applications, due to resource limitations (such as computing power and memory constraints) of the AI-MCU module, the integrated AI model can only process low-resolution images. In related technologies, the AI-MCU module acquires low-resolution images (e.g., 360p) and scales them to the size required by the AI ​​model (e.g., 64×64 or 96×96). If this processing method is used in the embodiments of this disclosure, the number of pixels in the image input to the AI ​​model will be relatively small, resulting in a short detection distance. It can only identify nearby targets, typically with a maximum detection distance of only 4-5 meters for humans. It will be even more difficult to distinguish small targets such as pets, easily misidentifying pets as humans or missing either people or pets.

[0043] To improve the AI-MCU module's ability to recognize medium- and long-range targets and small targets while maintaining low power consumption, this disclosure addresses the problems of limited detection distance and difficulty in recognizing small targets such as pets in low-resolution images. It proposes a scheme that first performs ROI cropping and then uses an AI model for target recognition. Specific embodiments are described in detail below.

[0044] In some embodiments, step S3 specifically includes steps S31 to S33.

[0045] Step S31: Detect the target region in the first image based on a preset ROI detection algorithm; Step S32: If the target region is detected, determine the ROI coordinates corresponding to the target region in the first image.

[0046] Step S33: Based on the ROI coordinates, crop out the corresponding target image block from the first image.

[0047] In this embodiment, a target region is detected in the first image based on a preset ROI detection algorithm. After detecting the target region, the ROI coordinates are determined, and then a target image patch is cropped from the first image based on these ROI coordinates. Since the cropped target image patch has the same resolution as the first image, but the proportion of pixels occupied by the potential target in the target image patch is significantly increased compared to the proportion of pixels occupied in the first image, it is equivalent to magnifying the distant target region. This cropping process is completed within the AI-MCU module, with a computational load far less than that of SoC inference. Thus, while maintaining low power consumption, the sensitivity of the AI-MCU module to medium- and long-distance targets is significantly improved, effectively increasing the detection distance. Moreover, the image size input to the AI ​​model after cropping is controllable and does not significantly increase the burden on the MCU. At the same time, focusing on the target activity area using ROI coordinates enhances feature fidelity, providing more refined texture and contour information for subsequent target recognition by the AI ​​model. This effectively distinguishes between people and pets, significantly reducing the false alarm rate for small targets such as pets, with an overall false alarm rate reduction of over 90%.

[0048] In some embodiments, step S31 specifically includes steps S311 to S315.

[0049] Step S311: Downsample the first image and convert it to grayscale according to a preset downsampling ratio to generate a first grayscale image.

[0050] Step S312: Perform pixel-level difference operation on the first grayscale image and the second grayscale image to obtain a difference image; wherein, the second grayscale image is generated by downsampling and grayscale converting the background frame image according to the downsampling ratio.

[0051] Step S313: The adaptive threshold of the difference image is calculated using a preset adaptive threshold segmentation algorithm, and the difference image is binarized according to the calculation result to obtain a binary image.

[0052] Step S314: When the binary image representation has a moving foreground, perform a morphological opening operation on the binary image to remove isolated noise points and form connected regions to obtain the processed binary image.

[0053] Step S315: If a connected region that meets the preset requirements exists in the processed binary image, the target region is determined to be detected, and the region defined by the outer rectangle of the connected region that meets the preset requirements is taken as the target region.

[0054] In this embodiment, a first grayscale image is generated by downsampling and grayscale conversion of the first image, and a difference image is obtained by pixel-level difference operation with the grayscale image of the background frame. This significantly reduces the amount of data for subsequent processing and reduces the computational burden on the AI-MCU module. An adaptive threshold segmentation algorithm is used to binarize the difference image, which automatically adapts to different lighting conditions without requiring manual threshold setting. When the binary image represents a moving foreground, morphological opening operations are performed on the binary image, effectively removing isolated noise points and forming connected regions, thereby eliminating interference caused by leaf swaying, light changes, etc. Finally, only when a connected region that meets preset requirements (such as area and aspect ratio) exists in the processed binary image is the region defined by the bounding rectangle of that connected region considered as the target region, further improving the accuracy of target region detection and reducing unnecessary computation and power consumption.

[0055] Moreover, unlike the processing method in related technologies that involves acquiring low-resolution images (such as 360p) and scaling them before inputting them into the AI ​​model, the embodiments of this disclosure acquire higher-resolution images (such as 720p or 1080p) at the AI-MCU end. Regions containing potential targets are located using ROI detection algorithms, then cropped. The cropped target image patches are then appropriately scaled and fed into the AI ​​model for target recognition. This achieves high-resolution image acquisition and low-resolution image processing, increasing both the detection distance and the target recognition accuracy. It should be noted that, compared to the low-resolution images acquired in related technologies, the image acquired by the AI-MCU module (i.e., the first image) can be referred to as a higher-resolution image.

[0056] In some embodiments, step S3 further includes step S34.

[0057] Step S34: If the target area is not detected, resume hibernation.

[0058] In this embodiment, when no target area is detected, the AI-MCU module directly resumes sleep mode and no longer performs subsequent target recognition and SoC wake-up operations. This mechanism avoids invalid wake-ups caused by false alarms from the infrared monitoring module (such as temperature changes, airflow interference, small animals passing by, etc.), thereby reducing the working time of the AI-MCU module and the number of times the SoC module is woken up, significantly reducing the average power consumption of the system and extending battery life.

[0059] In some embodiments, after step S313, step S316 is also performed.

[0060] Step S316: If the binary image representation does not contain a moving foreground, it is determined that the target region has not been detected.

[0061] In this embodiment, after obtaining the binary image, if the binary image representation does not contain a moving foreground, it is directly determined that no target region has been detected, without the need for subsequent morphological opening operations and connected component analysis. This early determination mechanism achieves rapid filtering during the moving foreground extraction stage, further reducing the computational overhead and power consumption of the AI-MCU module. At the same time, it effectively eliminates invalid processing flows caused by false alarms from the infrared monitoring module, improving the system's response efficiency and energy efficiency ratio.

[0062] In some embodiments, after step S314, step S317 is also performed.

[0063] Step S317: If there is no connected region that meets the preset requirements in the processed binary image, it is determined that the target region has not been detected.

[0064] In this embodiment, after obtaining the processed binary image through morphological opening operation, if there is no connected region that meets the preset requirements (such as too small an area or an unreasonable aspect ratio), it is determined that no target region has been detected. This effectively eliminates interference regions caused by leaf swaying, light changes, etc., avoiding subsequent ROI coordinate mapping and target recognition for these pseudo-motion regions. This reduces the invalid calculation of the AI-MCU module, further reduces the false alarm rate, and improves the system power consumption efficiency.

[0065] In some embodiments, step S31 further includes step S318.

[0066] Step S318: If the binary image representation corresponding to the first image acquired a preset number of times does not contain a moving foreground, update the background frame image to the most recently acquired first image.

[0067] In this embodiment, the dynamic background update strategy can effectively cope with scenarios such as slow changes in ambient light (e.g., from day to dusk) or temperature drift, avoiding large-scale false alarms caused by the accumulation of differences between the background and the current scene. This ensures the long-term stability and accuracy of moving foreground extraction, while eliminating the need for manual intervention and improving the robustness and environmental adaptability of the system.

[0068] In some embodiments, step S32 specifically includes: mapping the coordinates corresponding to the target region in the first grayscale image to the first image according to the downsampling ratio to obtain the ROI coordinates.

[0069] In this embodiment, the geometric correspondence between the target region coordinates detected on the low-resolution first grayscale image and the high-resolution first image is utilized to achieve a precise conversion from low-resolution motion detection to high-resolution ROI localization with minimal computational cost. This ensures that the cropped target image block can completely contain the potential target, while avoiding the high power consumption caused by performing motion detection directly on the high-resolution image, thus balancing detection accuracy and system power consumption.

[0070] In some embodiments, step S4 specifically includes steps S41 to S43.

[0071] Step S41: Scale the target image block to a preset size to obtain the scaled image block.

[0072] Step S42: Based on the scaled image patch, output the category recognition result through the trained AI model.

[0073] Step S43: If the category identification result includes a preset target type, determine that the target identification result characterizes the potential target as the preset target.

[0074] In this embodiment, the target image patch is scaled to a preset size and then input into a trained AI model for category recognition. Since the target image patch is cropped from a high-resolution first image, the proportion of potential targets in the scaled image patch is still significantly increased compared to their pixel proportion in the first image, thus providing the AI ​​model with more refined texture and contour information. Compared to directly acquiring and scaling low-resolution images, this method effectively improves the recognition rate of targets at medium and long distances, especially the ability to distinguish small targets such as pets, significantly reduces the false alarm rate, and maintains the low-power characteristics of the AI-MCU module.

[0075] In some embodiments, after step S4, the image processing method further includes step S6.

[0076] Step S6: If the target identification result indicates that the potential target is not the preset target, resume sleep mode.

[0077] In this embodiment, when the target recognition result indicates that the potential target is not a preset target (such as swaying leaves, small animals, or non-interested objects), the AI-MCU module directly resumes sleep mode without waking up the SoC module. Through secondary confirmation by the AI ​​model, false alarms from the infrared monitoring module and interference from non-preset targets are effectively filtered out, preventing the SoC module from being unnecessarily woken up. This significantly extends the battery life of the battery-powered device while ensuring that high-power recording and streaming operations are only triggered when a real preset target appears, achieving an optimal balance between system power consumption and security response.

[0078] In some embodiments, after the SoC module acquires a second image or video stream through the image acquisition module, it can use its powerful computing capabilities to perform in-depth analysis on the acquired image or video stream.

[0079] For example, the SoC module can efficiently compress and encode video streams using hardware encoders, such as H.264 or H.265, to reduce storage space and network bandwidth requirements. Simultaneously, the SoC module can utilize its built-in neural network processing unit to perform real-time intelligent analysis of image or video content, accurately identifying various targets such as people and pets in the scene. Based on these recognition results, the SoC module can also perform more advanced visual tasks, such as analyzing abnormal behaviors like intrusion, loitering, and departure, enabling fall detection for the elderly and child crying in home care scenarios, as well as tracking specific behaviors of pets or children.

[0080] Furthermore, the edge computing capabilities of the SoC module enable all video analysis and processing to be completed locally, eliminating the need to upload raw data to the cloud. This significantly reduces network latency and effectively protects user data privacy and security. While performing real-time analysis, the SoC module can also trigger local alarms based on preset alarm policies, or push alarm information and key video clips to user terminals via Wi-Fi or other methods, achieving instant response and remote notification. This meets the diverse application needs of scenarios such as home security, elderly care, and pet monitoring.

[0081] Secondly, referring to Figure 2 This disclosure provides an AI-MCU module, which includes: One or more processors 201; The memory 202 stores one or more programs, which, when executed by one or more processors, enable the one or more processors to implement any of the above-mentioned image processing methods. One or more I / O interfaces 203 are connected between the processor and the memory and configured to enable information exchange between the processor and the memory.

[0082] The processor 201 is a device with data processing capabilities, including but not limited to a central processing unit (CPU); the memory 202 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), and flash memory (FLASH); the I / O interface (read / write interface) 203 is connected between the processor 201 and the memory 202, enabling information exchange between the processor 201 and the memory 202, including but not limited to a data bus (Bus).

[0083] In some embodiments, the processor 201, memory 202, and I / O interface 203 are interconnected via bus 204, and thus connected to other components of the computing device.

[0084] Figure 3 This is a structural block diagram of a camera device provided in an embodiment of the present disclosure.

[0085] Thirdly, referring to Figure 3 This disclosure provides a camera device. For example... Figure 3 As shown, in this embodiment, the camera device includes: an infrared monitoring module 301, an image acquisition module 302, an AI-MCU module 303, and a SoC module 304. The AI-MCU module 303 is communicatively connected to the infrared monitoring module 301, the image acquisition module 302, and the SoC module 304, respectively.

[0086] The infrared monitoring module 301 is used to monitor changes in infrared radiation flux in the environment in real time, and when the changes in infrared radiation flux exceed a preset trigger threshold, it sends a first wake-up command to the AI-MCU module 303.

[0087] The AI-MCU module 303 is used to wake up from sleep mode after receiving a first wake-up command sent by the infrared monitoring module 301, acquire a first image through the image acquisition module 302, and perform target region detection on the first image. If a target region is detected, the first image is cropped to obtain a target image block, and target recognition is performed on the target image block to obtain a target recognition result. The target image block contains the target region, which is a region containing a potential target. If the target recognition result indicates that the potential target is a preset target, a second wake-up command is sent to the SoC module 304. In some embodiments, the resolution of the first image is greater than or equal to a preset first resolution threshold and less than a preset second resolution threshold.

[0088] The SoC module 304 is used to wake up from sleep mode after receiving the second wake-up command, and to acquire a second image or video stream through the image acquisition module 302. The video stream consists of multiple consecutive second images, and the resolution of the second images is greater than or equal to the second resolution threshold.

[0089] In this embodiment, a low-power AI-MCU module is introduced between the infrared monitoring module and the high-power SoC module. The infrared monitoring module first wakes up the AI-MCU module, and the SoC module is only woken up when the AI-MCU module detects a preset target. This achieves secondary filtering of the wake-up command of the infrared monitoring module by the AI-MCU module, effectively solving the problem of high false alarm / missing alarm rate caused by the infrared monitoring module being susceptible to interference from temperature changes, airflow, small animals, etc. At the same time, it avoids the power waste caused by frequent wake-up of the high-power SoC module due to false alarms, significantly reduces the overall power consumption of the system, extends the battery life of the battery-powered device, and improves the target detection accuracy.

[0090] In some embodiments, the AI-MCU module 303 and the SoC module 304 are respectively connected to a multiplexer via an integrated circuit bus, and the multiplexer is also connected to the image acquisition module 302 via an integrated circuit bus.

[0091] The AI-MCU module 303 is also used to switch the master device on the integrated circuit bus to the AI-MCU module 303 through the multiplexer after waking up from hibernation, and send a first acquisition command to the image acquisition module 302; the image acquisition module 302 is used to acquire a first image corresponding to the first acquisition command after receiving the first acquisition command, and send the first image to the AI-MCU module 303.

[0092] The SoC module 304 is also used to switch the master device on the integrated circuit bus to the SoC module 304 through the multiplexer after waking up from hibernation, and send a second acquisition command to the image acquisition module 302; the image acquisition module 302 is used to acquire a second image or video stream corresponding to the second acquisition command after receiving the second acquisition command, and send the second image or video stream to the SoC module 304.

[0093] In this embodiment, a control path that can be shared by two master devices in a time-sharing manner is constructed through an integrated circuit bus and a multiplexer. When the AI-MCU module wakes up from sleep mode, it switches itself as the master device on the integrated circuit bus through the multiplexer, thereby gaining control of the image acquisition module, sending a first acquisition command, and receiving a first image. When the SoC module is woken up, it similarly switches itself as the master device through the multiplexer, sending a second acquisition command and receiving a second image or video stream. In this way, the multiplexer, acting as a physical switching switch, effectively avoids integrated circuit bus conflicts caused by the AI-MCU module and the SoC module simultaneously attempting to control the image acquisition module, ensuring communication reliability. At the same time, it allows the two modules to obtain master device permissions in a time-sharing manner according to their own working state, without the need for additional bus arbitration logic, reducing the complexity and power consumption of the system design. This scheme realizes the safe and orderly sharing of the same image acquisition module between the low-power AI-MCU module and the high-power SoC module, providing a hardware foundation for a two-level wake-up mechanism and further optimizing the overall energy efficiency of the system.

[0094] To enable those skilled in the art to more clearly understand the technical solutions provided by the embodiments of this disclosure, the technical solutions provided by the embodiments of this disclosure will be described in detail below through specific embodiments: Figure 4 This is a structural block diagram of another camera device provided in an embodiment of the present disclosure. Figure 5 A schematic diagram illustrating the working principle of another camera device provided in an embodiment of this disclosure. (See diagram for example.) Figure 4 and Figure 5 As shown, in this embodiment, the camera device includes: a PIR module 401, a Camera module 402, an AI-MCU module 403, a SoC module 404, a Mux 405, and a WiFi module 406. The Mux is a multiplexer used to implement time-division multiplexing.

[0095] The PIR module 401 is communicatively connected to the AI-MCU module 403. The PIR module 401 monitors the changes in infrared radiation flux in the environment in real time. If the changes in infrared radiation flux in the environment exceed a preset trigger threshold, it will send a first wake-up command to the AI-MCU module 403 to wake up the AI-MCU module 403 from sleep mode.

[0096] In one example, the PIR module 401 can wake up the AI-MCU module 403 via a hardware interrupt mechanism.

[0097] Specifically, the PIR module has a digital output pin (OUT) connected to the GPIO pin of the AI-MCU module. When the PIR module detects a change in the infrared radiation flux in the environment that exceeds the trigger threshold, its OUT pin will change level. This level change is transmitted as a wake-up command to the GPIO pin of the AI-MCU module via the connection line. After detecting this wake-up command, the interrupt controller inside the AI-MCU module sends a wake-up signal to its CPU, waking the CPU from sleep mode. The entire process is completed in microseconds with extremely low power consumption.

[0098] The AI-MCU module 403 is also communicatively connected to the SoC module 404. After the AI-MCU module 403 wakes up from sleep mode, it acquires a first image from the Camera module 402 and performs target region detection on the first image. If a target region is detected, the first image is cropped to obtain a target image block, wherein the target image block contains the target region, which is a region containing a potential target. Then, target recognition is performed on the target image block to obtain a target recognition result. If the target recognition result indicates that the potential target is a preset target, a second wake-up command can be sent to the SoC module 404 via the INT line (Interrupt Line) to wake up the SoC module 404 from sleep mode. Specifically, when the AI-MCU module confirms the existence of a preset target, it sends a hardware interrupt signal (such as a level transition) via the INT line to wake up the SoC module, which is in deep sleep mode, and put it into working state.

[0099] The AI-MCU module 403 and SoC module 404 are connected to the Mux 405 via I2C (Integrated Circuit Bus), and the Mux 405 is also connected to the Camera module 402 via I2C. The Mux 405 coordinates the master control of the I2C port of the Camera module 402. Specifically, when the AI-MCU module 403 wakes up from sleep mode, it switches itself to the master device on the I2C via the Mux 405 and sends a first acquisition command to the Camera module 402 via I2C. The first acquisition command corresponds to a first image, thereby controlling the Camera module 402 to acquire the first image. When the SoC module 404 wakes up from sleep mode, it switches itself to the master device on the I2C via the Mux 405 and sends a second acquisition command to the Camera module 402 via I2C. The second acquisition command corresponds to a second image, thereby controlling the Camera module 402 to acquire the second image or video stream.

[0100] Camera module 402 connects to AI-MCU module 403 via low-power interface DVP / SPI to transmit a first image at a relatively low resolution and low frame rate (e.g., 720p or 1080p) for target recognition. Camera module 402 also connects to SoC module 404 via high-speed interface MIPI to transmit a second image or video stream at a high resolution (e.g., 4k) for streaming or image / video analysis.

[0101] Among them, DVP (Digital Video Port) is a digital video port. SPI (Serial Peripheral Interface) is a serial peripheral interface. MIPI (Mobile Industry Processor Interface) is a mobile industry processor interface.

[0102] The AI-MCU module 403 and the SoC module 404 can communicate bidirectionally after being woken up via UART (Universal Asynchronous Receiver / Transmitter).

[0103] In some embodiments, after the AI-MCU module 403 completes target recognition, it can send structured data such as the recognized target type (e.g., person, pet) and the target's ROI coordinates in the image to the SoC module 404 via UART. This allows the SoC module 404 to perform targeted processing (e.g., digital zoom, target box overlay) during recording or streaming. The SoC module 404 can query the AI-MCU module 403's current operating status (e.g., sleep / wake-up, battery level, background frame version) via UART, or dynamically adjust the detection parameters on the AI-MCU module 403 side (e.g., AI detection sensitivity, background frame update frequency) to achieve system-level collaborative optimization. The UART can also serve as a standard debugging interface, outputting the operation logs of the AI-MCU module 403 and the SoC module 404 for on-site troubleshooting. Simultaneously, it allows the SoC module 404 to perform firmware upgrades on the AI-MCU module 403 via UART, facilitating product iteration and maintenance.

[0104] WiFi module 406 can establish a bidirectional communication connection with AI-MCU module 403.

[0105] In some embodiments, the AI-MCU module 403 can establish a configuration network (such as enabling a built-in web server) via the WiFi module 406, allowing users to directly access and preview AI inference, adjust AI model parameters, and even remotely upgrade the AI ​​model. The AI-MCU module 403 can also be remotely woken up via the WiFi module 406. Furthermore, the AI-MCU module 403 can directly send internal operation logs and diagnostic information to developers via the WiFi module 406, facilitating troubleshooting or performance analysis.

[0106] Figure 6 A flowchart illustrating the operation of another camera device provided in an embodiment of this disclosure. (In conjunction with...) Figure 6 It is known that the camera device adopts a three-level power consumption architecture. The first-level power consumption architecture is as follows: most components in the camera device normally operate in a sleep state, with only the PIR module and a small number of wake-up circuits operating. The second-level power consumption architecture is as follows: when the PIR module detects a change in the infrared radiation flux in the environment and the change exceeds the trigger threshold, it wakes up the AI-MCU module. The AI-MCU module first acquires a first image, performs ROI cropping, and then sends the cropped image to the AI ​​model after appropriate scaling for target recognition. The third-level power consumption architecture is as follows: when the AI-MCU module detects a preset target, it wakes up the SoC module. The SoC module acquires a second image or video stream and performs subsequent processing. When the AI-MCU module does not detect the preset target, it returns to sleep mode and does not wake up the SoC module.

[0107] like Figure 6 As shown, the working process of the camera device includes the following steps S601 to S609.

[0108] In step S601, the camera device is in sleep mode, with only the PIR module and a small number of wake-up circuits operating; Step S602: After the PIR module is triggered, the AI-MCU module is woken up by an interrupt; Step S603: The AI-MCU module acquires the first image (e.g., 720p or 1080p). Step S604: The AI-MCU module runs the ROI detection algorithm to locate the ROI coordinates corresponding to the target region in the first image, where the target region is the region containing the potential target. Step S605: The AI-MCU module crops the corresponding target image block from the first image according to the ROI coordinates, and the target image block contains the target region; In step S606, the AI-MCU module appropriately scales the cropped target image block and sends it into the AI ​​model for target recognition to obtain the target recognition result. In step S607, the AI-MCU module determines whether a preset target has been detected. If yes, then step S608 is executed; otherwise, step S609 is executed. In step S608, the AI-MCU module wakes up the SoC module to acquire a second image or video stream (such as 4K); In step S609, the AI-MCU module and SoC module continue to sleep.

[0109] For the above workflow, a two-level wake-up mechanism (PIR physical trigger + AI-MCU logic confirmation) ensures that real intrusion events are not missed while filtering out invalid interference to the maximum extent. Moreover, the computational overhead of ROI detection and pruning is far less than that of a complete full-graph inference, and the AI-MCU module only works for a very short time after the PIR module is triggered, which can guarantee a low system power consumption level.

[0110] Figure 7 This is a flowchart of the ROI detection algorithm provided in an embodiment of this disclosure. The ROI detection algorithm is integrated into the AI-MCU module. Specifically, the AI-MCU module uses a lightweight ROI detection algorithm based on moving foreground extraction to extract regions where potential targets exist from the image, i.e., regions where targets may exist.

[0111] like Figure 7 As shown, the ROI detection algorithm includes the following steps S701 to S708.

[0112] Step S701: Control the Camera module to acquire the first image (e.g., 720p or 1080p).

[0113] The first image is a high-resolution image compared to the low-resolution images (such as 360p) acquired in related technologies. This high-resolution image can obtain more pixel information of potential targets, so that the target image blocks obtained by subsequent cropping retain more object features.

[0114] Step S702: Downsample and grayscale the first image to generate a low-resolution grayscale image.

[0115] After acquiring the first image from the camera module, a downsampled low-resolution grayscale image (preferably 320×240 or 160×120) can be generated via hardware or software as the grayscale image of the current frame. Let the downsampling factor be k. In one example, the first image resolution is 720p. Downsampling the 720p image to 320×240 results in k = 720 / 320 = 2.25. Downsampling significantly reduces the amount of data required for subsequent processing, and grayscale conversion eliminates color information interference, highlighting motion contours.

[0116] In addition, a grayscale image of the background frame is generated and cached using the same processing method.

[0117] Step S703: Extract the moving foreground using the frame difference method.

[0118] Perform pixel-level difference operations between the current frame grayscale image and the cached background frame grayscale image to obtain the difference image D.

[0119] The formula for difference operations is: D(x,y) = |I current (x,y) - I background (x,y)|; Among them, I current For the current frame's grayscale image, I current (x,y) represents the grayscale value of the current frame's grayscale image at position (x,y); background For the background frame grayscale image, I background (x,y) represents the gray value of the background frame grayscale image at position (x,y); D(x,y) represents the gray difference value at position (x,y) in the difference image D.

[0120] Furthermore, the background frame can adopt a dynamic update strategy: when no moving foreground is extracted for N consecutive frames, the background frame is updated to the first image of the most recently acquired frame (i.e. the latest current frame), where the value of N can be dynamically adjusted.

[0121] Step S704: Binarize the difference image.

[0122] The Otsu algorithm is used to calculate the adaptive threshold T of the difference image D, and the difference image D is binarized according to the calculation result to obtain the binary image B.

[0123] The Otsu algorithm is a classic global adaptive thresholding segmentation algorithm. Its core principle is to automatically find the optimal segmentation threshold T by maximizing the inter-class variance between moving and non-moving pixel classes in the difference image D, and then binarize the difference image D according to the threshold T to obtain a binary image of the moving foreground.

[0124] The binary image B is obtained using the following formula: .

[0125] In the binary image B, pixels with a value of 1 represent foreground points of motion; pixels with a value of 0 represent background points, corresponding to positions where there is no significant change between the current frame and the background frame.

[0126] Step S705: Perform morphological operations on the binary image.

[0127] When a binary image B represents a moving foreground, morphological opening is performed on the binary image B, specifically by erosion followed by dilation, to remove isolated noise points and form connected regions, resulting in the processed binary image B′.

[0128] The morphological opening operation specifically involves first eroding the binary image B, and then dilating the eroded result. Erosion involves sliding a structuring element S (e.g., a 3×3 structuring element) across the binary image B. The center pixel is retained as the moving foreground only when the structuring element S completely covers the moving foreground (pixels with a value of 1); otherwise, it is set to 0. Erosion can eliminate isolated noise and shrink the foreground region. Dilation involves sliding the eroded binary image using the same structuring element S. The center pixel is set as the moving foreground as long as there is at least one foreground pixel within the range of the structuring element S. Dilation can recover the eroded and shrunken region and connect nearby foreground fragments to form connected regions.

[0129] Step S706: Connectivity analysis generates the target region.

[0130] After scanning the processed binary image B′, all connected regions are marked. Then, the bounding rectangle parameters of each connected region are calculated, and the bounding rectangle of each connected region is obtained. At the same time, connected regions with too small an area (e.g., less than 1% of the total area of ​​the binary image B′) or unreasonable aspect ratio (e.g., aspect ratio > 5:1) are filtered out. These connected regions are usually formed by interference caused by leaf swaying, light changes, etc. Then, the area defined by the bounding rectangle of the remaining connected regions (i.e., connected regions that meet the preset requirements) is taken as the target region.

[0131] Step S707: ROI coordinate generation and mapping.

[0132] The coordinates of the bounding rectangle of the connected region in the low-resolution grayscale image generated in step S702, that is, the coordinates corresponding to the target region, are (x... low , y low , w low , h low Based on the downsampling factor k, the image is mapped back to the first image, resulting in the ROI coordinates: (x, y, w, h). Where x = x low ×k represents the column coordinate (horizontal position) of the top-left corner of the rectangle in the first image; y = y low ×k represents the row coordinates (vertical position) of the top-left corner of the rectangle in the first image; w = w low ×k represents the width of the rectangle (the number of pixels extending to the right from x); h = h low×k; represents the height of the rectangle (the number of pixels extending downwards from y).

[0133] Step S708, ROI cropping and scaling.

[0134] Based on the ROI coordinates obtained in step S707, the corresponding target image patch is cropped from the first image. The size of the cropped target image patch is w×h, but w and h may not be fixed for different first images. Since the subsequent AI model requires a fixed-size input, the target image patch needs to be uniformly scaled to a predetermined size (preferably 64×64 or 96×96 pixels) before being input into the AI ​​model for target recognition.

[0135] In one example, such as Figure 8 As shown, the original 720p image (i.e., the first image) is cropped and scaled by the ROI to obtain a scaled image of 64×64.

[0136] Practical verification shows that the embodiment of this disclosure adopts the operation of "first taking a large image and then cropping the ROI", which effectively magnifies distant targets before they are fed into the AI ​​model, and the detection distance can be improved by 50% to 100% (experimental data: for a human body with a height of 1.7 meters, the detection distance is increased from 5 meters in related technologies to more than 10 meters).

[0137] Table 1 below shows the comparison results between the embodiments of this disclosure and related technologies at the same distance: Table 1 It will be understood by those skilled in the art that all or some of the steps, systems, or apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software may be distributed on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0138] Example embodiments have been disclosed herein, and while specific terminology has been used, it is for illustrative purposes only and should be construed as such, and is not intended to be limiting. In some instances, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in connection with particular embodiments may be used alone, or in combination with features, characteristics, and / or elements described in connection with other embodiments, unless otherwise expressly indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of this disclosure as set forth by the appended claims.

Claims

1. An image processing method, characterized by, The image processing method is applied to an AI-MCU module, which is communicatively connected to an infrared monitoring module, an image acquisition module, and a SoC module. The image processing method includes: In response to a first wake-up command from the infrared monitoring module, the device wakes up from sleep mode. The first image is acquired through the image acquisition module; Target region detection is performed on the first image. If a target region is detected, the first image is cropped to obtain a target image block. The target image block contains the target region, which is a region containing a potential target. Target recognition is performed on the target image block to obtain the target recognition result; If the target recognition result indicates that the potential target is a preset target, a second wake-up command is sent to the SoC module to wake up the SoC module from hibernation and acquire a second image or video stream through the image acquisition module.

2. The image processing method of claim 1, wherein, The step of performing target region detection on the first image and cropping the first image to obtain a target image patch when a target region is detected includes: The target region is detected in the first image based on a preset ROI detection algorithm; If the target region is detected, the ROI coordinates corresponding to the target region in the first image are determined; Based on the ROI coordinates, the corresponding target image patch is cropped from the first image.

3. The image processing method of claim 2, wherein, The target region detection in the first image based on the preset ROI detection algorithm includes: The first image is downsampled and converted to grayscale according to a preset downsampling ratio to generate a first grayscale image; A pixel-level difference operation is performed between the first grayscale image and the second grayscale image to obtain a difference image; wherein, the second grayscale image is generated by downsampling and grayscale converting the background frame image according to the downsampling ratio; An adaptive threshold for the difference image is calculated using a preset adaptive threshold segmentation algorithm, and the difference image is binarized based on the calculation result to obtain a binary image. When the binary image representation contains a moving foreground, a morphological opening operation is performed on the binary image to remove isolated noise points and form connected regions, resulting in a processed binary image. If a connected region that meets the preset requirements exists in the processed binary image, the target region is determined to be detected, and the region defined by the outer rectangle of the connected region that meets the preset requirements is taken as the target region.

4. The image processing method according to claim 3, characterized in that, The step of detecting the target region in the first image further includes: If the target area is not detected, resume hibernation.

5. The image processing method according to claim 4, characterized in that, If the binary image representation does not contain a moving foreground, it is determined that the target region has not been detected. or If no connected region meeting the preset requirements exists in the processed binary image, it is determined that the target region has not been detected.

6. The image processing method according to claim 3, characterized in that, The method of detecting target regions in the first image based on a preset ROI detection algorithm further includes: If the binary image representation corresponding to the first image acquired a preset number of times does not contain a moving foreground, the background frame image is updated to the most recently acquired first image.

7. The image processing method according to claim 3, characterized in that, Determining the ROI coordinates corresponding to the target region in the first image includes: Based on the downsampling ratio, the coordinates corresponding to the target region in the first grayscale image are mapped to the first image to obtain the ROI coordinates.

8. The image processing method according to any one of claims 1-7, characterized in that, The step of performing target recognition on the target image patch to obtain the target recognition result includes: The target image block is scaled to a preset size to obtain a scaled image block; Based on the scaled image patch, the trained AI model outputs the category recognition result; If the category identification result includes a preset target type, the target identification result is determined to characterize the potential target as the preset target.

9. An AI-MCU module, characterized in that, The AI-MCU module includes: One or more processors; A memory having stored one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the image processing method according to any one of claims 1-8; One or more I / O interfaces are connected between the processor and memory and configured to enable information exchange between the processor and memory.

10. A camera device, characterized in that, The camera device includes: an infrared monitoring module, an image acquisition module, an AI-MCU module, and a SoC module; the AI-MCU module is communicatively connected to the infrared monitoring module, the image acquisition module, and the SoC module, respectively. After receiving the first wake-up command sent by the infrared monitoring module, the AI-MCU module wakes up from sleep mode, acquires a first image through the image acquisition module, and performs target region detection on the first image. If a target region is detected, the first image is cropped to obtain a target image block. The target image block is then used for target recognition to obtain a target recognition result. The target image block contains the target region, which is a region containing a potential target. If the target recognition result indicates that the potential target is a preset target, a second wake-up command is sent to the SoC module. Upon receiving the second wake-up command, the SoC module wakes up from sleep mode and acquires a second image or video stream through the image acquisition module.

Images