Image processing method for image acquisition device, electronic device and storage medium

Abstract

Disclosed in the present application are an image processing method for an image acquisition device, an electronic device and a storage medium. The image processing method for an image acquisition device comprises: in a low-frequency mode, a processor receives image data from an image sensor, at least some of functional modules in the low-frequency mode being in an unstarted state; in response to the received image data meeting a first image processing condition, adjusting the processor to a high-frequency mode to start each functional module, the first image processing condition comprising that the image data contains related information of a target object and / or the amount of the image data has reached a preset amount; and controlling each functional module to perform image processing on the image data. The solution can shorten the time of awakening each functional module in the image processing process of the image acquisition device, thereby reducing the consumption of battery power of the image acquisition device.

Description

Image processing method of image acquisition device, electronic device and storage medium

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] The present application claims priority to the Chinese patent application No. 2024106321211, filed on May 21, 2024, and entitled "Image processing method of image acquisition device, electronic device and storage medium", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0003] The present application relates to the technical field of computer, in particular to an image processing method of image acquisition device, an electronic device and a storage medium. BACKGROUND

[0004] With the rapid development of image acquisition technology, image acquisition devices capable of real-time image acquisition are applied more and more widely. In use, the existing image acquisition devices are in a start state so that each functional module can perform corresponding image processing after receiving corresponding image data. However, each functional module is in an awake state, which is essentially a high-power mode when no corresponding image processing is needed, resulting in serious power loss of the image acquisition device.

[0005] In view of the existing technical defects, how to provide an effective image processing scheme for image acquisition devices is a technical problem to be solved by those skilled in the art.

[0006] SUMMARY

[0007] The present application provides at least an image processing method of image acquisition device, an electronic device and a storage medium.

[0008] The present application provides an image processing method of image acquisition device, comprising: a processor receives image data from an image sensor in a low-frequency mode, wherein at least part of the functional modules in the low-frequency mode are in an unstarted state; in response to the received image data satisfying a first image processing condition, adjusting the processor to a high-frequency mode to start each functional module, the first image processing condition including that the image data contains related information of a target object and / or the number of image data reaches a preset number; controlling each functional module to perform image processing on the image data.

[0009] The application provides an image processing device of an image acquisition device, comprising a receiving module, an adjusting module and a control module; the receiving module is used for a processor to receive image data from an image sensor in a low-frequency mode, wherein at least part of function modules in the low-frequency mode are in an unstarted state; the adjusting module is used for adjusting the processor to a high-frequency mode to start the function modules in response to the received image data satisfying a first image processing condition, the first image processing condition comprising the image data containing relevant information of a target object and / or the quantity of the image data reaching a preset quantity; and the control module is used for controlling the function modules to perform image processing on the image data.

[0010] The application provides an electronic device comprising a memory and a processor, the processor being used to execute program instructions stored in the memory to implement the image processing method of the image acquisition device.

[0011] The application provides a computer-readable storage medium, the program instructions being stored on the computer-readable storage medium, the program instructions being executed by a processor to implement the image processing method of the image acquisition device.

[0012] The above scheme, in response to the image data received by the processor in the low-frequency mode satisfying the first image processing condition, adjusts the processor to the high-frequency mode to start the function modules, the first image processing condition comprising the image data containing relevant information of a target object and / or the quantity of the image data reaching a preset quantity, and controls the function modules to perform image processing on the image data, compared with the prior art in which the function modules are always in a wake-up state to wait for the image data to be issued and perform image processing, the application can reduce the wake-up time of the function modules in the image processing process of the image acquisition device, thereby reducing the consumption of the battery power of the image acquisition device.

[0013] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, but not limiting the application.

DRAWINGS

[0014] The accompanying drawings are incorporated into the specification and form a part of the specification, which show embodiments consistent with the application, and together with the specification, serve to explain the technical solutions of the application.

[0015] Fig. 1 is a flow schematic diagram of an embodiment of the image processing method of the image acquisition device of the application;

[0016] Fig. 2 is another flow schematic diagram of an embodiment of the image processing method of the image acquisition device of the application;

[0017] Fig. 3 is still another flow schematic diagram of an embodiment of the image processing method of the image acquisition device of the application;

[0018] FIG. 4 is a structural schematic diagram of an embodiment of an image processing apparatus of an image acquisition device according to the present application;

[0019] FIG. 5 is a structural schematic diagram of an embodiment of an electronic device according to the present application;

[0020] FIG. 6 is a structural schematic diagram of an embodiment of a computer readable storage medium according to the present application.

DETAILED DESCRIPTION

[0021] The scheme of the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0022] In the following description, specific details are set forth in order to provide a thorough understanding of the present application. However, persons having ordinary skill in the art will appreciate that the present application can be practiced without the specific details.

[0023] The term "and / or" herein is merely an associated relationship between associated objects, and means that there can be three relationships, for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone. In addition, the character " / " herein generally represents an "or" relationship between the front and rear associated objects. In addition, "multiple" herein means two or more than two. In addition, the term "at least one" herein means any one of multiple or any combination of at least two of multiple, for example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0024] The present application provides some image processing methods of image acquisition devices and image processing apparatuses of image acquisition devices. The application scenarios of the image processing methods of image acquisition devices include but are not limited to starting processes of multi-camera devices. The execution subject of the image processing method of the image acquisition device can be an image processing apparatus of the image acquisition device or a master control of the image acquisition device. For example, the image processing apparatus of the image acquisition device can be arranged in a terminal device or a server or other processing device, wherein the terminal device can be a device for image processing of the image acquisition device, a user equipment (User Equipment, UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (Personal Digital Assistant, PDA), a handheld device, a computing device, a vehicle-mounted device, etc. In some possible implementation manners, the image processing method of the image acquisition device can be realized by a processor calling computer readable instructions stored in a memory.

[0025] Please refer to FIG. 1, which is a flowchart of an embodiment of an image processing method of an image acquisition device. Specifically, the image acquisition device comprises an image sensor, a processor and a plurality of functional modules. The image processing method of the image acquisition device can comprise the following steps:

[0026] Step S11: The processor receives image data from the image sensor in a low-frequency mode.

[0027] At least part of the functional modules in the low-frequency mode are in an inactive state.

[0028] The processor can be a central processing unit (CPU) of the image acquisition device. The processor can be a processor with computing and control capabilities and can run at different frequencies.

[0029] The processor can have a low-frequency mode. The processor in the low-frequency mode has weak processing capability and low power consumption. In some application scenarios, the processor in the low-frequency mode can have the capability of receiving and sending image data. In some application scenarios, at least part of the functional modules in the image acquisition device are in an inactive state when the processor is in the low-frequency mode. The functional modules in the image acquisition device can be related modules capable of image processing. In some application scenarios, the functional modules in the image acquisition device can be hardware encoders. The hardware encoders can encode the received data into video files of different formats for storage in the memory. Specifically, the hardware encoders can be video encoders (ENC). The video encoders can be video encoders of different formats. For example, the video encoders can be first encoders (Video Encoder H264). The video encoders can be second encoders (Video Encoder H265). The video encoders can be third encoders (Video Encoder Motion Jpeg). The functional modules in the image acquisition device can also be image signal processors (ISP). The image sensor can be a sensor. The image sensor can send image data to the processor. The image sensor can have a motion detection mode. Specifically, the motion detection mode can be a motion detection function of the image sensor. In some application scenarios, the motion detection function can be a mobile detection function of the image sensor, which outputs a control signal when the picture changes. In some application scenarios, the processor mentioned above can be a master control chip connected to the image sensor.

[0030] Step S12: in response to the received image data satisfying the first image processing condition, adjusting the processor to a high frequency mode to start the function modules.

[0031] The first image processing condition includes that the image data contains relevant information of the target object and / or the number of the image data reaches a preset number.

[0032] The processor can have a high frequency mode. The processor in the high frequency mode has high processing capability and high power consumption. In some application scenarios, the processor in the high frequency mode can have other processing capabilities in addition to the capability of receiving and sending image data. The other processing capabilities can be sending corresponding data to the function modules to start the function modules to complete image data encoding. In some application scenarios, when the processor is in the high frequency mode, all function modules in the image acquisition device are in the starting state. It can be understood that after it is determined that the received image data satisfies the first image processing condition, the processor in the low frequency mode can be switched to the high frequency mode. After the processor is switched from the low frequency mode to the high frequency mode, the processor in the high frequency mode can start the function modules. After the processor is in the high frequency mode, the processor in the high frequency mode can start the function modules. In some application scenarios, the main control chip enters the high frequency mode, and a target software operating system is started synchronously to realize image processing related individualized setting requirements. Specifically, the process of starting the target software operating system synchronously can be a power management mode recovery into the operating system. The power management mode can be a target power mode STR (Suspend to RAM or Sleep, STR). The target power mode STR is the authority management of the operating system. According to different demand stages, the target power mode STR manages the authority of the operating system. In some application scenarios, the first demand stage can be that all operations of the operating system are stopped, but the memory still maintains power supply and keeps its content. In the first demand stage, the operating system saves energy and saves the battery energy of the image acquisition device when it is inactive. In other application scenarios, the second demand stage can be that the operating system is allowed to quickly recover to the full power running state when the demand condition is met. The demand condition can be the step of receiving the image data satisfying the first image processing condition or the step of adjusting the processor to the high frequency mode. Dividing the first demand stage and the second demand stage can improve the battery energy use efficiency.

[0033] In some application scenarios, it is determined whether the image data contains information related to a target object. The first image processing condition can be that the image data contains information related to the target object. The target object can be set according to requirements. Specifically, the target object can be a predetermined biological type, or a predetermined object type, or a predetermined vehicle type. In some application scenarios, it is determined whether the number of images corresponding to the image data reaches a preset number. The preset number can be that the number of images corresponding to the image data is 10. The first image processing condition can be that the number of images corresponding to the image data reaches the preset number. The images corresponding to the image data can be images obtained by the image acquisition device collecting the target region. In other application scenarios, the determination of whether the image data contains information related to the target object and the determination of whether the number of images corresponding to the image data reaches the preset number are performed synchronously. The first image processing condition can be that the image data contains information related to the target object and the number of images corresponding to the target object in the image data reaches the preset number.

[0034] Step S13: Control each functional module to perform image processing on the image data.

[0035] After starting each functional module, control each functional module to perform image processing on the image data. In some application scenarios, the image processing mode can be to send the image data into each functional module at one time to obtain video data. Illustratively, the form of the image data can be ImageRaw data. The image processing mode can be to send the image data into a hardware encoder for processing at one time, and wait for the encoding to be completed to obtain the video data. Specifically, the image data is sent into an image signal processor and a video encoder for processing at one time, and the encoding is waited to be completed to obtain the video data. After obtaining the video data, the above video data is taken as first target storage data, and the first target storage data is stored. After the first target storage data is stored, the processor is controlled to be powered off. In some application scenarios, the control of the processor to be powered off can be the control of the main control chip to be powered off.

[0036] In some embodiments, the image acquisition device further includes a memory. In some application scenarios, the storage mode of the video data can be to write the video data into the memory in the high-power consumption mode. In response to the storage of the video data being completed, the memory is adjusted to the low-power consumption mode, and the image sensor is controlled to enter the sleep state. After the memory is in the low-power consumption mode, the processor is controlled to be powered off. The control of the image sensor to enter the sleep state can be that the processor notifies the image sensor to enter the sleep state.

[0037] The above scheme adjusts the processor to a high-frequency mode to start the function modules in response to the image data received by the processor in the low-frequency mode satisfying a first image processing condition, the first image processing condition including the image data containing relevant information of a target object and / or the number of image data reaching a preset number, and the function modules performing image processing on the image data. Compared with the prior art in which the function modules are always in a wake-up state to wait for image data and perform image processing, the application can reduce the wake-up time of the function modules in the image processing process of the image acquisition device, thereby reducing the consumption of the battery power of the image acquisition device.

[0038] In some embodiments, the image acquisition device further includes a memory, and the image processing method of the image acquisition device can further include the following steps: first, in response to the received image data satisfying a second image processing condition, writing the image data to the memory in a high-power consumption mode. The second image processing condition includes that the image data does not contain relevant information of a target object and the number of image data does not reach a preset number. Subsequently, in response to the image data being stored, adjusting the memory to a low-power consumption mode, and controlling the image sensor to enter a sleep state.

[0039] The memory in the image acquisition device can be a random memory. For example, the memory can be a random memory DDR (Double Data Rate, DDR). Specifically, the random memory DDR can be a DDR SDRAM (Double Data Rate Synchronous Dynamic Random Access Memory). The random memory DDR can be a kind of memory technology mainly used for the random memory of a computer. The memory has a low-power consumption mode and a high-power consumption mode. In some application scenarios, the memory in the high-power consumption mode can be immediately executed after receiving a read-write instruction, and the speed is relatively fast. The memory in the high-power consumption mode has high power consumption. The memory in the high-power consumption mode can be a memory in a working state or a normal mode, and can read and write relevant data to be stored. In other application scenarios, the memory in the low-power consumption mode can automatically refresh the relevant data to be stored, and the speed is slow in this mode and can work at a low voltage. The memory in the low-power consumption mode has low power consumption. The memory in the low-power consumption mode can be a memory in a self-refresh mode and cannot read and write relevant data to be stored. It can be understood that the memory in the low-power consumption mode has the function of retaining historical storage data and does not have the function of reading and writing relevant data to be stored. The retained historical storage data can be video data stored by the memory in the high-power consumption mode before mode adjustment or image data stored by the memory in the high-power consumption mode before mode adjustment.

[0040] The second image processing condition includes that the image data does not contain the related information of the target object and the number of the image data does not reach the preset number. In response to the received image data satisfying the second image processing condition, the image data is written to the memory in the high power consumption mode. At this time, the image data can be original image data without being processed by the image signal processor and the video encoder. In response to the received image data satisfying the second image processing condition, the image data is taken as second target storage data, and the second target storage data is written to the memory in the high power consumption mode. After the storage of the second target storage data is completed, the memory is adjusted to the low power consumption mode, and the image sensor is controlled to enter the sleep state. After the memory is in the low power consumption mode, the processor is powered off. Controlling the image sensor to enter the sleep state can be that the processor notifies the image sensor to enter the sleep state.

[0041] It can be considered that, in the case that the received image data satisfies the second image processing condition, writing the image data to the memory in the high power consumption mode, adjusting the memory to the low power consumption mode, and controlling the image sensor to sleep and the processor to be powered off can cache the image data in the case that the motion state exists and the non-target object exists, and save the power consumption of each module in the image acquisition device, thereby improving the use efficiency of the battery of the image acquisition device.

[0042] In some embodiments, before the step of adjusting the processor to the high frequency mode in response to the received image data satisfying the first image processing condition, the image processing method of the image acquisition device can further include the following steps: first, the image data is cropped to obtain to-be-processed data. Then, the to-be-processed data is subjected to object detection processing and / or quantity statistical processing to obtain a data processing result. The image processing result includes an object detection result and / or a quantity statistical result, wherein the object detection result includes related information of the to-be-processed data containing the target object or related information of the to-be-processed data not containing the target object, and the quantity statistical result includes the total number corresponding to each image in the to-be-processed data. Then, whether the received image data satisfies the first image processing condition is determined based on the data processing result.

[0043] The image acquisition device further includes a video cropping module. The video cropping module can be a video cropping module VIF (Video Input Interface, VIF).

[0044] In some application scenarios, the video cropping module can transmit the received image data to the master control. The image data is subjected to object detection processing and / or quantity statistical processing to obtain a data processing result. In some application scenarios, the image data is subjected to object detection processing, and the obtained data processing result can be an object detection result. In some application scenarios, the image data is subjected to quantity statistical processing, and the obtained data processing result can be a quantity statistical result. In other application scenarios, the above-mentioned steps of subjecting the image data to object detection processing and the above-mentioned steps of subjecting the image data to quantity statistical processing are performed synchronously, and the obtained data processing result can be an object detection result and a quantity statistical result.

[0045] In other application scenarios, the video cropping module can crop the image data to obtain processed data. The video cropping module can transmit the processed data to the master control. The processed data is subjected to object detection processing and / or quantity statistical processing to obtain a data processing result. In some application scenarios, the processed data is subjected to object detection processing, and the obtained data processing result can be an object detection result. In some application scenarios, the processed data is subjected to quantity statistical processing, and the obtained data processing result can be a quantity statistical result. In other application scenarios, the above-mentioned steps of subjecting the processed data to object detection processing and the above-mentioned steps of subjecting the processed data to quantity statistical processing are performed synchronously, and the obtained data processing result can be an object detection result and a quantity statistical result. The relationship between the processed data and the image data can be that the processed data is the cropped image data, and the number of images in the processed data is the same as the number of images in the image data.

[0046] In some application scenarios, the above-mentioned step of subjecting the processed data to quantity statistical processing or the above-mentioned step of subjecting the image data to quantity statistical processing can be frame rate counting of the processed data or the image data. The frame rate counting can be that a historical statistical quantity is added to the number of images in the image data to obtain a quantity statistical result. Or, the historical statistical quantity is added to the number of images in the processed data to obtain a quantity statistical result.

[0047] Specifically, the process of frame rate counting can refer to formula (1):

[0048] RawCount1 = RawCount0 + N Formula (1);

[0049] wherein RawCount0 can represent the above-mentioned historical statistical quantity. The historical statistical quantity can be the number of historical image data images retained at a historical time in the memory. N can represent the number of images in the processed data or the image data at the current time. Exemplarily, N can be 1. RawCount1 can represent the quantity statistical result.

[0050] The image processing result includes an object detection result and / or a quantity statistical result. The object detection result includes information about whether the target object is contained in the to-be-processed data or not contained in the to-be-processed data. The quantity statistical result includes a total quantity corresponding to each image in the to-be-processed data. It is determined whether the received image data satisfies a first image processing condition based on the data processing result.

[0051] In some application scenarios, the object detection processing on the to-be-processed data is performed first, and the data processing result can be the object detection result. In the case that the object detection result is information about whether the target object is contained in the to-be-processed data or not contained in the to-be-processed data, the quantity statistical processing on the image data is performed again. The quantity statistical result is a total quantity corresponding to each image in the to-be-processed data. In the case that the total quantity corresponding to each image in the to-be-processed data corresponding to the quantity statistical result does not reach a preset quantity, it is determined that the image processing result determines that the received image data satisfies a second image processing condition. The to-be-processed data is stored as second target storage data, and the second target storage data is written into the storage in the high-power consumption mode. After the second target storage data is stored completely, the storage is adjusted to a low-power consumption mode, and the image sensor is controlled to enter a sleep state.

[0052] It can be understood that the object detection processing on the to-be-processed data and the quantity statistical processing on the to-be-processed data can be performed in a serial manner or in a parallel manner. The object detection processing on the image data can refer to the object detection processing on the to-be-processed data, which will not be described herein. The quantity statistical processing on the image data can refer to the quantity statistical processing on the to-be-processed data, which will not be described herein.

[0053] In some application scenarios, in the serial manner, the object detection processing on the to-be-processed data is performed first, and the data processing result can be the object detection result. In the case that the object detection result is information about whether the target object is contained in the to-be-processed data or not contained in the to-be-processed data, the quantity statistical processing on the to-be-processed data is performed again. In the case that a total quantity corresponding to each image in the to-be-processed data corresponding to the quantity statistical result reaches a preset quantity, it is determined that the image processing result determines that the received to-be-processed data satisfies a first image processing condition.

[0054] In some application scenarios, in a serial execution manner, the step of performing object detection processing on the to-be-processed data is performed first, and the data processing result can be an object detection result. In a case where the object detection result is information related to the to-be-processed data not containing the target object, the step of performing quantity counting processing on the to-be-processed data is performed again. In a case where the total quantity of each image corresponding to the to-be-processed data corresponding to the quantity counting result does not reach the preset quantity, it is determined that the image processing result determines that the received to-be-processed data satisfies the second image processing condition.

[0055] In some application scenarios, in a serial execution manner, the step of performing object detection processing on the to-be-processed data is performed first, and the data processing result can be an object detection result. In a case where the object detection result is information related to the to-be-processed data containing the target object, the step of performing quantity counting processing on the to-be-processed data is performed again. In a case where the total quantity of each image corresponding to the to-be-processed data corresponding to the quantity counting result reaches the preset quantity, it is determined that the image processing result determines that the received to-be-processed data satisfies the first image processing condition.

[0056] In some application scenarios, in a serial execution manner, the step of performing object detection processing on the to-be-processed data is performed first, and the data processing result can be an object detection result. In a case where the object detection result is information related to the to-be-processed data containing the target object, the step of performing quantity counting processing on the to-be-processed data is performed again. In a case where the total quantity of each image corresponding to the to-be-processed data corresponding to the quantity counting result reaches the preset quantity, it is determined that the image processing result determines that the received to-be-processed data satisfies the first image processing condition.

[0057] In some application scenarios, in a serial execution manner, the step of performing object detection processing on the to-be-processed data is performed first, and the data processing result can be an object detection result. In a case where the object detection result is information related to the to-be-processed data containing the target object, the step of performing quantity counting processing on the to-be-processed data is performed again. In a case where the total quantity of each image corresponding to the to-be-processed data corresponding to the quantity counting result reaches the preset quantity, it is determined that the image processing result determines that the received to-be-processed data satisfies the first image processing condition.

[0058] In some application scenarios, in response to the received to-be-processed data satisfying the first image processing condition, the processor is adjusted to the high-frequency mode to start the function modules. The function modules are controlled to perform image processing on the to-be-processed data to obtain video data. After obtaining the video data, the video data is taken as first target storage data, and the first target storage data is stored. After the first target storage data is completely stored, the storage is adjusted to the low-power consumption mode, and the image sensor is controlled to enter the sleep state. After the first target storage data is completely stored, the processor is controlled to be powered off.

[0059] In some application scenarios, in response to the received to-be-processed data satisfying the second image processing condition, the to-be-processed data is taken as second target storage data, and the second target storage data is written into the storage in the high-power consumption mode. After the second target storage data is completely stored, the storage is adjusted to the low-power consumption mode, and the image sensor is controlled to enter the sleep state. After the storage is in the low-power consumption mode, the processor is controlled to be powered off.

[0060] It can be considered that, in the case that the received image data satisfies the second image processing condition, writing the image data into the storage in the high-power consumption mode, adjusting the storage to the low-power consumption mode, and controlling the image sensor to sleep and the processor to be powered off can cache the image data in the case that the image acquisition device is in the motion state and there is a non-target object, and save the power consumption of the modules in the image acquisition device, thereby improving the use efficiency of the battery of the image acquisition device.

[0061] It can be considered that, in the case that the received image data satisfies the first image processing condition, writing the video data into the storage in the high-power consumption mode, adjusting the storage to the low-power consumption mode, and controlling the image sensor to sleep and the processor to be powered off can store the video data in the case that the image acquisition device is in the motion state and there is a target object and / or the total number of images corresponding to the to-be-processed data corresponding to the number statistics result reaches a preset number, and save the power consumption of the modules in the image acquisition device, thereby improving the use efficiency of the battery of the image acquisition device.

[0062] In some embodiments, the working state of the image sensor includes a motion detection mode and a picture output mode. Before step S11, the image processing method of the image acquisition device can further include the following steps: first, the image sensor in the motion detection mode is controlled to detect the image data to obtain a motion detection result. The motion detection result includes a first motion detection result related to the information about the motion object contained in the image data. Then, in response to the motion detection result being the first motion detection result, the image sensor is adjusted to the picture output mode so that the processor receives the image data from the image sensor in the low-frequency mode.

[0063] The image sensor has an active state and a sleep state. The image sensor in the active state can have a motion detection mode and an image output mode. Specifically, the motion detection mode can be a motion detection function of the image sensor. In some application scenarios, the motion detection function can be a mobile detection function of the image sensor, which outputs a control signal when the picture changes. Exemplarily, the motion detection mode can be a motion detection function (Motion Detect, MD). Specifically, the image output mode can be that the image sensor can send the collected image data to the processor.

[0064] The moving object can be all to-be-confirmed objects including the target object. The to-be-confirmed object can be a predetermined biological type, a predetermined object type, or a predetermined vehicle type. Specifically, the to-be-confirmed object can be an object type that can cause the picture corresponding to the target region to change in the collected image data. The image sensor in the motion detection mode is controlled to detect the image data to obtain a motion detection result. The motion detection result is a first motion detection result about related information of the moving object in the image data. In the case that the motion detection result is the first motion detection result, the image sensor is adjusted to the image output mode so that the processor receives the image data from the image sensor in the low-frequency mode.

[0065] In some embodiments, the motion detection result includes a second motion detection result about related information of the moving object not included in the image data, and the image processing method of the image acquisition device can further include the following steps: in response to the motion detection result being the second motion detection result, controlling the image sensor to enter the sleep state.

[0066] The image sensor in the motion detection mode is controlled to detect the image data to obtain a motion detection result. The motion detection result is a first motion detection result about related information of the moving object in the image data. In the case that the motion detection result is the second motion detection result, the image sensor is controlled to enter the sleep state.

[0067] In some embodiments, the first motion detection result comprises a plurality of current motion detection events, the plurality of current motion detection events representing a plurality of to-be-confirmed objects in a motion state, and the step of adjusting the image sensor to the image output mode in response to the motion detection result being the first motion detection result can comprise the following steps: in response to the first motion detection result satisfying a first preset condition, adjusting the image sensor to the image output mode, the first preset condition comprising that there is no historical motion detection event before a current timestamp corresponding to a current motion detection event in the first motion detection result. The historical motion detection event represents that the to-be-confirmed object is in a motion state before the current timestamp. Alternatively, in response to the first motion detection result satisfying a second preset condition, adjusting the image sensor to the image output mode, the second preset condition comprising that in the case that there is a historical motion detection event before the current timestamp corresponding to the current motion detection event in the first motion detection result, and the interval between the current timestamps corresponding to at least one current motion detection event in the first motion detection result is greater than or equal to a second preset time.

[0068] The first motion detection result comprises a plurality of current motion detection events. The plurality of current motion detection events can be one current motion detection event or a plurality of current motion detection events. The plurality of current motion detection events represents a plurality of to-be-confirmed objects in a motion state. The current motion detection event represents one to-be-confirmed object in a motion state. The to-be-confirmed objects between the plurality of current motion detection events can be different objects or the same object.

[0069] Exemplarily, in the case that the image sensor collects image data for the first time after the image sensor is powered on, it is determined that there is no historical motion detection event before the first time of collecting image data. Alternatively, in the case that the image sensor collects image data for multiple times, other image data before the last image data does not detect a historical motion detection event, the last image data detects a motion detection event, and there is no historical motion detection event before the current motion detection event corresponding to the last image data. There is no historical motion detection event can be that there is no historical image data or no historical to-be-processed data in the memory.

[0070] The historical motion detection event indicates that the object to be confirmed is in a motion state before the current timestamp. Each current motion detection event or historical motion detection event corresponds to a timestamp respectively. That is, the current timestamp can be a start time of the object to be confirmed being in a motion state in the current motion detection event. The current timestamp can be an end time of the object to be confirmed being in a motion state in the current motion detection event. The current timestamp can be a current time period of the object to be confirmed being in a motion state in the current motion detection event. The current time period includes a start time and an end time of the object to be confirmed being in a motion state. The historical timestamp can be a start time of the object to be confirmed being in a motion state in the historical motion detection event. The historical timestamp can be an end time of the object to be confirmed being in a motion state in the historical motion detection event. The historical timestamp can be a historical time period of the object to be confirmed being in a motion state in the historical motion detection event. The historical time period includes a start time and an end time of the object to be confirmed being in a motion state.

[0071] In some application scenarios, the first preset condition includes that there is no historical motion detection event before the current timestamp corresponding to the current motion detection event in the first motion detection result. It can be understood that in the case that there is no historical motion detection event before the current timestamp of a plurality of current motion detection events corresponding to the image data collected by the image sensor, the image sensor is directly adjusted to the image output mode.

[0072] In other application scenarios, the second preset condition includes that in the case that there is a historical motion detection event before the current timestamp corresponding to the current motion detection event in the first motion detection result, and the interval time between the current timestamps corresponding to at least one current motion detection event in the first motion detection result is greater than or equal to the second preset time. It can be understood that in the case that there is a historical motion detection event before the current timestamp of a plurality of current motion detection events corresponding to the image data collected by the image sensor, the relationship between the current timestamp of the plurality of current motion detection events and the historical timestamp of the historical motion detection event is determined. In response to the relationship between the current timestamp of the plurality of current motion detection events and the historical timestamp of the historical motion detection event satisfying the second preset time, the image sensor is adjusted to the image output mode. In some application scenarios, in response to the interval time between the current timestamp of at least one current motion detection event and the historical timestamp of the historical motion detection event being greater than or equal to the second preset time. In other application scenarios, in response to the interval time between the current timestamp of the first current motion detection event and the historical timestamp of the historical motion detection event being greater than or equal to the second preset time. The current timestamp of the first current motion detection event can be the time of the first motion detection event in the plurality of current timestamps corresponding to the plurality of current motion detection events. The historical motion detection event can be the motion detection event adjacent to the first current motion detection event and before the current timestamp corresponding to the first current motion detection event in the plurality of historical motion detection events.

[0073] It can be understood that the first preset time and the second preset time can be the same time or different time.

[0074] It can be considered that, in the case that there is no current motion detection event in the image processing data, the image sensor directly enters the sleep state, waits for the timing wake-up, and re-detects the motion detection event, which can reduce the wake-up time of the image sensor, so that the image sensor is not always in the working state in the wake-up state, and the power consumption of each module in the image acquisition device can be saved, thereby improving the use efficiency of the battery of the image acquisition device.

[0075] In some embodiments, the image processing method of the image acquisition device can further include the following steps: in response to the first motion detection result satisfying a third preset condition, controlling the image sensor to enter the sleep state, the third preset condition including the case that there is a historical motion detection event before the current timestamp corresponding to the current motion detection event in the first motion detection result, and the interval time between the current timestamp corresponding to all current motion detection events in the first motion detection result and the historical timestamp of the historical motion detection event is less than a third preset time.

[0076] In some application scenarios, the third preset condition includes the case that there is a historical motion detection event before the current timestamp corresponding to the current motion detection event in the first motion detection result, and the interval time between the current timestamp corresponding to all current motion detection events in the first motion detection result and the historical timestamp of the historical motion detection event is less than a third preset time. It can be understood that, in the case that there is a historical motion detection event before the current timestamp corresponding to the current motion detection event in the image data collected by the image sensor, the relationship between the current timestamp of the current motion detection event and the historical timestamp of the historical motion detection event is judged. In response to the relationship between the current timestamp of the current motion detection event and the historical timestamp of the historical motion detection event satisfying the third preset time, the image sensor is controlled to enter the sleep state. In some application scenarios, in response to the interval time between the current timestamp of each current motion detection event and the historical timestamp of the historical motion detection event being less than the third preset time. In other application scenarios, in response to the interval time between the current timestamp of the last current motion detection event and the historical timestamp of the historical motion detection event being less than the third preset time. The current timestamp of the last current motion detection event can be the time of the last motion detection event in the current timestamp corresponding to the current motion detection event. The historical motion detection event can be the motion detection event adjacent to the first current motion detection event and before the current timestamp corresponding to the first current motion detection event among the historical motion detection events.

[0077] It can be understood that the third preset time, the first preset time and the second preset time can be the same time or different time.

[0078] It can be considered that in the case that there is a current motion detection event in the image processing data, the relationship between the current motion detection event and the historical motion detection event or the relationship between the current motion detection events is judged, so as to determine that the image sensor is in the image output mode or the sleep state, so that the image sensor is not always in the wake-up state or the motion detection mode, the power consumption of each image sensor in the image acquisition device can be saved, and the use efficiency of the battery of the image acquisition device is improved.

[0079] In some embodiments, before the step S11, the image processing method of the image acquisition device can further include the following steps: first, in response to the power-on of the image sensor, setting a wake-up condition for the image sensor. Then, the image sensor is controlled to switch states at a predetermined frequency. The wake-up condition includes adjusting the image sensor in the sleep state to the working state after a target wake-up time, wherein the image sensor in the working state can be used to acquire image data.

[0080] In response to the power-on of the image sensor, a wake-up condition is set for the image sensor. Then, the image sensor is controlled to switch states at a predetermined frequency. The wake-up condition set for the image sensor can be that the image sensor will switch from the sleep state to the working state within a target wake-up time. Specifically, after the image sensor switches from the sleep state to the working state within the target wake-up time, the image sensor enters the motion detection mode. The wake-up condition includes adjusting the image sensor in the sleep state to the working state after a first preset time, wherein the image sensor in the working state can be used to acquire image data. The target wake-up time can be dynamically set according to the requirements of image processing.

[0081] It can be considered that after setting the wake-up condition, the image sensor can be awakened regularly, so that the image sensor is not always in the sleep state, the efficiency of the image sensor in acquiring image data and motion detection can be improved, and thus the efficiency of image processing of the image acquisition device is improved.

[0082] In some embodiments, before the processor receives the image data from the image sensor in the low-frequency mode, the image processing method of the image acquisition device can further include the following steps: adjusting the exposure parameter of the image acquisition device. The wake-up condition is set for the adjusted image sensor so that the adjusted image sensor switches states at a predetermined frequency.

[0083] The exposure parameter adjustment can be a Fast Auto Exposure (Fast Ae) which is a technique used in camera systems to quickly adjust exposure settings. The adjusted image sensor can be to find the best exposure setting as quickly as possible in order to quickly adapt to changing ambient light conditions. Ensuring that the image sensor captures images with consistent exposure can be stabilized in a short time.

[0084] It can be considered that the adjusted image sensor switches states at a predetermined frequency, and the collected image data is stable output data.

[0085] The above scheme, in response to the image data received by the processor in the low frequency mode satisfying the first image processing condition, adjusts the processor to the high frequency mode to start each functional module, the first image processing condition includes that the image data contains the related information of the target object and / or the number of image data reaches the preset number, and each functional module controls the image data to perform image processing. Compared with the prior art in which each functional module is always in a wake-up state to wait for image data and perform image processing, the application can reduce the wake-up time of each functional module in the image processing process of the image acquisition device, thereby reducing the consumption of the battery power of the image acquisition device.

[0086] Please refer to FIG. 2, which is another flowchart of an embodiment of the image processing method of the image acquisition device.

[0087] Steps S201, S202, S203 and S204 are executed in sequence. Step S201: power on the image sensor. Step S202: adjust the exposure parameter of the image acquisition device. Step S203: set a wake-up condition for the image sensor. Step S204: the image sensor enters a motion detection mode and detects the collected image data to obtain a motion detection result. Determine whether the motion detection result is a first motion detection result or a second motion detection result. In the case of the first motion detection result, execute step S205. Step S205: in response to the first motion detection result satisfying a first preset condition, adjust the image sensor to an image output mode. In the case of the second motion detection result, execute steps S206, S207 and S208 in sequence. Step S206: in response to the motion detection result being the second motion detection result, control the image sensor to enter a sleep state. Step S207: determine whether the image sensor satisfies the wake-up condition. Step S208: the image sensor remains in sleep.

[0088] It can be considered that controlling the image sensor to enter the sleep state and the working state under different conditions can reduce the number of false wake-ups of the image sensor, thereby improving the use efficiency of the battery of the image acquisition device.

[0089] Please refer to Fig. 3, which is another flowchart of an embodiment of the image processing method of the image acquisition device. As shown in Fig. 3, the flow of the image processing of the image acquisition device can include the following steps:

[0090] Steps S301 and S302 are executed in sequence. Step S301: the processor receives image data from the image sensor in the out-of-picture mode in the low-frequency mode. Step S302: the video cropping module crops the image data to obtain to-be-processed data. It is determined whether the to-be-processed data meets the first image condition and whether the to-be-processed data meets the second image processing condition. In the case where the to-be-processed data meets the first image processing condition, steps S303, S304 and S305 are executed in sequence. Step S303: in response to the received to-be-processed data meeting the first image processing condition, the processor is adjusted to the high-frequency mode. Step S304: the to-be-processed data is processed by the functional modules to obtain video data. Step S305: in response to the storage of the video data being completed, the storage is adjusted to the low-power mode, and the image sensor is controlled to enter the sleep state. In the case where the to-be-processed data meets the second image processing condition, steps S306 and S307 are executed in sequence. Step S306: in response to the received image data meeting the second image processing condition, the to-be-processed data is written to the storage in the high-power mode. Step S307: in response to the storage of the to-be-processed data being completed, the storage is adjusted to the low-power mode, and the image sensor is controlled to enter the sleep state.

[0091] It can be considered that, under different image processing conditions, the start of the functional modules in the image acquisition device is controlled, which can reduce the start of the functional modules by non-target objects, thereby improving the use efficiency of the battery of the image acquisition device. In addition, under different use requirements, the switching of the power modes of the modules in the image acquisition device can improve the use efficiency of the battery of the image acquisition device.

[0092] The above scheme, in response to the image data received by the processor in the low-frequency mode meeting the first image processing condition, adjusts the processor to the high-frequency mode to start the functional modules, the first image processing condition includes that the image data contains the related information of the target object and / or the number of image data reaches the preset number, and the functional modules are controlled to process the image data. Compared with the prior art in which the functional modules are always in the wake-up state to wait for the image data and process the image data, the present application can reduce the wake-up time of the functional modules in the image processing process of the image acquisition device, thereby reducing the consumption of the battery power of the image acquisition device.

[0093] Referring to FIG. 4, FIG. 4 is a structural schematic diagram of an embodiment of the image processing apparatus of the image acquisition device. The image processing apparatus 40 of the image acquisition device includes a receiving module 41, an adjusting module 42, and a control module 43. The receiving module 41 is configured to receive image data from the image sensor by the processor in a low-frequency mode, where at least part of the function modules in the low-frequency mode are in an unactivated state. The adjusting module 42 is configured to adjust the processor to a high-frequency mode to activate the function modules in response to the received image data satisfying a first image processing condition, where the first image processing condition includes that the image data contains relevant information of a target object and / or the number of image data reaches a preset number. The control module 43 is configured to control the function modules to perform image processing on the image data.

[0094] According to the above scheme, in response to the image data received by the processor in the low-frequency mode satisfying the first image processing condition, the processor is adjusted to the high-frequency mode to activate the function modules, the first image processing condition includes that the image data contains relevant information of a target object and / or the number of image data reaches a preset number, and the function modules are controlled to perform image processing on the image data. Compared with the prior art in which the function modules are always in an awakened state to wait for image data and perform image processing, the present application can reduce the awakening time of the function modules in the image processing process of the image acquisition device, thereby reducing the consumption of the battery power of the image acquisition device.

[0095] The functions performed by each module are described in the image processing method of the image acquisition device, which will not be repeated here.

[0096] Referring to FIG. 5, FIG. 5 is a structural schematic diagram of an embodiment of the electronic device. The electronic device 50 includes a memory 51 and a processor 52, where the processor 52 is configured to execute program instructions stored in the memory 51 to implement the steps in the above-mentioned embodiments of the image processing method of the image acquisition device. In a specific implementation scenario, the electronic device 50 can include but is not limited to a multi-camera device, a microcomputer, and a server. In addition, the electronic device 50 can also include a notebook computer, a tablet computer, and other mobile devices, which are not limited herein.

[0097] Specifically, the processor 52 is configured to control itself and the memory 51 to implement the steps in the embodiments of the image processing method of the image acquisition device. The processor 52 can also be referred to as a CPU (Central Processing Unit). The processor 52 can be an integrated circuit chip with processing capability. The processor 52 can also be a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. In addition, the processor 52 can be implemented by an integrated circuit chip together.

[0098] In response to the image data received by the processor in the low-frequency mode satisfying the first image processing condition, the processor is adjusted to the high-frequency mode to start the function modules, the first image processing condition includes that the image data contains the related information of the target object and / or the number of image data reaches a preset number, and the function modules are controlled to perform image processing on the image data. Compared with the prior art in which the function modules are always in an awake state to wait for image data to be issued and perform image processing, the application can reduce the wake-up time of the function modules in the image processing process of the image acquisition device, thereby reducing the consumption of the battery power of the image acquisition device.

[0099] Referring to FIG. 6, FIG. 6 is a structural schematic diagram of an embodiment of the computer readable storage medium of the application. The computer readable storage medium 60 has program instructions 601 stored thereon, and the program instructions 601 are executed by the processor to implement the steps in any of the embodiments of the image processing method of the image acquisition device.

[0100] In response to the image data received by the processor in the low-frequency mode satisfying the first image processing condition, the processor is adjusted to the high-frequency mode to start the function modules, the first image processing condition includes that the image data contains the related information of the target object and / or the number of image data reaches a preset number, and the function modules are controlled to perform image processing on the image data. Compared with the prior art in which the function modules are always in an awake state to wait for image data to be issued and perform image processing, the application can reduce the wake-up time of the function modules in the image processing process of the image acquisition device, thereby reducing the consumption of the battery power of the image acquisition device.

[0101] In some embodiments, the apparatus provided by the embodiments of the present disclosure has functions or includes modules that can be used to perform the methods described in the above method embodiments, and the specific implementation can be referred to the description of the above method embodiments. For brevity, details are not repeated here.

[0102] The above description of various embodiments tends to emphasize the differences between various embodiments, and the same or similar parts can be mutually referred to, and for brevity, details are not repeated here.

[0103] In several embodiments provided in the present application, it should be understood that the disclosed methods and apparatuses can be implemented in other ways. For example, the above-described apparatus implementation is only schematic, and for example, the division of the modules or units is only a logical function division, and there can be another division manner in actual implementation, for example, a unit or component can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the shown or discussed mutual elements can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms.

[0104] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0105] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that makes a contribution to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor execute all or part of the steps of the methods in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

Claims

1. An image processing method of an image capturing apparatus, characterized by, The image acquisition device comprises an image sensor, a processor and a plurality of functional modules, and the method comprises: The processor receives image data from the image sensor in a low-frequency mode, wherein at least part of the functional modules in the low-frequency mode are in an inactive state; In response to the received image data satisfying a first image processing condition, the processor is adjusted to a high-frequency mode to activate the functional modules, the first image processing condition comprising relevant information of a target object contained in the image data and / or a quantity of the image data reaching a preset quantity; The functional modules control the image data for image processing.

2. The method of claim 1, wherein, The image acquisition device further comprises a memory, and the method further comprises: In response to the received image data satisfying a second image processing condition, the image data is written to the memory in a high-power consumption mode, the second image processing condition comprising relevant information of the target object not being contained in the image data and the quantity of the image data not reaching the preset quantity; In response to the image data being stored, the memory is adjusted to a low-power consumption mode, and the image sensor is controlled to enter a sleep state.

3. The method of claim 2, wherein, Before the processor is adjusted to the high-frequency mode in response to the received image data satisfying the first image processing condition, the method further comprises: The image data is cropped to obtain to-be-processed data; The to-be-processed data is subjected to object detection processing and / or quantity statistical processing to obtain a data processing result, the data processing result comprising an object detection result and / or a quantity statistical result, wherein the object detection result comprises relevant information of the target object contained in the image data or relevant information of the target object not being contained in the image data, and the quantity statistical result comprises a total quantity corresponding to each image in the to-be-processed data; Whether the received image data satisfies the first image processing condition is determined based on the data processing result.

4. The method according to any one of claims 1 to 3, characterized in that, The working state of the image sensor comprises a motion detection mode and an image output mode, and before the processor receives image data from the image sensor in a low-frequency mode, the method further comprises: The image sensor in the motion detection mode detects the image data to obtain a motion detection result, the motion detection result comprising a first motion detection result about relevant information of a motion object contained in the image data; In response to the motion detection result being the first motion detection result, the image sensor is adjusted to the image output mode so that the processor receives image data from the image sensor in a low-frequency mode.

5. The method of claim 4, wherein, The first motion detection result comprises a plurality of current motion detection events, the plurality of current motion detection events representing a plurality of to-be-confirmed objects being in a motion state, and in response to the motion detection result being the first motion detection result, the image sensor is adjusted to the image output mode, comprising: in response to the first motion detection result satisfying a first preset condition, adjusting the image sensor to the out-picture mode, the first preset condition including that there is no historical motion detection event before a current timestamp corresponding to the current motion detection event in the first motion detection result, the historical motion detection event representing that the object to be confirmed is in a motion state before the current timestamp; or, in response to the first motion detection result satisfying a second preset condition, adjusting the image sensor to the out-picture mode, the second preset condition including that there is a historical motion detection event before a current timestamp corresponding to the current motion detection event in the first motion detection result, and a time interval between current timestamps corresponding to at least one of the current motion detection events in the first motion detection result is greater than or equal to a second preset time.

6. The method of claim 5, wherein, The method further includes: in response to the first motion detection result satisfying a third preset condition, controlling the image sensor to enter a sleep state, the third preset condition including that there is a historical motion detection event before a current timestamp corresponding to the current motion detection event in the first motion detection result, and a time interval between a current timestamp corresponding to all of the current motion detection events in the first motion detection result and a historical timestamp of the historical motion detection event is less than a third preset time.

7. The method of claim 4, wherein, The motion detection result includes a second motion detection result related to information about the image data not containing a motion object, and the method further includes: in response to the motion detection result being the second motion detection result, controlling the image sensor to enter the sleep state.

8. The method according to any one of claims 1 to 3, characterized in that, Before the processor receives image data from the image sensor in a low-frequency mode, the method further includes: in response to the image sensor being powered on, setting a wake-up condition for the image sensor; controlling the image sensor to switch states at a predetermined frequency, the wake-up condition including adjusting the image sensor in a sleep state to a working state after a target wake-up time, wherein the image sensor in the working state can be used to collect the image data.

9. An electronic device, comprising: comprise: a memory and a processor, wherein the memory stores program instructions, and the processor retrieves the program instructions from the memory to execute the method according to any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, comprise: a program file stored in the memory, and the program file is executed by the processor to implement the method according to any one of claims 1-8.

Images