Image processing control system and method, electronic equipment and storage medium

Abstract

The invention provides an image processing control system and method, electronic equipment and a storage medium, relates to the field of control systems, and is used for closing the control system or operating the control system with low power consumption under the condition of detecting that a target object does not appear in to-be-detected image data. Because the RISC-V architecture has the characteristics of simple instruction set, strong formulability, support of power management technology (such as dynamic voltage frequency adjustment, power consumption gating and the like), optimized micro-architecture design (multi-emission and out-of-order execution) and the like, the instruction execution can be simpler and more efficient, necessary functional modules of the control system are reserved, redundant parts are removed, and the system performance is improved. Voltage, frequency and the like are dynamically adjusted according to the size of the load. And in combination with target object detection, closing the control system or operating the control system with low power consumption under the condition that the target object does not appear in the to-be-detected image data. The purposes of effectively reducing the power consumption of the system and prolonging the endurance time of the system are achieved.

Description

Technical Field

[0001] The present application relates to the field of control systems, and in particular to an image processing control system, method, electronic device and storage medium. Background Art

[0002] With the development of electronic devices, the functions of electronic devices are becoming increasingly complex. Some electronic devices are generally equipped with battery power to meet the needs of portability and flexible deployment. However, due to the limitation of battery power, how to reduce the overall power consumption of electronic devices and extend the battery life of electronic devices is a technical problem that needs to be solved urgently.

[0003] Depending on the application scenario and the functions to be implemented, electronic devices can be composed of different functional components. For example, electronic devices in application scenarios such as area acquisition and video security, or to implement functions such as shooting and recording, usually need to be equipped with camera components and components such as image signal processors. However, as a high-power component, the camera will increase the power consumption of electronic devices and reduce the battery life of electronic devices if it is kept turned on all the time. Summary of the invention

[0004] The embodiments of the present application provide an image processing control system, method, electronic device and storage medium, which can reduce the power consumption of the electronic device and increase the battery life of the electronic device.

[0005] In a first aspect, the present application provides an image processing control system, the control system comprising: an image collector, an image signal processor, a detection module, a memory manager, a power manager, and a controller; the control system comprises a sleep state, a working state, and a shutdown state, the power consumption of the sleep state is less than the power consumption of the working state; The controller is configured to: receiving a control signal, and controlling the power manager to set the control system to the working state; Controlling the image signal processor to perform automatic exposure and automatic white balance processing according to the original image data collected by the image collector to obtain image data to be detected; Controlling the memory manager to send the image data to be detected to the detection module; Controlling the detection module to perform target object recognition on the image data to be detected to obtain a recognition result; When the recognition result is a first recognition result, the power manager is controlled to set the control system from the working state to the sleep state, or to the shutdown state; the first recognition result is used to characterize that the target object does not exist in the image data to be detected.

[0006] In some embodiments, after executing controlling the power manager to set the control system to the working state, the controller is further configured to: The image collector is controlled to output raw image data at a first frame rate; the first frame rate is related to the hardware performance of the image collector and the exposure parameters sent by the image signal processor to the image collector.

[0007] In some embodiments, the controller controls the image signal processor to perform automatic exposure and automatic white balance processing according to the original image data collected by the image collector to obtain the image data to be detected, which is specifically configured as follows: Controlling the image signal processor to perform exposure analysis on the raw image data of the current frame, and adjusting exposure parameters according to the exposure analysis result; the exposure analysis result includes that the raw image data is overexposed or the raw image data is underexposed; Controlling the image signal processor to perform color temperature analysis on the original image data of the current frame, and adjusting the color temperature value according to the color temperature analysis result; the color temperature analysis result includes that the color tone of the original image data shifts toward a cold color tone, or the color tone of the original image data shifts toward a warm color tone; Controlling the image collector to collect the next frame of raw image data according to the adjusted exposure parameter and the adjusted color temperature value; The process is executed in a loop until the collected raw image data of the current frame meets the exposure requirement and the white balance requirement.

[0008] In some embodiments, the exposure parameters include at least an aperture value, a shutter speed, and a sensitivity; The controller adjusts the exposure parameters according to the exposure analysis result, and is specifically configured as follows: Controlling the image signal processor to fix the shutter speed according to the first frame rate; Based on the exposure analysis result and the shutter speed fixed at the first frame rate, the aperture value and the sensitivity are adjusted.

[0009] In some embodiments, the controller controls the detection module to perform target object recognition on the image data to be detected, and after obtaining the recognition result, the controller is further configured to: According to the second recognition result output by the detection module, controlling the image collector to collect the original image data at a second frame rate; the second frame rate is less than the first frame rate; the second recognition result is used to indicate that the target object exists in the image data to be detected; The image signal processor is controlled to optimize the original image data to obtain target image data; the target image data is at least used to record the motion state of the target object.

[0010] In some embodiments, the controller is connected to a system timer; The controller is also configured to: When the control system is in the dormant state or the closed state, controlling the system timer to start timing; Receive a control signal sent by the system timer, and control the power manager to set the control system from the sleep state or the shutdown state to the working state; when the timing duration of the system timer is greater than a preset duration, the control signal is sent.

[0011] In some embodiments, the controller is connected to a sensor; The controller is also configured to: When the control system is in the sleep state or the shutdown state, receiving a control signal input by the sensor, and controlling the power manager to set the control system from the sleep state or the shutdown state to the working state; The control signal is sent to the control system by the sensor when it detects that the environmental signal value is greater than the trigger threshold; the environmental signal value includes at least one of a sound signal value, a vibration signal value, a temperature signal value, and a light intensity signal value.

[0012] In a second aspect, the present application provides an image processing control method, which is applied to an image processing control system, wherein the control system comprises: an image collector, an image signal processor, a detection module, a memory manager, a power manager, and a controller; the control system comprises a sleep state, a working state, and a shutdown state, wherein the power consumption of the sleep state is less than the power consumption of the working state; The method comprises: receiving a control signal, and controlling the power manager to set the control system to the working state; Controlling the image signal processor to perform automatic exposure and automatic white balance processing according to the original image data collected by the image collector to obtain image data to be detected; Controlling the memory manager to send the image data to be detected to the detection module; Controlling the detection module to perform target object recognition on the image data to be detected to obtain a recognition result; According to the first recognition result output by the detection module, the power manager is controlled to set the control system from the working state to the sleep state, or to the shutdown state; the first recognition result is used to characterize that the target object does not exist in the image data to be detected.

[0013] In a third aspect, the present application provides an electronic device, comprising a processor and a memory, wherein the memory is used to store at least one program, and the at least one program is loaded by the processor and executes the image processing control method as described in the second aspect above.

[0014] In a fourth aspect, the present application provides a computer-readable storage medium, wherein at least one program is stored in the computer-readable storage medium, and the at least one program is loaded and executed by a processor to implement the image processing control method as described in the second aspect above.

[0015] It can be seen from the above technical content that the present application provides an image processing control system, method, electronic device and storage medium, which can shut down the control system or run the control system at low power consumption when it is detected that the target object does not appear in the image data to be detected. Since the RISC-V architecture has the characteristics of concise instruction set, strong customizability, support for power management technology (for example, dynamic voltage and frequency adjustment, power gating, etc.), optimized micro-architecture design (multi-transmission, out-of-order execution), etc., it can make instruction execution simpler and more efficient, retain the necessary functional modules of the control system, remove redundant parts, and dynamically adjust the voltage, frequency, etc. according to the size of the load. In combination with target object detection, when the target object does not appear in the image data to be detected, the control system is shut down, or the control system is run at low power consumption. The purpose of effectively reducing system power consumption and extending system battery life is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the embodiments are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0017] Figure 1 An image processing control system architecture diagram provided in an embodiment of the present application; Figure 2 An example diagram of a controller execution process applicable to an image processing control system provided in an embodiment of the present application; Figure 3 An example diagram of a connection between an image processing control system and an input device provided in an embodiment of the present application; Figure 4 An example diagram of a connection between an image processing control system and a sensor provided in an embodiment of the present application; Figure 5 An example diagram of a connection between an image processing control system and a system timer provided in an embodiment of the present application; Figure 6 An example diagram of a connection between an image processing control system and an external device provided in an embodiment of the present application; Figure 7 An example flowchart of an image processing control system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0018] The embodiments will be described in detail below, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation methods described in the following embodiments do not represent all implementation methods consistent with the present application. They are only examples of systems and methods consistent with some aspects of the present application.

[0019] In this application, the terms "first", "second", etc. are used to distinguish between identical or similar items with substantially the same effects and functions. It should be understood that there is no logical or temporal dependency between "first", "second", and "nth", nor is there a limitation on the quantity and execution order. It should also be understood that although the following description uses the terms first, second, etc. to describe various elements, these elements should not be limited by the terms.

[0020] These terms are only used to distinguish one element from another element. For example, without departing from the scope of various examples, a first action can be referred to as a second action, and similarly, a second action can also be referred to as a first action. Both the first action and the second action can be actions, and in some cases, can be separate and different actions.

[0021] Here, at least one means one or more than one, for example, at least one action can be one action, two actions, three actions, or any other action that is an integer greater than or equal to one. And multiple means two or more than two, for example, multiple actions can be two actions, three actions, or any other action that is an integer greater than or equal to two.

[0022] The IoT camera is an image acquisition device based on IoT technology. It is connected to the Internet through a network and can send the collected video to the user's mobile phone, computer and other terminal devices through the network.

[0023] IoT cameras are simple in design and easy to install. They usually only need to be connected to the network and power to work. In places where it is inconvenient to wire, IoT cameras can be deployed with battery power and wireless network connection.

[0024] IoT cameras can also be equipped with intelligent analysis algorithms, enabling them to have functions such as face recognition, behavior analysis, and object recognition. For example, they can determine whether the behavior of target objects such as pedestrians and vehicles is abnormal, improving detection efficiency and accuracy. Furthermore, abnormal situations can be reported to the user's terminal device through the network, allowing users to grasp abnormal situations in the first place.

[0025] Since the installation location of IoT cameras is uncertain, in some application scenarios, some IoT cameras are often installed in places where there is no power supply nearby or where the power supply cannot be connected. Therefore, these IoT cameras need to be powered by batteries.

[0026] However, battery-powered IoT cameras are limited by battery power and need to consider power consumption. In addition, most of the image data collected by IoT cameras is used for target detection. If there is no target object (for example, a person or a vehicle) in the collection area, the images collected by the IoT camera are invalid images, wasting battery power.

[0027] The embodiment of the present application provides an image processing control system, which can shut down the control system or run the control system at low power consumption when it is detected that the target object does not appear in the image data to be detected. Since the RISC-V architecture has the characteristics of concise instruction set, strong customizability, support for power management technology (for example, dynamic voltage and frequency adjustment, power gating, etc.), optimized micro-architecture design (multi-transmission, out-of-order execution), etc., it can make instruction execution simpler and more efficient, retain the necessary functional modules of the control system, remove redundant parts, and dynamically adjust the voltage and frequency according to the size of the load. In combination with target object detection, when the target object does not appear in the image data to be detected, the control system is shut down, or the control system is run at low power consumption. The purpose of effectively reducing system power consumption and extending system battery life is achieved.

[0028] like Figure 1 As shown, an architecture diagram of an image processing control system provided by an embodiment of the present application is introduced. The control system includes: an image collector 100, an image signal processor 200, a detection module 300, a memory manager 400, a power manager 500 and a controller 600. The controller 600 is connected to the image collector 100, the image signal processor 200, the detection module 300, the memory manager 400, and the power manager 500 through a control bus.

[0029] The controller 600 is used to execute control signals, coordinate data transmission and storage between components within the system, etc. The internal components include, for example, an image collector 100, an image signal processor 200, a detection module 300, a memory manager 400, a power manager 500, and the like.

[0030] The controller 600 may also communicate with external devices such as a display, a sensor, a keyboard, a mouse, etc. through various interfaces and protocols to receive control signals sent by external devices and transmit image data to external devices.

[0031] The image collector 100 can be, for example, an Internet camera (hereinafter referred to as the camera). The camera may include a lens assembly and a photosensitive element. The lens assembly is provided with a photosensitive element and a lens. The lens can refract light through multiple lenses so that the light is irradiated on the photosensitive element. The photosensitive element can be made of a charge-coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) material. The light signal captured by the photosensitive element is converted into an electrical signal, and then the electrical signal is converted into raw video data and output.

[0032] The image signal processor 200 is mainly responsible for providing data preprocessing (for example, de-mosaicing, noise suppression, bad pixel removal, color correction, etc.), image enhancement and optimization (for example, gamma correction, edge enhancement, lens distortion correction, etc.), automatic control functions (for example, automatic exposure, automatic white balance, automatic focus), and image compression, format conversion and other functions.

[0033] The detection module 300 is mainly responsible for target object recognition, and identifies whether there is a target object in the image data to be detected by deploying a target detection algorithm such as a convolutional neural network and a recurrent neural network, and outputs a recognition result. Among them, the recognition result includes a first recognition result and a second recognition result, the first recognition result is used to characterize that there is no target object in the image data to be detected, and the second recognition result is used to characterize that there is a target object in the image data to be detected.

[0034] The memory manager 400 is mainly responsible for memory allocation and recovery, memory protection, memory mapping, etc. In the embodiment of the present application, the memory manager 400 is used to store image data to be detected, software programs of various components, etc. These software programs implement various functions by calling the controller 600.

[0035] The power manager 500 is mainly responsible for power distribution and control, power state detection, energy saving management, etc. In the embodiment of the present application, the power manager 500 is used to manage the operating state of the entire control system. The control system may include, for example, a sleep state, a working state, and a shutdown state, wherein the power consumption of the sleep state is less than the power consumption of the working state.

[0036] In some embodiments, the dormant state means that the power manager 500 stops supplying power to other internal components of the control system except the controller 600, and reduces the clock frequency of the controller 600, so that the operation speed of the controller 600 slows down, thereby achieving the purpose of reducing power consumption. The working state means that the power manager 500 supplies power to all internal components. The shutdown state means that the power manager 500 stops supplying power to all internal components including the controller 600.

[0037] An image processing control system provided in an embodiment of the present application can be applied to smart home, regional security, automotive electronics and other fields.

[0038] For example, in the field of smart home, the control system can be deployed in smart door locks. Since smart door locks are set on the door and it is difficult to wire, they are generally powered by batteries. The control system can periodically collect images of the door area, or enter the working state to collect images of the door area after receiving signals such as vibration and sound. If the image of the door area is identified to contain a target object (for example, a person, an animal, or other moving object), the control system continues to work and collect images of the door area. If the image of the door area does not contain a target object, the control system enters a dormant state or a closed state, reducing the overall power consumption of the smart door lock and increasing the battery life.

[0039] For another example, in the field of regional security, the control system can be deployed in a security system as part of the security system, which also includes a display device, a sensor, and other input devices. The display device has a display function and a storage function, and can display and store image data transmitted by the control system. The input device and the sensor can input control signals to the control system. The control system can periodically collect images of the target area, or enter a working state to collect images of the target area after receiving signals such as vibration and sound. If the image of the target area is identified to have a target object, the control system continues to maintain a working state and continues to collect images of the door area. If the image of the door area does not have a target object, the control system enters a dormant state or a closed state to reduce the power consumption of the control system, thereby enabling the entire security system to work for a longer time.

[0040] For another example, in the field of automotive electronics, some smart cars often have a "sentinel mode". "Sentinel mode" means that after the vehicle is powered off and locked, the camera deployed on the outside of the car continues to remain on and continuously collect images around the car. When the vehicle with this control system turns on the "sentinel mode", it can periodically collect images of the area around the car, or enter the working state to collect images of the area around the car after receiving signals such as vibration and sound, and identify whether there is a target object in the image of the area around the car. When it is detected that there is no target object in the image outside the car, the control system enters a dormant state or a closed state, and the camera stops collecting images. When the next collection cycle arrives, or when a signal such as vibration or sound is received again, the control system enters a working state, and the camera starts working again. In this way, the operating time of the camera can be reduced, the overall power consumption of the vehicle battery can be reduced, and the operating time of the "sentinel mode" can be increased.

[0041] It can be known that the application scenarios introduced in the above embodiments are only some of the scenarios in which the image processing control system provided in the embodiments of the present application can be applied. The control system can also be applied in other scenarios that require reducing camera power consumption, which will not be introduced one by one here.

[0042] The following is a detailed introduction to the execution process of an image processing control system provided in an embodiment of the present application.

[0043] like Figure 2 As shown in the example diagram of a controller execution process, the controller 600 is configured to perform the following steps: S201 , receiving a control signal, and controlling the power manager 500 to set the control system to a working state.

[0044] The control signal may be generated by an input device, a sensor or a system timer.

[0045] In some embodiments, Figure 3 As shown in the diagram, an example of a connection between an image processing control system and an input device is shown. The controller 600 can communicate with the input device 700 through an interface and a protocol. The input device 700 can be a mouse, a keyboard, or a touch screen.

[0046] The user can use these input devices 700 to input control signals, and the control signals are transmitted to the controller 600 through the interface. After receiving the control signals, the controller 600 controls the power manager 500 to increase the voltage and frequency, and supplies power to the image collector 100, the image signal processor 200, the detection module 300, and the memory manager 400, so that the control system is in working state.

[0047] In some embodiments, Figure 4 As shown in the example diagram of a connection between a control system and a sensor, the controller 600 can communicate with the sensor 800 through an interface and a protocol. The sensor 800 is responsible for detecting the value of an environmental signal and sending a control signal to the controller 600 if the value of the environmental signal is greater than a trigger threshold.

[0048] According to different detection principles, the sensor 800 may be, for example, a vibration sensor, an acoustic sensor, a temperature sensor, a photosensitive sensor, etc. Different sensors 800 are responsible for detecting different types of environmental signal values. For example, the vibration sensor is responsible for detecting the vibration signal value, the acoustic sensor is responsible for detecting the sound signal value, the temperature sensor is responsible for detecting the temperature signal value, and the photosensitive sensor is responsible for detecting the light intensity signal value. It can be known that the trigger thresholds corresponding to different types of environmental signal values ​​are also different, and can be set according to actual conditions.

[0049] In some embodiments, the controller 600 is further configured to: when the control system is in a dormant state or in a closed state, receive a control signal input by the sensor 800 to set the control system from the dormant state or the closed state to a working state; For example, taking a smart door lock equipped with an image processing control system provided by an embodiment of the present application as an example, the smart door lock is also equipped with a vibration sensor. When a person or other object touches the smart door lock, the vibration sensor will detect a vibration signal. If the vibration signal value is greater than the trigger threshold, the vibration sensor will send a control signal to the controller 600. After receiving the control signal, the controller 600 controls the power manager 500 to set the control system to a working state. At this time, the image collector 100 starts to work and collects images of the door area.

[0050] In some embodiments, Figure 5 As shown in the example diagram of a connection between a control system and a system timer, the controller 600 can communicate with the system timer 900 through an interface and a protocol. When the timing duration is greater than a preset duration, the system timer 900 sends a control signal to the controller 600.

[0051] In some embodiments, the controller 600 is also configured to: start the control system timer 900 when the control system is in a sleep state or in a shutdown state; receive a control signal sent by the system timer 900, and control the power manager 500 to set the control system from a sleep state or a shutdown state to a working state.

[0052] Among them, the preset duration of the system timer 900 can be set according to actual needs. For example, different preset durations can be set according to different focuses. If the focus is on avoiding missing the recognition of the target object, a shorter preset duration can be set, such as 1 second. That is to say, when the control system can be in a dormant state or in a closed state, the system timer 900 starts timing, and when the timing reaches 1 second, a control signal is sent to the controller 600, and the controller 600 controls the power manager 500 to set the control system to a working state. If the focus is on reducing power consumption, a longer preset duration can be set, such as 5 seconds.

[0053] In addition, the preset duration of the system timer 900 can also be dynamically adjusted according to the battery power. For example, when the remaining battery power is 80% of the total power, the preset duration of the system timer 900 can be set to 1 second; when the remaining battery power is 50% of the total power, the preset duration of the system timer 900 can be set to 3 seconds; when the remaining battery power is 20% of the total power, the preset duration of the system timer 900 can be set to 5 seconds.

[0054] In some embodiments, the preset duration of the system timer 900 can be set according to actual needs or dynamically adjusted according to the battery power. For example, the preset duration of the system timer 900 is set to 3 seconds according to actual needs, but when the remaining battery power is 20% of the total power, the preset duration of the system timer 900 can be changed to 5 seconds to ensure the battery life of the system. In addition, there may be other possible combinations that will not be described one by one here.

[0055] In some embodiments, Figure 6 As shown in the example diagram of a connection between a control system and external devices, the controller 600 can communicate with external devices such as an input device 700, a sensor 800, a system timer 900, etc. respectively. After receiving a control signal sent by any external device such as the input device 700, the sensor 800, the system timer 900, etc., the control power manager 500 sets the control system to a working state.

[0056] S202: Control the image collector 100 to output raw image data at a first frame rate.

[0057] The first frame rate is related to the hardware performance of the image collector and the exposure parameters sent by the image signal processor 200 to the image collector 100 .

[0058] In some embodiments, the hardware performance of the image collector determines the maximum number of frames of raw image data that the image collector can collect per second. Depending on the hardware performance of the image collector, the maximum number of frames of raw image data that the image collector can collect per second is generally between 30FPS-230FPS. Exposure parameters include: aperture value, shutter speed, sensitivity, etc. The shutter speed in the exposure parameters determines the number of frames of raw image data output by the image collector per second, that is, the first frame rate. The first frame rate cannot exceed the maximum number of frames of raw image data that the image collector 100 can collect per second.

[0059] In some embodiments, the shutter speed can be set according to the maximum number of frames of raw image data that the image collector 100 can collect per second. For example, if the maximum number of frames of raw image data that the image collector 100 can collect per second is 60 FPS (frames per second), the shutter speed can be set to one-sixtieth of a second or faster, and the aperture value and the sensitivity are set to an initial value on this basis. After the image signal processor 200 sends the exposure parameter to the image collector 100, the image collector 100 outputs the raw image data according to the first frame rate indicated by the exposure parameter.

[0060] By setting the shutter speed according to the maximum number of frames of raw image data that the image collector 100 can collect per second, the image collector 100 can be controlled to output the raw image data in the shortest time. Since the image signal processor 200 needs multiple frames of raw image data for automatic exposure convergence and automatic white balance convergence, outputting the raw image data in the shortest time can accelerate the image signal processor 200 to perform automatic exposure convergence and automatic white balance convergence. In this way, the image collector 100 outputs the image to be detected that meets the exposure requirements and white balance requirements in the fastest time. Then, after identifying that there is no target object in the image to be detected, the controller 600 controls the power manager 500 to set the control system to a dormant state or a closed state. In this way, when there is no target object in the image to be detected, the time that the control system is in a working state can be shortened as much as possible, and the power consumption of the control system can be effectively reduced.

[0061] S203 , controlling the image signal processor 200 to perform automatic exposure and automatic white balance processing according to the original image data collected by the image collector 100 to obtain image data to be detected.

[0062] Among them, the image signal processor 200 integrates multiple algorithms, can analyze the exposure of the picture according to the original image data to obtain the exposure analysis result, and adjust the exposure parameters according to the exposure analysis effect. The image signal processor 200 sends the adjusted exposure parameters to the image collector 100, and the image collector 100 outputs the original image data according to the adjusted exposure parameters. The image signal processor 200 continues to analyze the original image data and adjust the exposure parameters until the original image data output by the image collector 100 meets the exposure requirements.

[0063] In some embodiments, satisfying the exposure requirement includes, for example, the brightness of the image being between 30 dB and 60 dB.

[0064] The exposure analysis results include overexposure or underexposure. If the image is overexposed, you can adjust the exposure parameters by reducing the aperture value to reduce the light entering the photosensitive element per unit time, or by reducing the sensitivity to reduce the sensitivity of the photosensitive element to light, or by reducing the shutter speed to reduce the time the photosensitive element receives light.

[0065] If the picture is underexposed, you can adjust the exposure parameters by increasing the aperture value, increasing the sensitivity, or increasing the shutter speed.

[0066] It is known that adjusting any one or more of the aperture value, sensitivity, and shutter speed can adjust the exposure parameters, thereby affecting the exposure analysis results. The adjustment of one or more of these three values ​​is determined according to actual needs.

[0067] The image signal processor 200 can also analyze the ambient light according to the original image data to obtain a color temperature analysis result, adjust the color temperature value according to the color temperature analysis result, and send the adjusted color temperature value to the image collector 100. The image collector 100 outputs the original image data according to the adjusted color temperature value. The image signal processor 200 continues to analyze the scene light according to the original image data to obtain a color temperature analysis result until the original image data output by the image collector 100 meets the white balance requirements.

[0068] The color temperature analysis result includes the color temperature of the original image data shifting toward a cool color tone, or the color temperature of the original image data shifting toward a warm color tone. In some embodiments, there are different threshold intervals according to different scenes of ambient light. If it is in a sunny daylight scene, the normal color temperature value is between 5000K-6000K (Kelvin). If the color temperature value of the original image data is greater than 6000K, it means that the color temperature of the original image data shifts toward a warm color tone. If the color temperature value of the original image is less than 5000K, it means that the color temperature of the original image data shifts toward a cool color tone. In a fluorescent light scene, the normal color temperature value is between 3500K-4500K. If the color temperature value of the original image data is greater than 4500K, it means that the color temperature of the original image data shifts toward a warm color tone. If the color temperature value of the original image data is less than 3500K, it means that the color temperature of the original image shifts toward a cool color tone.

[0069] In some embodiments, the image data to be detected is characterized as raw image data that meets exposure requirements and white balance requirements.

[0070] Next, a controller 600 controls the image signal processor 200 to perform automatic exposure and automatic white balance processing on the original image data collected by the image collector 100 to obtain the execution process of the image data to be detected: In some embodiments, the controller 600 controls the image signal processor 200 to perform the following processing on the current frame raw image data acquired by the image acquirer 100: The image signal processor 200 is controlled to perform exposure analysis on the raw image data of the current frame, and the exposure parameters are adjusted according to the exposure analysis result.

[0071] The image signal processor 200 is controlled to perform color temperature analysis on the original image data of the current frame, and the color temperature value is adjusted according to the color temperature analysis result; the color temperature analysis result includes the color temperature of the original image data shifting toward a cold tone, or the color temperature of the original image data shifting toward a warm tone.

[0072] The image collector 100 is controlled to collect the next frame of raw image data according to the adjusted exposure parameter and the adjusted color temperature value.

[0073] The loop is executed until the collected raw image data of the current frame meets the exposure requirements and white balance requirements.

[0074] In some embodiments, when adjusting the exposure parameters, the shutter speed can be set according to the maximum number of frames of raw image data that the image collector 100 can collect per second. For example, if the hardware performance of the image collector 100 supports a maximum of 60 FPS, the shutter speed is fixed at 1 / 60 second or faster, and then the aperture and sensitivity are adjusted. When adjusting the exposure parameters later, the shutter speed is guaranteed to be no less than 1 / 60 second. In this way, the image collector 100 can always output the raw image data at the fastest speed, and the image signal processor 200 can complete the automatic exposure convergence and automatic white balance convergence in the fastest time.

[0075] S204 , controlling the memory manager 400 to send the image data to be detected to the detection module 300 .

[0076] In some embodiments, the image data to be detected is stored in a memory, and the controller 600 controls the memory manager 400 to retrieve the image data to be detected from the memory and send the image data to be detected to the detection module 300 .

[0077] S205 , controlling the detection module 300 to perform target object recognition on the image data to be detected, and obtaining a recognition result.

[0078] In some embodiments, the detection module 300 may be deployed with at least one target object detection model, which can identify whether a target object exists in the image to be detected and output a recognition result.

[0079] The recognition result output by the detection module 300 includes a first recognition result and a second recognition result. The first recognition result is used to indicate that the target object does not exist in the image data to be detected, and the second recognition result is used to indicate that the target object exists in the image data to be detected.

[0080] S206 . According to the first recognition result output by the detection module 300 , the power manager 500 is controlled to set the control system from the working state to the sleeping state, or to the shutdown state.

[0081] Since the detection module 300 outputs the first recognition result indicating that there is no target object in the image to be detected, if the control system continues to maintain the working state at this time, unnecessary power consumption will be consumed. Therefore, after the detection module 300 outputs the first recognition result, controlling the power manager 500 to set the control system from the working state to the sleep state, or to the shutdown state can effectively reduce power consumption.

[0082] In some embodiments, the controller 600 executes control of the detection module 300 to perform target object recognition on the image data to be detected. After obtaining the recognition result, the controller 600 is also configured to: control the image acquisition device 100 to collect original image data at a second frame rate according to the second recognition result output by the detection module 300; control the image signal processor 200 to optimize the original image data to obtain the target image data.

[0083] The second frame rate is 24FPS or 30FPS, which can ensure that the video picture composed of the target image data is played smoothly while reducing power consumption as much as possible.

[0084] In some embodiments, the optimization processing may include, for example, gamma correction, edge enhancement, lens distortion correction, etc., and one or more of them are selected according to actual needs to process the original image data to obtain the target image data.

[0085] like Figure 7 As shown, an example flow chart of an image processing control system provided by an embodiment of the present application is introduced. The control system includes a controller 600 configured to execute the following process: The controller 600 receives the control signal; controls the power manager 500 to set the control system to the working mode; controls the image collector 100 to start; after starting, the image collector 100 starts to output the original image data according to the exposure parameters sent by the image signal processor 200.

[0086] The image signal processor 200 is controlled to perform automatic exposure processing and automatic white balance processing according to the original image data output by the image collector 100; after the original image data output by the image collector 100 meets the exposure requirements and white balance requirements, the controller 600 controls the memory manager 400 to send the image data to be detected to the detection module 300.

[0087] The control detection module 300 identifies whether there is a target object in the image to be detected. If there is a target object, the power manager 500 is controlled to set the control system to sleep mode or shut down the control system; if there is no target object, the working mode is maintained.

[0088] The embodiment of the present application also provides an image processing control method, which is applied to an image processing control system. The control system includes: an image collector 100, an image signal processor 200, a detection module 300, a memory manager 400, a power manager 500, and a controller 600; the control system includes a sleep state, a working state, and a shutdown state, and the power consumption of the sleep state is less than the power consumption of the working state; the image processing control method includes: receiving a control signal, and controlling the power manager 500 to set the control system to a working state; Controlling the image signal processor 200 to perform automatic exposure and automatic white balance processing according to the original image data collected by the image collector 100 to obtain image data to be detected; Controlling the memory manager 400 to send the image data to be detected to the detection module 300; Control the detection module 300 to perform target object recognition on the image data to be detected to obtain a recognition result; According to the first recognition result output by the detection module 300 , the power manager 500 is controlled to set the control system from the working state to the sleeping state, or to the shutdown state.

[0089] It should be noted that the image processing control system and the image processing control method provided in the above embodiments belong to the same concept, and their specific implementation process is detailed in the device embodiment, which will not be repeated here.

[0090] The embodiment of the present application provides an image processing control method that can shut down the control system or run the control system at low power consumption when it is detected that the target object does not appear in the image data to be detected. Since the RISC-V architecture has the characteristics of concise instruction set, strong customizability, support for power management technology (for example, dynamic voltage and frequency adjustment, power gating, etc.), optimized micro-architecture design (multi-transmission, out-of-order execution), etc., it can make instruction execution simpler and more efficient, retain the necessary functional modules of the control system, remove redundant parts, and dynamically adjust the voltage, frequency, etc. according to the size of the load. In combination with target object detection, when the target object does not appear in the image data to be detected, the control system is shut down, or the control system is run at low power consumption. The purpose of effectively reducing system power consumption and extending system battery life is achieved.

[0091] The embodiment of the present application also provides a security system, which includes an image processing-based control system and external devices, the external devices include: an input device 700, a sensor 800, and a system timer 900; the control system includes: an image collector 100, an image signal processor 200, a detection module 300, a memory manager 400, a power manager 500, and a controller 600; the control system includes a sleep state, a working state, and a shutdown state, and the power consumption of the sleep state is less than the power consumption of the working state; When an external device inputs a control signal to the control system, the controller 600 is configured as follows: Control the power manager 500 to set the control system to a working state; Controlling the image signal processor 200 to perform automatic exposure and automatic white balance processing on the original image data collected by the image collector 100 to obtain image data to be detected; Control the memory manager 400 to send the image data to be detected to the detection module 300; Control the detection module 300 to perform target object recognition on the image data to be detected, and obtain a recognition result; According to the first recognition result output by the detection module 300, the power manager 500 is controlled to set the control system from the working state to the sleeping state, or to the shutdown state; the first recognition result is used to indicate that there is no target object in the image data to be detected.

[0092] The present application also provides an electronic device, which includes: a processor, and a memory communicatively connected to the processor; wherein the memory stores instructions executable by the processor so that the processor executes the steps of the method as in any one of the above embodiments.

[0093] In some embodiments, the processor may include one or more processing cores, such as a 4-core processor, a 4-core processor, and the like. The processor may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor may also include a main processor and a coprocessor. The main processor is a processor for processing data in an awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0094] The memory may include one or more computer-readable storage media, which may be non-transitory. The memory may also include high-speed random access memory, and non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory is used to store at least one program code, which is used to be executed by the processor to implement the process executed by the terminal in the method for data format conversion provided in the method embodiment of the present application.

[0095] The embodiment of the present application also provides a computer-readable storage medium, the computer-readable storage medium includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the above method for data format conversion. In some embodiments, the computer-readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a compact disc (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, etc.

[0096] Similar parts between the embodiments provided in this application can be referenced to each other. The specific implementation methods provided above are only a few examples under the general concept of this application and do not constitute a limitation on the protection scope of this application. For those skilled in the art, any other implementation methods expanded based on the scheme of this application without creative work belong to the protection scope of this application.

Claims

1. An image processing control system, characterized in that: The control system includes: an image collector, an image signal processor, a detection module, a memory manager, a power manager, and a controller; the control system includes a sleep state, a working state, and a shutdown state, and the power consumption of the sleep state is less than the power consumption of the working state; The controller is configured to: receiving a control signal, and controlling the power manager to set the control system to the working state; Controlling the image signal processor to perform automatic exposure and automatic white balance processing according to the original image data collected by the image collector to obtain image data to be detected; Controlling the memory manager to send the image data to be detected to the detection module; Controlling the detection module to perform target object recognition on the image data to be detected to obtain a recognition result; When the recognition result is a first recognition result, the power manager is controlled to set the control system from the working state to the sleep state, or to the shutdown state; the first recognition result is used to characterize that the target object does not exist in the image data to be detected.

2. The image processing control system according to claim 1, characterized in that: After executing the control of the power manager to set the control system to the working state, the controller is further configured to: The image collector is controlled to output raw image data at a first frame rate; the first frame rate is related to the hardware performance of the image collector and the exposure parameters sent by the image signal processor to the image collector.

3. The image processing control system according to claim 2, characterized in that: The image data to be detected is original image data that meets the exposure requirements and white balance requirements; The controller controls the image signal processor to perform automatic exposure and automatic white balance processing according to the original image data collected by the image collector to obtain the image data to be detected, and is specifically configured as follows: Controlling the image signal processor to perform exposure analysis on the raw image data of the current frame, and adjusting the exposure parameters according to the exposure analysis result; the exposure analysis result includes that the raw image data is overexposed or the raw image data is underexposed; Controlling the image signal processor to perform color temperature analysis on the original image data of the current frame, and adjusting the color temperature value according to the color temperature analysis result; the color temperature analysis result includes that the color tone of the original image data shifts toward a cold color tone, or that the color tone of the original image data shifts toward a warm color tone; Controlling the image collector to collect the next frame of raw image data according to the adjusted exposure parameter and the adjusted color temperature value; The process is executed in a loop until the collected raw image data of the current frame meets the exposure requirement and the white balance requirement.

4. The image processing control system according to claim 3, characterized in that: The exposure parameters include at least aperture value, shutter speed, and sensitivity; The controller adjusts the exposure parameters according to the exposure analysis result, and is specifically configured as follows: Controlling the image signal processor to fix the shutter speed according to the first frame rate; Based on the exposure analysis result and the shutter speed fixed at the first frame rate, the aperture value and the sensitivity are adjusted.

5. The image processing control system according to any one of claims 1 to 4, characterized in that: The controller controls the detection module to perform target object recognition on the image data to be detected, and after obtaining the recognition result, the controller is further configured to: According to the second recognition result output by the detection module, controlling the image collector to collect the original image data at a second frame rate; the second frame rate is less than the first frame rate; the second recognition result is used to indicate that the target object exists in the image data to be detected; The image signal processor is controlled to optimize the original image data to obtain target image data; the target image data is at least used to record the motion state of the target object.

6. The image processing control system according to claim 1, characterized in that: The controller is connected to a system timer; The controller is also configured to: When the control system is in the dormant state or the closed state, controlling the system timer to start timing; Receive a control signal sent by the system timer, and control the power manager to set the control system from the sleep state or the shutdown state to the working state; when the timing duration of the system timer is greater than a preset duration, the system timer sends a control signal.

7. The image processing control system according to claim 1, characterized in that: The controller is connected to the sensor; The controller is also configured to: When the control system is in the sleep state or the shutdown state, receiving a control signal input by the sensor, and controlling the power manager to set the control system from the sleep state or the shutdown state to the working state; The control signal is sent to the control system by the sensor when it detects that the environmental signal value is greater than the trigger threshold; the environmental signal value includes at least one of a sound signal value, a vibration signal value, a temperature signal value, and a light intensity signal value.

8. An image processing control method, characterized in that: Applied to an image processing control system, the image processing control system comprises: an image collector, an image signal processor, a detection module, a memory manager, a power manager, and a controller; the image processing control system comprises a sleep state, a working state, and a shutdown state, and the power consumption of the sleep state is less than the power consumption of the working state; The image processing control method comprises: receiving a control signal, and controlling the power manager to set the control system to the working state; Controlling the image signal processor to perform automatic exposure and automatic white balance processing according to the original image data collected by the image collector to obtain image data to be detected; Controlling the memory manager to send the image data to be detected to the detection module; Controlling the detection module to perform target object recognition on the image data to be detected to obtain a recognition result; According to the first recognition result output by the detection module, the power manager is controlled to set the control system from the working state to the sleep state, or to the shutdown state; the first recognition result is used to characterize that the target object does not exist in the image data to be detected.

9. An electronic device, characterized in that: The electronic device comprises a processor and a memory, wherein the memory is used to store at least one program, and the at least one program is loaded by the processor and executes the image processing control method as claimed in claim 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one program, and the at least one program is loaded and executed by the processor to implement the image processing control method according to claim 8.

Images