Camera equipment control method and device, equipment and storage medium

Abstract

The embodiment of the invention provides a camera equipment control method and device, equipment and a storage medium, applied to camera equipment, the camera equipment comprises a controller and an image sensor, the image sensor is electrically connected with the controller, the controller is electrically connected with a brightness sensor used for detecting ambient brightness, and the brightness sensor is electrically connected with the controller. The camera equipment control method comprises the following steps: continuously acquiring the detection voltage of the brightness sensor, and determining the voltage variation between the currently acquired detection voltage and the previously acquired detection voltage; whenever the voltage variation is greater than or equal to a variation threshold and the currently acquired detection voltage is within a preset adjustment range, determining target exposure time and target gain based on the currently acquired detection voltage; and configuring the target exposure time and the target gain to the image sensor. According to the embodiment of the invention, the control response time of the image sensor can be shortened.

Description

Technical Field

[0001] The present application relates to but is not limited to the field of camera technology, and in particular to a camera device control method, device, equipment and storage medium. Background Art

[0002] At present, in order to ensure image brightness and some image details in night scenes, cameras usually rely on ISP image signal processors (ISP) to control image sensors. However, the automatic exposure algorithm used by the image signal processor is highly complex, which leads to a long control response time of the image sensor, resulting in long-term overexposure in bright scenes and slow darkening in dark scenes. Summary of the invention

[0003] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0004] The embodiments of the present application provide a camera device control method, apparatus, device and storage medium, which can shorten the control response time of an image sensor.

[0005] To achieve the above-mentioned purpose, the first aspect of an embodiment of the present application proposes a camera device control method, which is applied to a camera device, wherein the camera device includes a controller and an image sensor, the image sensor is electrically connected to the controller, and the controller is electrically connected to a brightness sensor for detecting ambient brightness. The camera device control method includes: continuously acquiring a detection voltage of the brightness sensor, and determining a voltage change between a currently acquired detection voltage and a previously acquired detection voltage; whenever the voltage change is greater than or equal to a change threshold, and the currently acquired detection voltage is within a preset adjustment range, determining a target exposure time and a target gain based on the currently acquired detection voltage; and configuring the target exposure time and the target gain to the image sensor.

[0006] In some embodiments, determining the target exposure time and the target gain based on the currently acquired detection voltage includes: constructing a plurality of candidate parameter models, wherein each of the candidate parameter models includes a first sub-model for determining the exposure time and a second sub-model for determining the gain; determining a target parameter model from the plurality of candidate parameter models based on the currently acquired detection voltage; inputting the currently acquired detection voltage into the first sub-model in the target parameter model for prediction to obtain the target exposure time; and inputting the currently acquired detection voltage into the second sub-model in the target parameter model for prediction to obtain the target gain.

[0007] In some embodiments, each of the candidate parameter models is configured with a corresponding reference voltage range, each of the reference voltage ranges is continuous and non-overlapping, each of the reference voltage ranges is within the adjustment range, and the ambient brightness indicated by the voltage lower limit of each of the reference voltage ranges is in an exponentially decreasing relationship, and the target parameter model is determined in multiple candidate parameter models based on the currently acquired detection voltage, including: determining a target voltage range in each of the reference voltage ranges based on the currently acquired detection voltage; and determining the candidate parameter model corresponding to the target voltage range as the target parameter model among multiple candidate parameter models.

[0008] In some embodiments, each of the first sub-models is configured with an upper limit and a lower limit of exposure time, the output of the first sub-model is negatively correlated with the input and changes linearly, the output of the first sub-model is negatively correlated with the input of the first sub-model and changes linearly, the output change amplitude of the first sub-model increases with the decrease of the voltage lower limit of the target voltage range of the candidate parameter model, and the detection voltage currently obtained is input into the first sub-model in the target parameter model for prediction to obtain the target exposure time, including: inputting the detection voltage currently obtained into the first sub-model in the target parameter model, and interpolating and predicting the exposure time corresponding to the detection voltage according to the exposure time upper limit, the exposure time lower limit, the voltage lower limit of the target voltage range, and the voltage upper limit of the target voltage range configured by the first sub-model to obtain the target exposure time.

[0009] In some embodiments, each of the second sub-models is configured with a gain upper limit and a gain lower limit, the output of the second sub-model is negatively correlated with the input and changes linearly, the output of the second sub-model is negatively correlated with the input of the second sub-model and changes linearly, the output change amplitude of the second sub-model increases with the decrease of the voltage lower limit of the target voltage range of the candidate parameter model, and the target gain is obtained by inputting the currently acquired detection voltage into the second sub-model in the target parameter model for prediction, including: inputting the currently acquired detection voltage into the second sub-model in the target parameter model, and interpolating and predicting the gain corresponding to the detection voltage according to the gain upper limit, the gain lower limit, the voltage lower limit of the target voltage range, and the voltage upper limit of the target voltage range configured by the second sub-model to obtain the target gain.

[0010] In some embodiments, the camera device also includes a multispectral sensor, which is electrically connected to the controller. The target exposure time and the target gain are configured to the image sensor, including: acquiring a test image captured by the image sensor, and spectral data captured by the multispectral sensor when capturing the test image; extracting image features of the test image, extracting spectral features of the spectral data, splicing the image features and the spectral features, and inputting them into a light source type recognition model for recognition, and determining a target light source type from a plurality of candidate light source types based on the recognition result, wherein each of the candidate light source types is configured with a first weight for adjusting the exposure time and a second weight for adjusting the gain; weighting the target exposure time based on the first weight configured for the target light source type to obtain a weighted exposure time; weighting the target gain based on the second weight configured for the target light source type to obtain a weighted gain; and configuring the weighted exposure time and the weighted gain to the image sensor.

[0011] To achieve the above-mentioned purpose, the second aspect of an embodiment of the present application proposes a camera device, comprising: an image sensor; a controller, the image sensor is electrically connected to the controller, the controller is electrically connected to a brightness sensor for detecting ambient brightness, and the controller is used to execute the camera device control method described in the first aspect above.

[0012] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application proposes a camera device control apparatus, which is applied to a camera device, wherein the camera device includes a controller and an image sensor, the image sensor is electrically connected to the controller, and the controller is electrically connected to a brightness sensor for detecting ambient brightness. The camera device control method includes: an acquisition module, which is used to continuously acquire a detection voltage of the brightness sensor, and determine a voltage change between the currently acquired detection voltage and the previously acquired detection voltage; an operation module, which is used to determine a target exposure time and a target gain based on the currently acquired detection voltage whenever the voltage change is greater than or equal to a change threshold and the currently acquired detection voltage is within a preset adjustment range; and a configuration module, which is used to configure the target exposure time and the target gain to the image sensor.

[0013] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, the memory stores a computer program, and the processor implements the camera device control method described in the first aspect when executing the computer program.

[0014] To achieve the above-mentioned purpose, the fifth aspect of an embodiment of the present application proposes a storage medium, which is a computer-readable storage medium, and the storage medium stores a computer program. When the computer program is executed by a processor, it implements the camera device control method described in the first aspect above.

[0015] The embodiments of the present application include at least the following beneficial effects: by continuously acquiring the detection voltage of the brightness sensor, determining the voltage change between the currently acquired detection voltage and the previously acquired detection voltage, and then whenever the voltage change is greater than or equal to the change threshold, and the currently acquired detection voltage is within a preset adjustment range, determining the target exposure time and target gain based on the currently acquired detection voltage, and then configuring the target exposure time and target gain to the image sensor, compared with the automatic exposure algorithm that requires repeated closed-loop calculations, the target exposure time and target gain can be quickly determined, thereby shortening the control response time of the image sensor; in addition, the brightness changes of the external environment are accurately reflected by detecting the voltage change, thereby optimizing the target exposure time and target gain, and thus improving the image quality of the camera device.

[0016] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings are used to provide further understanding of the technical solution of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the technical solution of the present application and do not constitute a limitation on the technical solution of the present application.

[0018] Figure 1 A schematic diagram of an optional flow chart of a camera device control method provided in an embodiment of the present application;

[0019] Figure 2 An optional flowchart of target prediction provided in an embodiment of the present application;

[0020] Figure 3 A schematic diagram of an optional process for determining a target parameter model provided in an embodiment of the present application;

[0021] Figure 4 A schematic diagram of an optional flow chart for determining a target exposure time provided in an embodiment of the present application;

[0022] Figure 5 A schematic diagram of an optional process for determining a target gain provided in an embodiment of the present application;

[0023] Figure 6 An optional flow chart of weighted optimization provided in an embodiment of the present application;

[0024] Figure 7 A schematic diagram of an optional process for configuring an image sensor provided in an embodiment of the present application;

[0025] Figure 8 A schematic diagram of an optional structure of a camera device provided in an embodiment of the present application;

[0026] Fig. 9 A schematic diagram of an optional structure of a camera device control device provided in an embodiment of the present application;

[0027] Fig.10 A schematic diagram of an optional hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0029] It should be noted that in each specific embodiment of the present application, when it comes to the need to perform relevant processing based on data related to the characteristics of the target object such as target object attribute information or attribute information set, the permission or consent of the target object will be obtained first, and the collection, use and processing of these data will comply with relevant laws, regulations and standards. Among them, the target object can be a user. In addition, when the embodiment of the present application needs to obtain the target object attribute information, the target object's separate permission or separate consent will be obtained by means of a pop-up window or jumping to a confirmation page, etc., and after the separate permission or separate consent of the target object is clearly obtained, the necessary target object-related data for enabling the normal operation of the embodiment of the present application will be obtained.

[0030] In the description of this application, “several” means one or more, “more” means more than two, “greater than”, “less than”, “exceed”, etc. are understood to exclude the number, and “above”, “below”, “within”, etc. are understood to include the number.

[0031] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims or the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0032] To facilitate understanding of the technical solutions provided in the embodiments of the present application, some key terms used in the embodiments of the present application are explained here:

[0033] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0034] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. Basic artificial intelligence technologies generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operating / interactive systems, mechatronics and other technologies. Artificial intelligence software technologies mainly include computer vision technology, speech processing technology, natural language processing technology, as well as machine learning / deep learning, autonomous driving, smart transportation and other major directions.

[0035] Machine Learning (ML) is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and teaching learning.

[0036] At present, in order to ensure image brightness and some image details in night scenes, cameras usually rely on ISP image signal processors (ISP) to control image sensors. However, the automatic exposure algorithm used by the image signal processor is highly complex, which leads to a long control response time of the image sensor, resulting in long-term overexposure in bright scenes and slow darkening in dark scenes.

[0037] In response to the problem of long control response time of an image sensor, the present application provides a camera device control method, apparatus, equipment and storage medium, which are applied to a camera device, wherein the camera device includes a controller and an image sensor, the image sensor is electrically connected to the controller, and the controller is electrically connected to a brightness sensor for detecting ambient brightness. The camera device control method includes: continuously acquiring a detection voltage of the brightness sensor, determining a voltage change between a currently acquired detection voltage and a previously acquired detection voltage; whenever a voltage change is greater than or equal to a change threshold, determining a target exposure time and a target gain based on the currently acquired detection voltage; and configuring the target exposure time and target gain to the image sensor. According to the solution provided in the embodiment of the present application, the detection voltage of the brightness sensor is continuously acquired to determine the voltage change between the currently acquired detection voltage and the previously acquired detection voltage. Then, whenever the voltage change is greater than or equal to the change threshold and the currently acquired detection voltage is within a preset adjustment range, the target exposure time and target gain are determined based on the currently acquired detection voltage. Then, the target exposure time and target gain are configured to the image sensor. Compared with the automatic exposure algorithm that requires repeated closed-loop calculations, the target exposure time and target gain can be quickly determined, thereby shortening the control response time of the image sensor. In addition, the brightness changes of the external environment are accurately reflected by detecting the voltage change, thereby optimizing the target exposure time and target gain, thereby improving the image quality of the camera device.

[0038] The camera device control method, apparatus, device and storage medium provided in the embodiments of the present application are specifically described through the following embodiments. First, the camera device control method in the embodiments of the present application is described.

[0039] The camera device control method provided in the embodiment of the present application relates to the field of computer technology. The camera device control method provided in the embodiment of the present application is applied to the camera device and can be software running in the camera device. In some embodiments, the camera device can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the software can be an application that implements the camera device control method, etc., but is not limited to the above forms.

[0040] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0041] The embodiments of the present application are further described below in conjunction with the accompanying drawings.

[0042] like Figure 1 As shown, Figure 1 An optional flow chart of a camera device control method provided in an embodiment of the present application, the camera device control method is applied to a camera device, the camera device includes a controller and an image sensor, the image sensor is electrically connected to the controller, the controller is electrically connected to a brightness sensor for detecting ambient brightness, the camera device control method includes but is not limited to the following steps S110 to S130:

[0043] Step S110 , continuously acquiring the detection voltage of the brightness sensor, and determining a voltage change between the currently acquired detection voltage and the previously acquired detection voltage.

[0044] The brightness sensor may be a photodiode, which can convert light signals into electrical signals, and specifically can convert light brightness of different intensities into voltages of different magnitudes.

[0045] Among them, the brightness sensor converts the brightness of the external environment into a detection voltage that can be processed by the camera equipment.

[0046] Among them, the time difference between the last detection voltage obtained and the current detection voltage obtained is the detection period. The detection period can be pre-set for the camera device, or determined by a dynamic generation method, which is not limited in the embodiments of the present disclosure.

[0047] The difference between the currently acquired detection voltage and the previously acquired detection voltage is determined as the voltage variation.

[0048] It can be understood that, since the detection voltage can reflect the brightness of the external light, the voltage change can reflect the severity of the external light change.

[0049] Step S120 , whenever the voltage variation is greater than or equal to the variation threshold and the currently acquired detection voltage is within a preset adjustment range, a target exposure time and a target gain are determined based on the currently acquired detection voltage.

[0050] Step S130, configuring the target exposure time and the target gain to the image sensor.

[0051] Among them, the change threshold is used to indicate whether the change in external brightness requires the camera device to adjust the target exposure time and target gain. The change threshold can be obtained through experiments or dynamically generated according to user habits, and the embodiments of the present disclosure are not limited here.

[0052] The adjustment range of the detection voltage corresponds to the ambient brightness range. When the brightness of the external environment changes to outside the ambient brightness range, the adjustment force of the camera device is the largest, and the limit values ​​of the target exposure time and the target gain can be directly configured to the image sensor. For example, the upper and lower limits of the adjustment range of the detection voltage are 600uv and 10uv, respectively, and the upper and lower limits of the corresponding ambient brightness range are 1.0Lux and 0.0001Lux, respectively, the upper and lower limits of the target exposure time are 500ms and 0ms, respectively, and the upper and lower limits of the target gain are 1536dB and 0dB, respectively. When the brightness of the external environment is 2. When the brightness of the external environment is 0Lux, the corresponding detection voltage is 1uv, which exceeds the adjustment range of the detection voltage. In this case, the lower limit values ​​of the target exposure time and the target gain are directly configured to the image sensor, that is, the target exposure time is determined to be 0ms, the target gain is determined to be 0dB, and they are configured to the image sensor; when the brightness of the external environment is 0.0001Lux, the corresponding detection voltage is 1uv, which exceeds the adjustment range of the detection voltage. In this case, the upper limit values ​​of the target exposure time and the target gain are directly configured to the image sensor, that is, the target exposure time is determined to be 500ms, the target gain is determined to be 1536dB, and they are configured to the image sensor.

[0053] Based on this, by continuously acquiring the detection voltage of the brightness sensor, the voltage change between the currently acquired detection voltage and the previously acquired detection voltage is determined, and then whenever the voltage change is greater than or equal to the change threshold, and the currently acquired detection voltage is within a preset adjustment range, the target exposure time and target gain are determined based on the currently acquired detection voltage, and then the target exposure time and target gain are configured to the image sensor. Compared with the automatic exposure algorithm that requires repeated closed-loop calculations, the target exposure time and target gain can be quickly determined, thereby shortening the control response time of the image sensor; in addition, the brightness changes of the external environment can be accurately reflected by detecting the voltage change, thereby optimizing the target exposure time and target gain, and thus improving the image quality of the camera equipment.

[0054] In addition, refer to Figure 2 In one embodiment, determining the target exposure time and the target gain based on the currently acquired detection voltage includes but is not limited to the following steps:

[0055] Step S210, constructing multiple candidate parameter models.

[0056] Step S220 , determining a target parameter model from a plurality of candidate parameter models based on the currently acquired detection voltage.

[0057] Step S230: input the currently acquired detection voltage into the first sub-model in the target parameter model for prediction to obtain the target exposure time.

[0058] Step S240: input the currently acquired detection voltage into the second sub-model in the target parameter model for prediction to obtain the target gain.

[0059] Each candidate parameter model includes a first sub-model for determining exposure time and a second sub-model for determining gain.

[0060] The candidate parameter model may be a linear model or a regression model, which is not limited in the embodiments of the present disclosure.

[0061] Based on this, multiple candidate parameter models are constructed, and based on the currently acquired detection voltage, a target parameter model is determined among the multiple candidate parameter models. Then, the currently acquired detection voltage is input into the first sub-model in the target parameter model for prediction to obtain the target exposure time, and the currently acquired detection voltage is input into the second sub-model in the target parameter model for prediction to obtain the target gain, so that the target parameter model is most suitable for processing the currently acquired detection voltage, thereby maximizing the improvement of the image quality by the output target exposure time and target gain, and shortening the control response time of the image sensor to the optimum.

[0062] In addition, refer to Figure 3In one embodiment, each candidate parameter model is configured with a corresponding reference voltage range, each reference voltage range is continuous and non-overlapping, each reference voltage range is within an adjustment range, and the ambient brightness indicated by the voltage lower limit of each reference voltage range is in an exponentially decreasing relationship. Based on the currently acquired detection voltage, determining a target parameter model from a plurality of candidate parameter models includes but is not limited to the following steps:

[0063] Step S310 , determining a target voltage range in each reference voltage range based on the currently acquired detection voltage.

[0064] Step S320: Determine, among multiple candidate parameter models, a candidate parameter model corresponding to the target voltage range as the target parameter model.

[0065] Each reference voltage range has a corresponding ambient brightness range, and each ambient brightness range is continuous and non-overlapping.

[0066] For example, the voltage lower limits of each reference voltage range are 420uv, 150uv, and 10uv, respectively. At this time, the ambient brightness indicated by the voltage lower limits of each reference voltage range are 0.1Lux, 0.001Lux, and 0.0001Lux, respectively. That is, the ambient brightness is exponentially decreasing. Specifically, the ambient brightness indicated by the voltage lower limit of the previous reference voltage range is the ambient brightness indicated by the voltage upper limit of the next reference voltage range, and the ambient brightness indicated by the voltage upper limit of the first reference voltage range can specifically be 1Lux.

[0067] Among them, from the lower limits of each reference voltage range, the one that is closest in size and less than or equal to the lower limit of the currently obtained detection voltage is taken as the target lower limit, and from the upper limits of each reference voltage range, the one that is closest in size and greater than or equal to the upper limit of the currently obtained detection voltage is taken as the target upper limit, and the reference voltage range corresponding to the target lower limit and the target upper limit is determined as the target voltage range.

[0068] Among them, each reference voltage range corresponds to each candidate parameter model one by one, so that the target parameter model can be determined according to the target voltage range.

[0069] Based on this, based on the currently acquired detection voltage, the target voltage range is determined in each reference voltage range, and among multiple candidate parameter models, the candidate parameter model corresponding to the target voltage range is determined as the target parameter model. Since each candidate parameter model is more effective in processing the detection voltage within the corresponding reference voltage range, determining the target voltage range and then determining the target parameter model can maximize the improvement in image quality through the output target exposure time and target gain, and shorten the control response time of the image sensor to the optimum.

[0070] In addition, refer to Figure 4 In one embodiment, each first sub-model is configured with an upper limit and a lower limit of exposure time, the output of the first sub-model is negatively correlated with the input of the first sub-model and changes linearly, and the output change amplitude of the first sub-model increases as the voltage lower limit of the target voltage range of the candidate parameter model decreases. The currently acquired detection voltage is input into the first sub-model in the target parameter model for prediction to obtain the target exposure time, including but not limited to the following steps:

[0071] Step S410, input the currently acquired detection voltage into the first sub-model in the target parameter model, and interpolate and predict the exposure time corresponding to the detection voltage according to the upper limit of exposure time, the lower limit of exposure time, the lower limit of voltage of the target voltage range, and the upper limit of voltage of the target voltage range configured by the first sub-model to obtain the target exposure time.

[0072] Among them, the exposure time upper limit, exposure time lower limit, voltage lower limit of the target voltage range and voltage upper limit of the target voltage range configured by the first sub-model can all be obtained through multiple experiments, or dynamically generated by the model, and the embodiments of the present disclosure are not limited here.

[0073] Among them, the first sub-model is a linear model, and the first sub-model is mapped to a rectangular coordinate system with the detection voltage as the horizontal coordinate and the exposure time as the vertical coordinate. The lower limit of the exposure time and the upper limit of the voltage of the target voltage range constitute the first reference coordinate point, and the upper limit of the exposure time and the lower limit of the voltage of the target voltage range constitute the second reference coordinate point. The first reference coordinate point and the second reference coordinate point are used to determine the first model straight line corresponding to the first sub-model, and then the currently obtained detection voltage is used as the horizontal coordinate value and substituted into the first model straight line to determine the vertical coordinate value, that is, the reference exposure time, and finally the difference between the upper limit of the exposure time and the reference exposure is determined as the target exposure time.

[0074] Specifically, the calculation method is as follows: the difference between the lower limit of the exposure time and the upper limit of the exposure time is determined as the first intermediate difference, the difference between the voltage upper limit of the target voltage range and the voltage lower limit of the target voltage range is determined as the second intermediate difference, the difference between the currently acquired detection voltage and the voltage upper limit of the target voltage range is determined as the third intermediate difference, the quotient of the first intermediate difference and the second intermediate difference is determined as the slope of the first model straight line, the product of the third intermediate difference and the slope of the first model straight line is determined as the reference exposure time, and finally the difference between the exposure time upper limit and the reference exposure is determined as the target exposure time.

[0075] The output variation amplitude of the first sub-model is the absolute value of the slope of the first model straight line.

[0076] It should be noted that an example of determining the output variation of the first sub-model and determining the target exposure time is as follows: Assuming there are three candidate parameter models, there are three corresponding first sub-models. The first reference coordinate points corresponding to the three first sub-models are (10, 500), (150, 142) and (420, 40), which is equivalent to the corresponding second reference coordinate points being (150, 142), (420, 40) and (600, 0), and the output variation of the slopes corresponding to the three first sub-models is 2.557, 0.378 and 0.222, respectively; assuming the detection voltage is 510uv and the exposure time is

[0077] Based on this, by inputting the currently acquired detection voltage into the first sub-model in the target parameter model, the exposure time corresponding to the detection voltage is interpolated and predicted according to the upper limit of exposure time, the lower limit of exposure time, the lower limit of the target voltage range and the upper limit of the target voltage range configured by the first sub-model, so that the first sub-model can determine the target exposure time more efficiently with the scientifically configured calculation parameters, so that the output target exposure time can maximize the improvement of image quality and shorten the control response time of the image sensor to the optimum.

[0078] In addition, refer to Figure 5 In one embodiment, each second sub-model is configured with a gain upper limit and a gain lower limit, the output of the second sub-model is negatively correlated with the input of the second sub-model and changes linearly, and the output change amplitude of the second sub-model increases as the voltage lower limit of the target voltage range of the candidate parameter model decreases. The currently acquired detection voltage is input into the second sub-model in the target parameter model for prediction to obtain the target gain, including but not limited to the following steps:

[0079] Step S510, input the currently acquired detection voltage into the second sub-model in the target parameter model, and interpolate and predict the gain corresponding to the detection voltage according to the gain upper limit, gain lower limit, voltage lower limit of the target voltage range and voltage upper limit of the target voltage range configured by the second sub-model to obtain the target gain.

[0080] Among them, the gain upper limit, gain lower limit, voltage lower limit of the target voltage range and voltage upper limit of the target voltage range configured by the second sub-model can all be obtained through multiple experiments, or dynamically generated by the model, and the embodiments of the present disclosure are not limited here.

[0081] Among them, the second sub-model is a linear model, and the second sub-model is mapped to a rectangular coordinate system with the detection voltage as the horizontal coordinate and the gain as the vertical coordinate. The gain lower limit and the voltage upper limit of the target voltage range constitute the third reference coordinate point, and the gain upper limit and the voltage lower limit of the target voltage range constitute the fourth reference coordinate point. The second model straight line corresponding to the second sub-model is determined by the third reference coordinate point and the fourth reference coordinate point, and then the currently obtained detection voltage is used as the horizontal coordinate value and substituted into the second model straight line to determine the vertical coordinate value, that is, the gain, and finally the difference between the gain upper limit and the reference gain is determined as the target gain.

[0082] Specifically, the calculation method is as follows: the difference between the gain lower limit and the gain upper limit is determined as the fourth intermediate difference, the difference between the voltage upper limit of the target voltage range and the voltage lower limit of the target voltage range is determined as the fifth intermediate difference, the difference between the currently obtained detection voltage and the voltage upper limit of the target voltage range is determined as the sixth intermediate difference, the quotient of the fourth intermediate difference and the fifth intermediate difference is determined as the slope of the second model straight line, the product of the sixth intermediate difference and the slope of the second model straight line is determined as the reference gain, and finally the difference between the gain upper limit and the reference gain is determined as the target gain.

[0083] The output variation range of the second sub-model is the absolute value of the slope of the second model straight line.

[0084] It should be noted that an example of determining the output change amplitude of the second sub-model and determining the target gain is as follows: Assuming that there are three candidate parameter models, there are three corresponding second sub-models. The third reference coordinate points corresponding to the three second sub-models are (10, 1536), (150, 512) and (420, 8), which is equivalent to the corresponding fourth reference coordinate points being (150, 512), (420, 8) and (600, 0), and the output change amplitudes of the slopes corresponding to the three second sub-models are 7.314, 1.867 and 0.0444, respectively; assuming that the detection voltage is 510uv and the gain is

[0085] Based on this, by inputting the currently acquired detection voltage into the second sub-model in the target parameter model, the gain corresponding to the detection voltage is interpolated and predicted according to the gain upper limit, gain lower limit, voltage lower limit of the target voltage range and voltage upper limit of the target voltage range configured by the second sub-model, so that the second sub-model can determine the target gain more efficiently with the scientifically configured calculation parameters, so that the output target gain can maximize the improvement of image quality and shorten the control response time of the image sensor to the optimum.

[0086] It can be understood that the relationship between the ambient brightness, the detection voltage, the exposure time and the gain is shown in Table 1 below:

[0087] Table 1

[0088]

[0089] In addition, refer to Figure 6 In one embodiment, the camera device further includes a multispectral sensor, which is electrically connected to the controller, and the target exposure time and the target gain are configured to the image sensor, including but not limited to the following steps:

[0090] Step S610: acquiring a test image captured by an image sensor and spectral data captured by a multispectral sensor when the test image is captured.

[0091] Step S620, extracting image features of the test image, extracting spectral features of the spectral data, splicing the image features and the spectral features and inputting them into a light source type recognition model for recognition, and determining a target light source type from a plurality of candidate light source types based on the recognition result.

[0092] Step S630: weighting the target exposure time based on the first weight configured for the target light source type to obtain a weighted exposure time.

[0093] Step S640: weighting the target gain based on the second weight configured for the target light source type to obtain a weighted gain.

[0094] Step S650: configuring the weighted exposure time and the weighted gain to the image sensor.

[0095] Each candidate light source type is configured with a first weight for adjusting exposure time and a second weight for adjusting gain.

[0096] The multi-spectral sensor is used to detect the spectral information of the external environment to obtain spectral data of the external environment including multiple channels, for example, 10 channels, where different channels can correspond to different wavelength intervals. Since the spectral data is collected when the test image is collected, the spectral value of the external environment in the spectral data can correspond to the light source recorded in the test image.

[0097] The multispectral sensor may use a single-pixel multispectral sensor or a high-resolution multispectral sensor, as long as the photosensitivity range of the multispectral sensor includes the visible light range. This application has no restrictions on this.

[0098] The image feature of the test image may be a numerical vector corresponding to multiple pixel value intervals of the test image, and the feature vector is obtained by concatenating the image feature and the spectral feature.

[0099] The recognition result is the probability distribution of each candidate light source type output by the classification model, and the candidate light source type with the highest probability is determined as the target light source type.

[0100] Among them, the first weight and the second weight are both values ​​greater than or equal to 0 and less than or equal to 1. The first weight and the second weight can be obtained through multiple experiments or dynamically generated by the model, and the embodiments of the present disclosure are not limited here.

[0101] It should be noted that the first weight corresponds to the weight of the currently acquired detection voltage. The difference between the value 1 and the first weight is determined as the first reference weight. The first reference weight corresponds to the detection voltage acquired last time. The detection voltage acquired last time is also configured with a corresponding pre-exposure time. The product of the first weight and the target exposure time is added to the product of the first reference weight and the pre-exposure time to obtain the weighted exposure time. For example, if the first weight is 0.5, then the first reference weight is 0.5. At this time, the weighted exposure time is the average of the target exposure time and the pre-exposure time.

[0102] It should be noted that the second weight corresponds to the weight of the currently acquired detection voltage. The difference between the value 1 and the second weight is determined as the second reference weight. The second reference weight corresponds to the detection voltage acquired last time. The detection voltage acquired last time is also configured with a corresponding pre-gain. The product of the second weight and the target gain is added to the product of the second reference weight and the pre-gain to obtain the weighted gain. For example, if the second weight is 0.8, the second reference weight is 0.2. At this time, the product of 0.8 times the target gain and 0.2 times the pre-gain is determined as the weighted gain.

[0103] Based on this, a test image collected by an image sensor and spectral data collected by a multispectral sensor when collecting the test image are obtained, and then image features of the test image are extracted, and spectral features of the spectral data are extracted, so as to reduce the amount of data input into the light source type recognition model in advance and reduce the data processing cost. Then, the image features and spectral features are spliced ​​and input into the light source type recognition model for recognition, and the target light source type is determined from multiple candidate light source types based on the recognition result, so as to identify the light source type, reduce the phenomenon of metamerism, and greatly improve the accuracy of light source type recognition. Then, the target exposure time is weighted based on the first weight configured for the target light source type to obtain the weighted exposure time, and the target gain is weighted based on the second weight configured for the target light source type to obtain the weighted gain. Finally, the weighted exposure time and the weighted gain are configured to the image sensor, so as to improve the accuracy of the weighted exposure time and the weighted gain, so that the output weighted exposure time and the weighted gain can maximize the improvement of the image quality, and shorten the control response time of the image sensor to the best.

[0104] In a possible implementation, the light source type recognition model is a pre-trained classification model, and the training steps of the light source type recognition model are as follows:

[0105] Firstly, the test image samples, spectral data samples and the corresponding light source type labels are obtained, the features of the test image samples are extracted as the reference image features, the features of the spectral data samples are extracted as the reference spectral features, the reference image features and the reference spectral features are spliced ​​and input into the light source type recognition model for classification processing to obtain the reference light source type; then, the model loss is determined according to the reference light source type and the light source type label, and the light source type recognition model is trained based on the model loss, so that the light source type recognition model can learn and understand the intrinsic connection between the test image, spectral data and light source type, and then accurately predict the target light source type by combining the test image and spectral data in the inference stage.

[0106] Specifically, refer to Figure 7 , Figure 7 An optional flow chart of configuring an image sensor provided in an embodiment of the present application, at the beginning, continuously acquiring a detection voltage of a brightness sensor, determining a voltage variation between a currently acquired detection voltage and a previously acquired detection voltage, and whenever a voltage variation is greater than or equal to a variation threshold, and the currently acquired detection voltage is within a preset adjustment range, determining a target voltage range in each reference voltage range based on the currently acquired detection voltage, determining a candidate parameter model corresponding to the target voltage range as a target parameter model from among multiple candidate parameter models, and determining a target parameter model from among multiple candidate parameter models, wherein the target parameter model is configured with a corresponding reference voltage. The target parameter model comprises a reference voltage range, an exposure time upper limit, an exposure time lower limit, a gain upper limit and a gain lower limit, and the first sub-model of the target parameter model and the second sub-model of the target parameter model are jointly configured with the above-mentioned reference voltage range, the first sub-model of the target parameter model is configured with the above-mentioned exposure time upper limit and the above-mentioned exposure time lower limit, and the second sub-model of the target parameter model is configured with the above-mentioned gain upper limit and the above-mentioned gain lower limit. The currently acquired detection voltage is input into the first sub-model in the target parameter model for prediction to obtain the target exposure time, the currently acquired detection voltage is input into the second sub-model in the target parameter model for prediction to obtain the target gain, and finally the target exposure time and the target gain are configured into the image sensor through the I2C (Inter Integrated Circuit) bus to end the configuration of the image sensor; in addition, whenever the voltage change is less than the change threshold, the detection voltage of the brightness sensor is continuously acquired again.

[0107] In addition, refer to Figure 8 The present application also provides a camera device 800, including:

[0108] Image sensor 810;

[0109] The controller 820, the image sensor 810 is electrically connected to the controller 820, the controller 820 is electrically connected to a brightness sensor 830 for detecting ambient brightness, and the controller 820 is used to execute the camera device control method of the embodiment of the present application.

[0110] It can be understood that the specific implementation of the camera device 800 is basically the same as the specific embodiment of the above-mentioned camera device control method, and will not be repeated here.

[0111] In addition, refer to Fig. 9 The present application also provides a camera device control device 900, which is applied to a camera device. The camera device includes a controller and an image sensor. The image sensor is electrically connected to the controller. The controller is electrically connected to a brightness sensor for detecting ambient brightness. The camera device control method includes:

[0112] An acquisition module 910 is used to continuously acquire a detection voltage of a brightness sensor and determine a voltage change between a currently acquired detection voltage and a previously acquired detection voltage;

[0113] A calculation module 920, configured to determine a target exposure time and a target gain based on the currently acquired detection voltage whenever the voltage variation is greater than or equal to the variation threshold and the currently acquired detection voltage is within a preset adjustment range;

[0114] The configuration module 930 is used to configure the target exposure time and the target gain to the image sensor.

[0115] It can be understood that the specific implementation of the camera device control apparatus 900 is substantially the same as the specific implementation of the above-mentioned camera device control method, and will not be described in detail herein.

[0116] In addition, refer to Fig.10 , Fig.10 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0117] The processor 1001 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0118] The memory 1002 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1002 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1002, and the processor 1001 calls and executes the camera device control method of the embodiment of the present application.

[0119] Input / output interface 1003, used to implement information input and output;

[0120] The communication interface 1004 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);

[0121] A bus 1005 , which transmits information between various components of the device (e.g., the processor 1001 , the memory 1002 , the input / output interface 1003 , and the communication interface 1004 );

[0122] The processor 1001 , the memory 1002 , the input / output interface 1003 and the communication interface 1004 are connected to each other in communication within the device via the bus 1005 .

[0123] An embodiment of the present application also provides a storage medium, which is a computer-readable storage medium used for computer-readable storage. The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the above-mentioned camera device control method.

[0124] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0125] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0126] It can be understood by those skilled in the art that Figures 1 to 10 The technical solutions shown in the figure do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than those shown in the figure, or a combination of certain steps, or different steps.

[0127] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0128] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0129] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0130] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0131] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0132] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0133] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0134] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0135] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A camera device control method, characterized in that: Applied to a camera device, the camera device includes a controller and an image sensor, the image sensor is electrically connected to the controller, the controller is electrically connected to a brightness sensor for detecting ambient brightness, and the camera device control method includes: Continuously acquiring a detection voltage of the brightness sensor, and determining a voltage change between the currently acquired detection voltage and the previously acquired detection voltage; Whenever the voltage variation is greater than or equal to a variation threshold and the currently acquired detection voltage is within a preset adjustment range, a target exposure time and a target gain are determined based on the currently acquired detection voltage; The target exposure time and the target gain are configured to the image sensor.

2. The camera control method according to claim 1, characterized in that: The determining of the target exposure time and the target gain based on the currently acquired detection voltage comprises: Constructing a plurality of candidate parameter models, wherein each of the candidate parameter models comprises a first sub-model for determining exposure time and a second sub-model for determining gain; Determining a target parameter model from a plurality of candidate parameter models based on the currently acquired detection voltage; Inputting the currently acquired detection voltage into the first sub-model in the target parameter model for prediction to obtain a target exposure time; The currently acquired detection voltage is input into the second sub-model in the target parameter model for prediction to obtain a target gain.

3. The camera control method according to claim 2, characterized in that: Each of the candidate parameter models is configured with a corresponding reference voltage range, each of the reference voltage ranges is continuous and non-overlapping, each of the reference voltage ranges is within the adjustment range, and the ambient brightness indicated by the voltage lower limit of each of the reference voltage ranges is in an exponentially decreasing relationship. Determining a target parameter model from a plurality of the candidate parameter models based on the currently acquired detection voltage includes: Determining a target voltage range in each of the reference voltage ranges based on the currently acquired detection voltage; Among the multiple candidate parameter models, the candidate parameter model corresponding to the target voltage range is determined as the target parameter model.

4. The camera control method according to claim 3, characterized in that: Each of the first sub-models is configured with an upper limit and a lower limit of exposure time, the output of the first sub-model is negatively correlated with the input of the first sub-model and changes linearly, the output change amplitude of the first sub-model increases as the voltage lower limit of the target voltage range of the candidate parameter model decreases, and the currently acquired detection voltage is input into the first sub-model in the target parameter model for prediction to obtain the target exposure time, including: The currently acquired detection voltage is input into the first sub-model in the target parameter model, and the exposure time corresponding to the detection voltage is interpolated and predicted according to the upper limit of exposure time, the lower limit of exposure time, the lower limit of voltage of the target voltage range and the upper limit of voltage of the target voltage range configured by the first sub-model to obtain the target exposure time.

5. The camera control method according to claim 3, characterized in that: Each of the second sub-models is configured with a gain upper limit and a gain lower limit, the output of the second sub-model is negatively correlated with the input of the second sub-model and changes linearly, the output change amplitude of the second sub-model increases as the voltage lower limit of the target voltage range of the candidate parameter model decreases, and the currently acquired detection voltage is input into the second sub-model in the target parameter model for prediction to obtain the target gain, including: The currently acquired detection voltage is input into the second sub-model in the target parameter model, and the gain corresponding to the detection voltage is interpolated and predicted according to the gain upper limit, the gain lower limit, the voltage lower limit of the target voltage range and the voltage upper limit of the target voltage range configured by the second sub-model to obtain the target gain.

6. The camera control method according to claim 1, characterized in that: The camera device further includes a multispectral sensor, which is electrically connected to the controller, and the configuring the target exposure time and the target gain to the image sensor includes: Acquire a test image captured by the image sensor and spectral data collected by the multispectral sensor when the test image is captured; Extracting image features of the test image, extracting spectral features of the spectral data, splicing the image features and the spectral features and inputting them into a light source type recognition model for recognition, and determining a target light source type from a plurality of candidate light source types based on the recognition result, wherein each of the candidate light source types is configured with a first weight for adjusting exposure time and a second weight for adjusting gain; Weighting the target exposure time based on a first weight configured for the target light source type to obtain a weighted exposure time; Weighting the target gain based on a second weight configured for the target light source type to obtain a weighted gain; The weighted exposure time and the weighted gain are assigned to the image sensor.

7. A camera device, characterized in that: include: Image sensor; A controller, wherein the image sensor is electrically connected to the controller, the controller is electrically connected to a brightness sensor for detecting ambient brightness, and the controller is used to execute the camera device control method according to any one of claims 1 to 6.

8. A camera control device, characterized in that: Applied to a camera device, the camera device includes a controller and an image sensor, the image sensor is electrically connected to the controller, the controller is electrically connected to a brightness sensor for detecting ambient brightness, and the camera device control method includes: An acquisition module, used for continuously acquiring a detection voltage of the brightness sensor, and determining a voltage change between the currently acquired detection voltage and the previously acquired detection voltage; A calculation module, configured to determine a target exposure time and a target gain based on the currently acquired detection voltage whenever the voltage variation is greater than or equal to a variation threshold and the currently acquired detection voltage is within a preset adjustment range; A configuration module is used to configure the target exposure time and the target gain to the image sensor.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the camera device control method according to any one of claims 1 to 6 when executing the computer program.

10. A storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the imaging device control method according to any one of claims 1 to 6 is implemented.

Images