Parameter configuration method and device and storage medium

Abstract

The invention discloses a parameter configuration method and device, and a storage medium, for each image sensor in at least one image sensor, a configuration parameter set corresponding to the image sensor is determined, and the configuration parameter set is generated by an image processor based on an image collected by the corresponding image sensor. The configuration parameter set comprises at least two configuration parameters; for each configuration parameter set, selecting candidate configuration parameters from the configuration parameter sets to determine at least one candidate configuration parameter, and storing the at least one candidate configuration parameter; the candidate configuration parameters are effective configuration parameters in the configuration parameter set; under the condition that a target image sensor is started, the at least one candidate configuration parameter is read, a target configuration parameter is selected from the read at least one candidate configuration parameter, and the target configuration parameter is used for the image processor to process target image data collected by the target image sensor.

Description

Technical Field

[0001] The present application relates to terminal control technology, and in particular to a parameter configuration method and device, and a storage medium. Background Art

[0002] When the camera of an electronic device is started, the 5A algorithm starts running. The configuration parameters of the 5A algorithm when it starts running are the configuration parameters of the 5A hardware module when the camera application was last run. The configuration parameters used are called "warm data". The use of 5A warm data can speed up the convergence of the 5A algorithm and obtain a faster camera stability response effect.

[0003] For the camera system of the homogeneous platform, the Camera Hardware Abstract Layer (HAL) in the Application Processor Computing Processing Unit (ACPU) directly controls and monitors the operating status of the Image Signal Processing (ISP) hardware system in real time, and the 5A algorithm is deployed on the ACPU side. The calculation and result output of the 5A algorithm and the image frames are synchronized. The Camera HAL in the ACPU can directly record the configuration parameters of the 5A hardware module corresponding to the last valid image frame when the preview stream of a camera ends. However, in the Camera system of the heterogeneous platform, the operating status of the ISP hardware system is controlled and monitored in real time by the microprocessor (MCU), but the Camera HAL is deployed in the APCU, which makes it impossible for the ACPU to directly obtain the configuration parameters on the 5A hardware module at the last valid frame. Summary of the invention

[0004] The embodiments of the present application provide a parameter configuration method and device, and a storage medium, which can determine the configuration parameters, namely, thermal data, used when a target image sensor is started.

[0005] The technical solution of the embodiment of the present application is implemented as follows:

[0006] The present application provides a parameter configuration method, the method comprising:

[0007] For each image sensor in at least one image sensor, determine a configuration parameter set corresponding to the image sensor, the configuration parameter set is generated by an image processor based on an image acquired by the corresponding image sensor, the configuration parameter set includes at least two configuration parameters, and different configuration parameters correspond to different frame images;

[0008] For each configuration parameter set, selecting a candidate configuration parameter from the configuration parameter set to determine at least one candidate configuration parameter, and storing the at least one candidate configuration parameter; the candidate configuration parameter is a valid configuration parameter in the configuration parameter set;

[0009] When the target image sensor is started, the at least one candidate configuration parameter is read, and a target configuration parameter is selected from the at least one candidate configuration parameter read, the target configuration parameter being used by the image processor to process the target image data acquired by the target image sensor, the target image sensor being one of the at least one image sensor.

[0010] An embodiment of the present application provides an electronic device, the electronic device comprising:

[0011] a determining unit configured to determine, for each image sensor of at least one image sensor, a configuration parameter set corresponding to the image sensor, wherein the configuration parameter set is generated by an image processor based on an image acquired by the corresponding image sensor, and the configuration parameter set includes at least two configuration parameters, and different configuration parameters correspond to different frame images;

[0012] A first selection unit is configured to select, for each configuration parameter set, a candidate configuration parameter from the configuration parameter set to determine at least one candidate configuration parameter; the candidate configuration parameter is a valid configuration parameter in the configuration parameter set;

[0013] a storage unit configured to store the at least one candidate configuration parameter;

[0014] The second selection unit is configured to read the at least one candidate configuration parameter and select a target configuration parameter from the at least one candidate configuration parameter read when the target image sensor is started, wherein the target configuration parameter is used by the image processor to process the target image data collected by the target image sensor, and the target image sensor is one of the at least one image sensor.

[0015] An embodiment of the present application provides an electronic device, including a processor, wherein the processor is configured to:

[0016] For each image sensor in at least one image sensor, determine a configuration parameter set corresponding to the image sensor, the configuration parameter set is generated by an image processor based on an image acquired by the corresponding image sensor, the configuration parameter set includes at least two configuration parameters, and different configuration parameters correspond to different frame images;

[0017] For each configuration parameter set, selecting a candidate configuration parameter from the configuration parameter set to determine at least one candidate configuration parameter, and storing the at least one candidate configuration parameter; the candidate configuration parameter is a valid configuration parameter in the configuration parameter set;

[0018] When the target image sensor is started, the at least one candidate configuration parameter is read, and a target configuration parameter is selected from the at least one candidate configuration parameter read, the target configuration parameter being used by the image processor to process the target image data acquired by the target image sensor, the target image sensor being one of the at least one image sensor.

[0019] An embodiment of the present application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps in the above-mentioned parameter configuration method are implemented.

[0020] An embodiment of the present application provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned parameter configuration method is implemented.

[0021] The chip provided in the embodiment of the present application is used to implement the above-mentioned parameter configuration method. The chip includes: a processor, which is used to call and run a computer program from a memory so that a device equipped with the chip executes the above-mentioned parameter configuration method.

[0022] The parameter configuration method, device, and storage medium provided in the embodiments of the present application determine, for each image sensor in at least one image sensor, a configuration parameter set corresponding to the image sensor, the configuration parameter set being generated by an image processor based on an image captured by the corresponding image sensor, the configuration parameter set including at least two configuration parameters, and different configuration parameters corresponding to different frame images; for each configuration parameter set, select candidate configuration parameters from the configuration parameter set to determine at least one candidate configuration parameter, and store the at least one candidate configuration parameter; the candidate configuration parameter is a valid configuration parameter in the configuration parameter set; when a target image sensor is started, read the at least one candidate configuration parameter, and select a target configuration parameter from the at least one candidate configuration parameter read, the target configuration parameter being used by the image processor to process target image data captured by the target image sensor, the target image sensor being one of the at least one image sensor; thereby selecting valid configuration parameters from the configuration parameter set including configuration parameters of different frame images for storage, and when the target image sensor is started, selecting a target configuration parameter for currently processing an image frame captured by the target sensor from a plurality of stored valid configuration parameters, and reducing the complexity of round-trip matching and inter-core communication when the configuration parameter used when the target image sensor is started is determined to be hot data. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is an optional flow chart of the parameter configuration method provided in the embodiment of the present application;

[0024] Figure 2 is an optional structural diagram of an electronic device provided in an embodiment of the present application;

[0025] Figure 3 This is an optional flow chart of the parameter configuration method provided in the embodiment of the present application;

[0026] Figure 4 This is an optional flow chart of the parameter configuration method provided in the embodiment of the present application;

[0027] Figure 5 It is an optional structural diagram of a Camera hardware system based on a heterogeneous platform provided in an embodiment of the present application;

[0028] Figure 6 is a schematic diagram of a thermal data determination process provided in an embodiment of the present application;

[0029] Figure 7 It is a schematic diagram of the data transmission software transmission path of the Camera system based on the heterogeneous platform provided in the embodiment of the present application;

[0030] Figure 8 This is a schematic diagram of the 5A operation result cache when the camera is running provided by the embodiment of the present application;

[0031] Fig. 9 This is a schematic diagram of the selection and storage of the 5A operation result when the camera is running provided by the embodiment of the present application;

[0032] Fig.10 This is a schematic diagram of the 5A calculation result loading process provided in the embodiment of the present application;

[0033] Fig.11 This is a schematic diagram of the 5A hot data loading strategy provided in the embodiment of the present application;

[0034] Fig.12 is an optional schematic structural diagram of an electronic device provided in an embodiment of the present application;

[0035] Fig.13 is an optional schematic structural diagram of an electronic device provided in an embodiment of the present application;

[0036] Fig.14 is an optional schematic structural diagram of an electronic device provided in an embodiment of the present application;

[0037] Fig.15 It is an optional schematic structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0039] The embodiments of the present application can provide a parameter configuration method, device and storage medium. In practical applications, the parameter configuration method can be implemented by an electronic device, and each functional entity in the electronic device can be collaboratively implemented by hardware resources of a computer device (such as a terminal device, a network device, etc.), such as computing resources such as a processor, and communication resources (such as for supporting various communication methods such as optical cables and cellular).

[0040] Of course, the embodiments of the present application are not limited to being provided as methods and hardware, and there may be multiple implementation methods, such as being provided as a storage medium (storing instructions for executing the parameter configuration method provided by the embodiments of the present application).

[0041] The following describes various embodiments of the parameter configuration method, device, and storage medium provided in the embodiments of the present application.

[0042] The parameter configuration method provided in the embodiment of the present application can be as follows Figure 1 As shown, the method may include:

[0043] S101. The electronic device determines, for each image sensor of at least one image sensor, a configuration parameter set corresponding to the image sensor, where the configuration parameter set is generated by an image processor based on an image captured by the corresponding image sensor, and the configuration parameter set includes at least two configuration parameters, and different configuration parameters correspond to different frame images.

[0044] In an embodiment of the present application, an electronic device is provided with one or more image sensors. For one of the image sensors of the electronic device, when the image sensor is running to collect images, a configuration parameter set corresponding to the image sensor is generated according to the image collected by the image sensor. Among them, one or more image sensors can be run simultaneously in the electronic device. Among them, the image sensor can be understood as a camera. A configuration parameter set corresponding to an image sensor is composed of configuration parameters corresponding to the image sensor, wherein the configuration parameters corresponding to the image sensor include: configuration parameters of the image sensor, configuration parameters of the data processing path for processing the image collected by the image sensor, the data processing path is used to perform image processing functions such as automatic exposure, automatic focusing, and automatic white balance on the image collected by the image sensor, and the data processing path can be composed of hardware modules that realize various image processing functions, wherein the hardware modules in the data path can run on the image processor. In the embodiment of the present application, no limitation is imposed on the image processing function in the data processing path. Among them, the configuration parameters of the image sensor are used to control the image sensor, and the configuration parameters of the image processing path are used to control the image processing path.

[0045] A configuration parameter set may include configuration parameters corresponding to different frames of images. When an image sensor is running, the image sensor captures an image, and the image processor generates configuration parameters corresponding to the frame of image based on a frame of image, thereby forming a configuration parameter set corresponding to the image sensor based on the configuration parameters corresponding to each frame of image in at least one frame of image captured by the image sensor. In the embodiment of the present application, the configuration parameters generated by the image processor are used by the image processor to process the image subsequently captured by the image sensor.

[0046] In the embodiment of the present application, for a frame of image, the corresponding configuration parameters may include one or more, and the multiple configuration parameters may be considered to constitute a configuration parameter cluster, which can be understood as a configuration parameter set including multiple configuration parameter set clusters, different configuration parameter set clusters corresponding to different frame images, and a configuration parameter cluster corresponding to a frame of image includes one or more configuration parameters. Among them, the different configuration parameters corresponding to a frame of image are configuration parameters of different functions. For example: configuration parameters of the image sensor, configuration parameters of the 5A algorithm, the configuration parameters of the 5A algorithm can be understood as the configuration parameters of the hardware module that implements the 5A algorithm in the data processing path. Among them, the 5A algorithm includes at least one of: Auto Exposure (AE), Auto Focus (AF), Auto While Balance (AWB), Auto Lens Shading Correction (ALSC) and Auto Anti-Flicker (ATF). Among them, different configuration parameters among the multiple configuration parameters corresponding to a frame of image are used to configure different hardware modules running on the image processor to achieve corresponding functions. For example, the configuration parameters corresponding to a frame of image include: AE configuration parameters, AF configuration parameters, and AWB configuration parameters. The AE configuration parameters, AF configuration parameters, and AWB configuration parameters are respectively used to configure: hardware module 1 for implementing the AE algorithm running on the image processor, hardware module 2 for implementing the AF algorithm running on the image processor, and hardware module 3 for implementing the AWB algorithm running on the image processor.

[0047] In the embodiment of the present application, for the AE algorithm, its configuration parameters may be called AE configuration parameters, and the AE configuration parameters include: exposure, exposure time, gain, etc.; for the AF algorithm, its configuration parameters may be called AF configuration parameters, and the AF configuration parameters include: the code value pushed by the motor in the image sensor, i.e., the motor pushing distance, and the focal distance; for the AWB algorithm, its configuration parameters may be called AWB configuration parameters, and the AWB configuration parameters include: color temperature, color image space CDM correction parameters; for the ALSC algorithm, its configuration parameters may be called ALSC configuration parameters, and the ALSC configuration parameters include: a lens correction table for correcting lens shading; for the ATF algorithm, its configuration parameters may be called ATF configuration parameters, and the ATF configuration parameters include: the frequency of the public screen light.

[0048] In an embodiment of the present application, multiple configuration parameters corresponding to a frame of image may be generated by one image processor or by multiple image processors. The image processor may include at least one of the following: an ACPU and an MCU. In one example, multiple configuration parameters corresponding to a frame of image are generated by an ACPU. In one example, multiple configuration parameters corresponding to a frame of image may be generated by an ACPU. In one example, some of the multiple configuration parameters of a frame of image are generated by an ACPU, and other configuration parameters are generated by an MCU. The configuration parameters generated by the ACPU are used by the ACPU to process images captured by the image sensor, and the configuration parameters generated by the MCU are used by the MAC to process images captured by the graphic sensor.

[0049] For an image sensor, the configuration parameters included in the corresponding configuration parameter set are for different images. For example: the electronic device has image sensor 1, and the configuration parameter set 1 corresponding to image sensor 1 includes: configuration parameters corresponding to the mth frame image, and configuration parameters corresponding to the m+1th frame image. For another example: the electronic device has image sensor 1 and image sensor 2, and the configuration parameter set 1 corresponding to image sensor 1 includes: configuration parameters corresponding to the mth frame image, and configuration parameters corresponding to the m+1th frame image, and the configuration parameter set 2 corresponding to image sensor 2 includes: configuration parameters corresponding to the m+2th frame image, and configuration parameters corresponding to the m+3th frame image.

[0050] In the embodiment of the present application, the configuration parameter sets corresponding to different image sensors can be stored in the same buffer area: the first buffer area, wherein different configuration parameter sets are stored in different buffer queues in the first buffer area. When the configuration parameter is stored, the corresponding image frame number, timestamp and other auxiliary information are also stored, wherein the timestamp is the generation time of the corresponding image. The electronic device can select the candidate configuration parameters based on the validity identification of each configuration parameter in the configuration parameter set and the auxiliary information.

[0051] In the embodiment of the present application, when the image sensor is working, the image processor can generate a set of configuration parameters corresponding to the corresponding image sensor in real time based on the image generated by the image sensor.

[0052] S102. The electronic device selects, for each configuration parameter set, candidate configuration parameters from the configuration parameter set to determine at least one candidate configuration parameter, and stores the at least one candidate configuration parameter; the candidate configuration parameter is a valid configuration parameter in the configuration parameter set.

[0053] For each configuration parameter set, the electronic device selects a valid configuration parameter from the configuration parameter set as a candidate configuration parameter corresponding to the image sensor, thereby constructing at least one candidate configuration parameter based on the candidate configuration parameters corresponding to each image sensor. The electronic device stores the constructed at least one candidate configuration parameter.

[0054] In the embodiment of the present application, when the electronic device stores the candidate configuration parameter, it may store at least one of the following information corresponding to the candidate configuration parameter: the identifier of the image sensor corresponding to the candidate configuration parameter, the validity identifier, and the timestamp. Among them, the identifier of the image sensor corresponding to the candidate configuration parameter is used to identify which image sensor the candidate configuration parameter corresponds to, the validity identifier is used to identify the candidate configuration parameter as a valid configuration parameter, and the timestamp represents the acquisition time of the image corresponding to the candidate configuration parameter to determine whether the candidate configuration parameter can be used as the target configuration parameter.

[0055] In an embodiment of the present application, if a configuration parameter is stored, a validity identifier and a timestamp may be stored accordingly. Then, when a configuration parameter is stored as a candidate configuration parameter, the corresponding image sensor identifier may be directly added based on the validity identifier and the timestamp.

[0056] In the embodiment of the present application, there is no limitation on the method for the electronic device to select candidate configuration parameters from the configuration parameter set.

[0057] The candidate configuration parameters stored in the electronic device may be understood as a data source of thermal data, wherein the thermal data may be understood as corresponding configuration parameters for the next target image sensor.

[0058] S103. When the target image sensor is started, the electronic device reads the at least one candidate configuration parameter and selects a target configuration parameter from the at least one candidate configuration parameter read, wherein the target configuration parameter is used by the image processor to process the target image data collected by the target image sensor, and the target image sensor is one of the at least one image sensor.

[0059] When the electronic device uses the target image sensor to capture an image, it reads at least one stored candidate configuration parameter, and selects the target configuration parameter as thermal data from the data source of the at least one candidate configuration parameter, i.e., thermal data. The image processor is configured through the target configuration parameter, so that the image processor processes the image captured by the target image sensor based on the configuration of the configuration parameter, thereby realizing the function corresponding to the configuration parameter.

[0060] In an embodiment of the present application, the target image sensor is one of the at least one image sensor, and the target configuration parameter selected by the electronic device from at least one candidate configuration parameter may be a candidate configuration parameter corresponding to the target image sensor, or may be a candidate configuration parameter corresponding to other image sensors.

[0061] The target image sensor can be started in a scenario where no image sensor is started in the electronic device, or when other image sensors are started, the image sensor for image acquisition is switched from other image sensors to the target image sensor, thereby using the target image sensor for image acquisition.

[0062] In the embodiment of the present application, after the electronic device determines a candidate configuration parameter corresponding to an image sensor, the candidate configuration parameter may be used as a configuration parameter for the current image sensor when capturing an image subsequently.

[0063] The parameter configuration method provided in the embodiment of the present application determines, for each image sensor in at least one image sensor, a configuration parameter set corresponding to the image sensor, the configuration parameter set is generated by an image processor based on an image captured by the corresponding image sensor, the configuration parameter set includes at least two configuration parameters, and different configuration parameters correspond to different frame images; for each configuration parameter set, selects candidate configuration parameters from the configuration parameter set to determine at least one candidate configuration parameter, and stores the at least one candidate configuration parameter; the candidate configuration parameter is a valid configuration parameter in the configuration parameter set; when a target image sensor is started, reads the at least one candidate configuration parameter, and selects a target configuration parameter from the at least one candidate configuration parameter read, the target configuration parameter is used by the image processor to process target image data captured by the target image sensor, and the target image sensor is one of the at least one image sensor; thereby selecting a valid configuration parameter from the configuration parameter set including configuration parameters of different frame images for storage, and when the target image sensor is started, selecting a target configuration parameter for currently processing an image frame captured by the target sensor from the stored multiple valid configuration parameters, and reducing the complexity of round-trip matching and inter-core communication when determining the configuration parameter used when the target image sensor is started.

[0064] In some embodiments, the configuration parameters include: a first configuration parameter and a second configuration parameter, the first configuration parameter is generated by a first image processor, and the second configuration parameter is generated by a second image processor, S101 determines a configuration parameter set corresponding to the image sensor, including:

[0065] When the image sensor captures a first frame of image, a first configuration parameter corresponding to the first frame of image is stored corresponding to a first frame identifier, and a second configuration parameter corresponding to a second frame of image captured by the image sensor is stored corresponding to the second frame identifier, wherein the first frame identifier is used to identify the first frame of image, and the second frame identifier is used to identify the second frame of image, and the second frame of image is a previous frame of image of the first frame of image;

[0066] When the image sensor captures a third frame, the first configuration parameter corresponding to the third frame image is stored corresponding to the third frame identifier, and the second configuration parameter corresponding to the first frame image is stored corresponding to the first frame identifier, the third frame image is a subsequent frame image of the first frame image, and the third frame identifier is used to identify the third frame image;

[0067] The first configuration parameter corresponding to the first frame identifier and the second configuration parameter corresponding to the first frame identifier constitute the configuration parameter corresponding to the first frame image in the configuration parameter set.

[0068] In an embodiment of the present application, for an image sensor, when the image sensor captures a frame of image, a first configuration parameter corresponding to the frame of image and a second configuration parameter of a previous frame of image are generated. The first configuration parameter is generated by a first image processor, and the second configuration parameter is generated by a second image processor. Here, the second frame of image is the previous frame of image of the first frame of image, and the first frame of image is the previous frame of image of the third frame of image.

[0069] In the embodiment of the present application, the first configuration parameter may include: AE configuration parameter, i.e., AE operation result, AF configuration parameter, i.e., AF operation result, the first image processor is MCU, wherein the ISP running on the MCU generates AE operation result and AF operation result, the second configuration parameter may include: AWB configuration parameter, i.e., AWB operation result, ALCS configuration parameter, i.e., ALCS operation result, ATF configuration parameter, i.e., ATF operation result, the second image processor is ACPU. The ACPU may include camera HALs of different image sensors, and different camera HALs are used to generate AWB operation results, ALCS operation results, and ATF operation results corresponding to corresponding image sensors.

[0070] In one example, the configuration parameters are configuration parameters of the hardware module for the A5 algorithm. The A5 algorithm includes the following algorithms: AE, AF, AWB, ALSC, and ATF. The structure of the camera hardware system in the electronic device is as follows: Figure 2As shown, it includes: MCU, ISP, ACPU. MCU is used to control ISP, ACPU includes camera HAL, wherein ISP obtains AE configuration parameters or AF configuration parameters according to image data respectively input by AE algorithm and AF algorithm. Camera HAL in ACPU obtains AWB configuration parameters, ALSC configuration parameters and ATF configuration parameters according to image data respectively input by AWB algorithm, ALSC algorithm and ATF algorithm.

[0071] In some embodiments, the first image processor (ACPU) is used to control the at least one image sensor, and the second image processor (MCU) is used to pre-process the data collected by the image sensor; and the determining of the configuration parameter set corresponding to the image sensor further includes:

[0072] S1011. Process the second frame image by the second image processor to generate second configuration parameters corresponding to the second frame image;

[0073] S1012: Send the second configuration parameters and metadata of the first frame of image acquired by the image sensor to the first image processor;

[0074] S1013: Processing metadata of the first frame of image by the first image processor to generate first configuration parameters corresponding to the second frame of image;

[0075] S1014. Store, by the first image processor, second configuration parameters corresponding to the second frame image and first configuration parameters corresponding to the first frame image in a cache subspace corresponding to the image sensor in a first cache space.

[0076] For the first frame image, after the statistical information of different functions is determined, the second image processor can only determine the second configuration parameters based on the statistical information of the second function (the function corresponding to the second configuration parameters), and the second image processor sends the second configuration parameters and the statistical information of the first function (the function corresponding to the first configuration parameters) to the first image processor, and the first image processor determines the first configuration parameters based on the statistical information of the first function. When the second image processor determines the second configuration parameters based on the statistical information of the second function, it does not obtain the metadata of the first frame image. At this time, the second configuration parameters are determined based on the metadata of the first frame image and the statistical information of the second function. When the second image processor sends the statistical information of the first function to the first image processor, it can be sent to the first image processor together with the metadata of the first frame image, and the first image processor determines the first configuration parameters based on the metadata of the first frame image and the statistical information of the first function. The second frame image may be the previous frame image of the first frame image, for example: if the second frame image is the mth frame image, then the first frame image is the m+1th frame image. For another example: if the second frame image is the N-1th frame image, then the first frame image is the Nth frame image

[0077] Here, the second configuration parameter may include: AE configuration parameter, i.e., AE operation result, AF configuration parameter, i.e., AF operation result; an ISP is running on the second image processor, wherein the AE configuration parameter and the AF configuration parameter are used to configure the AE control module and the AF control module in the ISP; the first configuration parameter may include: AWB configuration parameter, i.e., AWB operation result, ALCS configuration parameter, i.e., ALCS operation result, ATF configuration parameter, i.e., ATF operation result; the camera HAL running on the ACPU may include: AWB module, ALSC module, and ATF module. The ISP may send the second configuration parameter and the statistical information of the first function to the camera HAL on the ACPU through the communication interface between the MCU and the ACPU.

[0078] In one example, the configuration parameters are configuration parameters of the hardware module for the A5 algorithm. The A5 algorithm includes the following algorithms: AE, AF, AWB, ALSC, and ATF. The structure of the camera hardware system, i.e., the image processing system, in the electronic device is as follows: Figure 2 As shown, it includes: MCU, ISP, ACPU. MCU is used to control ISP, ACPU includes camera HAL, wherein ISP obtains AE configuration parameters or AF configuration parameters according to image data respectively input by AE algorithm and AF algorithm. Camera HAL in ACPU obtains AWB configuration parameters, ALSC configuration parameters and ATF configuration parameters according to image data respectively input by AWB algorithm, ALSC algorithm and ATF algorithm.

[0079] like Figure 3As shown, the ISP firmware calculates the AE configuration parameters and AF configuration parameters corresponding to the N-1th frame image based on the statistical information of AE and AF, and sends the AE configuration parameters, AF configuration parameters, AWB, ALSC, ATF statistical information and the metadata of the N-1th frame image to the camera HAL in the ACPU. The camera HAL adds the AE configuration parameters and AF configuration parameters corresponding to the N-1 frame image to the configuration parameter group corresponding to the N-1 frame, and inputs them into the AWB algorithm, ALSC algorithm, and ATF algorithm respectively according to the metadata of the N-1th frame image to obtain the AWB configuration parameters, ALSC configuration parameters, and ATF configuration parameters.

[0080] In an embodiment of the present application, in a process of determining configuration data once, some configuration parameters of a previous frame image are determined, and some configuration parameters of a current frame image are determined, so that in a process of determining configuration parameters once, the complete configuration parameters are not determined. Through multiple configuration parameter determination processes, the complete configuration parameters are determined once, and the complete configuration parameters are stored as a parameter cluster.

[0081] In one example, the first configuration parameters include AE ​​configuration parameters and AF configuration parameters, and the second configuration parameters include AWB configuration parameters, ALSC configuration parameters, and ATF configuration parameters; for the m-th frame image, after the ISP determines the statistical information of each algorithm of AE, AF, AWB, ALSC, and ATF, based on the statistical information of AE and AF and the metadata of the m-1-th frame image, determines the AF configuration parameters and AF configuration parameters corresponding to the m-1-th frame image, and determines the metadata of the m-th frame image, and sends the AF configuration parameters and AF configuration parameters corresponding to the m-th frame image and the metadata, AWB, ALSC, and ATF statistical information of the m-th frame image to the ACPU, and the ACPU determines the AWB configuration parameters, ALSC configuration parameters, and ATF configuration parameters of the m-th frame image based on the metadata of the m-th frame image and the statistical information of AWB, ALSC, and ATF, and sends the AF configuration parameters and AF configuration parameters corresponding to the m-1-th frame image and the AWB configuration parameters, ALSC configuration parameters, and ATF configuration parameters of the m-th frame image; For the m+1th frame image, after the ISP determines the statistical information of each algorithm of AE, AF, AWB, ALSC and ATF, it determines the AF configuration parameters and AF configuration parameters corresponding to the m-th frame image based on the statistical information of AE and AF and the metadata of the m-th frame image, and determines the metadata of the m+1-th frame image, and sends the AF configuration parameters and AF configuration parameters corresponding to the m+1-th frame image and the metadata, AWB, ALSC and ATF statistical information of the m+1-th frame image to the ACPU, and the ACPU determines the AWB configuration parameters, ALSC configuration parameters and ATF configuration parameters of the m+1-th frame image based on the metadata of the m+1-th frame image and the statistical information of AWB, ALSC and ATF, and sends the AF configuration parameters and AF configuration parameters corresponding to the m-th frame image and the AWB configuration parameters, ALSC configuration parameters and ATF configuration parameters of the m+1-th frame image. At this time, the AF configuration parameters and AF configuration parameters, AWB configuration parameters, ALSC configuration parameters and ATF configuration parameters corresponding to the m-th frame image are stored as a cluster.

[0082] In an example, a complete configuration parameter structure is as follows:

[0083]

[0084]

[0085] In some embodiments, the step of selecting a candidate configuration parameter from the configuration parameter set in S102 includes:

[0086] When a first time condition is met, a candidate configuration parameter is selected from the configuration parameter set; wherein the first time condition includes at least one of the following:

[0087] The first image sensor corresponding to the configuration parameter set is turned off;

[0088] The image sensor for image acquisition is switched from the first image sensor to a second image sensor, wherein the first image sensor and the second image sensor are different image sensors in the at least one image sensor;

[0089] A time point determined based on a set cycle is reached.

[0090] In the embodiment of the present application, the timing when the electronic device selects a candidate configuration parameter from the configuration parameter set includes at least one of the following:

[0091] Timing 1: the first image sensor corresponding to the configuration parameter set is turned off;

[0092] Opportunity 2: the image sensor for image acquisition is switched from the first image sensor to the second image sensor;

[0093] Timing 3: Reaching a time point determined based on a set cycle.

[0094] For opportunity one, when the camera is normally turned off, the camera resources are normally released. Before the 5A business program is released, the 5A algorithm result data marked as Flag-Valid, that is, the valid configuration parameters, are stored in the file as hot data.

[0095] For opportunity two, when camera switching occurs, such as simple switching, SAT scaling conversion, etc., the 5A algorithm result data marked as Flag-Valid position is extracted as hot data into the multi-camera hot data concentration area, and stored in the file as hot data.

[0096] For opportunity three, combined with the validity period of thermal data, the 5A algorithm result data marked as Flag-Valid is regularly extracted as thermal data to the multi-camera thermal data concentration area, and stored in the file as thermal data. In order to solve the problem of untimely data preservation during Crash anomalies, regular storage is performed and the validity of thermal data is judged during loading.

[0097] Here, the multi-camera thermal data concentration area can be understood as a centralized area for storing candidate configuration parameters of multiple cameras.

[0098] In the embodiment of the present application, the electronic device can select the 5A algorithm result data marked as Flag-Valid from the configuration parameter set according to the effective hot data selection strategy. The effective hot data selection strategy can be set based on the validity identification, timestamp and other information of the configuration parameter.

[0099] In some embodiments, the step of selecting a candidate configuration parameter from the configuration parameter set in S102 includes:

[0100] The configuration parameter with the shortest generation time among the valid configuration parameters in the configuration parameter set is determined as a candidate configuration parameter of the configuration parameter set.

[0101] In the embodiment of the present application, when the electronic device selects candidate configuration parameters, it checks the validity of each configuration parameter one by one from the configuration parameter combined with the configuration parameter set corresponding to the last frame image in the configuration parameters included, and determines the first valid configuration parameter as the candidate configuration parameter. Here, when generating a configuration parameter, a validity identifier corresponding to the configuration parameter can be generated.

[0102] In one example, the configuration parameter set corresponding to the last frame of the configuration parameter set is the configuration parameter corresponding to the Nth frame. At this time, the configuration parameter is invalid, and the configuration parameter corresponding to the N-1th frame is searched. If the N-1th frame configuration parameter is valid, the N-1th frame configuration parameter is determined as the candidate configuration parameter. Here, the reason why the Nth frame configuration parameter is invalid may include: the metadata of the Nth frame is invalid, or it may include not including complete configuration parameters.

[0103] In some embodiments, the electronic device further performs the following processing:

[0104] For each configuration parameter in the configuration parameter set, the validity of the configuration parameter is judged according to the validity flag of the configuration parameter, and the validity flag is related to the validity of the metadata of the image corresponding to the corresponding configuration parameter and whether the configuration parameter is complete.

[0105] The validity identification is determined based on the validity of the image metadata required by the corresponding configuration parameters and the integrity of the corresponding configuration parameters. For example, the configuration parameter set of the image sensor X includes the configuration parameters corresponding to the m-th frame image, the m+1-th frame image, and the m+2-th frame image, wherein the configuration parameters corresponding to one frame image include: configuration parameter 1, configuration parameter 2, and configuration parameter 3. Taking the configuration parameters corresponding to the m-th frame image as an example, the validity identification of the configuration parameters corresponding to the m-th frame image is determined based on the validity of the metadata of the m-th frame image and whether the configuration parameters include the three configuration parameters of configuration parameter 1, configuration parameter 2, and configuration parameter 3. If the metadata of the m-th frame image is valid and the configuration parameters completely include parameter 1, parameter 2, and parameter 3, then the validity identification of the configuration parameters indicates that the configuration parameters are valid; if the metadata of the m-th frame image is invalid or the configuration parameters do not completely include parameter 1, parameter 2, and parameter 3, then the validity identification of the configuration parameters indicates that the configuration parameters are invalid.

[0106] In the embodiment of the present application, the configuration parameters correspond to the validity mark, i.e., the validity identifier, frame number, timestamp, and the calculation result of the 5A algorithm for each frame of the image. The electronic device checks whether valid metadata can be obtained, whether the 5A algorithm is successfully processed, i.e., whether it includes complete configuration parameters, etc., and records the 5A warm data validity mark. When each frame of the 5A algorithm result is stored, the complete and valid data results are checked one by one in reverse order starting from frame N-1. The first valid 5A algorithm result checked is marked with the position Flag-Valid.

[0107] When the camera is turned off or switched away or periodically, the data marked as Flag-Valid is stored in a file as hot data, which can be used as hot data at the next startup and input into the 5A algorithm.

[0108] In some embodiments, storing the at least one candidate configuration parameter at S102 includes:

[0109] Storing the at least one candidate configuration parameter in a configuration parameter file, and storing the configuration parameter file in a set file path;

[0110] The at least one candidate configuration parameter is cached in the second cache space.

[0111] In an embodiment of the present application, after determining at least one candidate configuration parameter, the electronic device stores the at least one candidate configuration parameter in a file to a set file path, and stores the at least one candidate configuration parameter in a second cache space, thereby storing the at least one candidate configuration parameter in multiple ways.

[0112] In one example, if Figure 4 As shown, the camera in the electronic device includes camera 1 and camera 2, then at least one configuration parameter group includes: configuration parameter set 1 of camera 1 and configuration parameter set 2 of camera 2. Configuration parameter set 1 includes configuration parameter 1 corresponding to image frame 1 and configuration parameter 2 corresponding to image 2, configuration parameter set 2 includes configuration parameter 3 corresponding to image frame 3 and configuration parameter 4 corresponding to image 4, the candidate configuration parameter selected from configuration parameter set 1 is configuration parameter 1, and the candidate configuration parameter selected from configuration parameter set 2 is configuration parameter 4, then configuration parameter 1 and matching parameter 2 are added to a file, and the file is stored to a set file path, and configuration parameter 1 and matching parameter 2 are cached in the cache.

[0113] In some embodiments, S103 reads the at least one candidate configuration parameter, including:

[0114] Determine a startup scenario of the target image sensor, the startup scenario including a first startup scenario and a second startup scenario, the second cache space is cleared in the first startup scenario, and the second cache space is not cleared in the second startup scenario;

[0115] The at least one candidate configuration parameter is read from the set file path or the second cache space according to the startup scenario.

[0116] In the first startup scenario, the electronic device does not have other image sensors before the target image sensor is started, which can also be called the initial startup scenario. In the second startup scenario, the electronic device has other image sensors before the target image sensor is started, so that the image sensor for collecting images is switched from other image sensors to the current target image sensor, which can also be called the image sensor switching scenario.

[0117] In the first startup scenario, before the target image sensor is started, the camera resources are released normally and the second cache space is cleared, so the electronic device reads the file from the set file path and parses the candidate configuration parameters from the read file. In the second startup scenario, before the target image sensor is started, there are other image sensors and the second cache space is not cleared, so the electronic device reads the candidate configuration parameters from the second cache space.

[0118] In some embodiments, S103 selects a target configuration parameter for configuring the target image sensor from the at least one candidate configuration parameter, including:

[0119] The target configuration parameter is selected from the at least one candidate configuration parameter according to a set configuration parameter loading strategy, wherein the configuration parameter loading strategy is related to at least one of the following information: an operating time of an adjacent image sensor and a sensor type of the image sensor.

[0120] The electronic device selects target configuration parameters for the current target image sensor from at least one candidate configuration parameter through a configuration parameter loading strategy, wherein the configuration parameter loading strategy can be constructed based on factors such as the thermal data life cycle, the operating timing of adjacent cameras, and the proximity of camera types, so as to judge whether to load its own thermal data or the thermal data of adjacent cameras in the current scene, so as to solve the problems of missing and expired thermal data when multiple cameras are started or switched. The life cycle of thermal data includes the following states: continued, expired, and default.

[0121] The configuration parameter loading strategy may also be referred to as a hot data loading strategy. If the configuration parameter is a 5A configuration parameter, the corresponding hot data loading strategy may be referred to as a 5A hot data loading strategy.

[0122] In the embodiment of the present application, the configuration parameter to be determined can be determined from the candidate configuration parameters read from the setting file path or the second cache space based on the configuration parameter loading strategy, and whether the configuration parameter to be determined is a valid configuration parameter can be determined based on the timestamp of the configuration parameter to be determined. If it is a valid configuration parameter, the configuration parameter to be determined is determined as the target configuration parameter. If the configuration parameter to be determined is an invalid configuration parameter, a new configuration parameter to be determined can be selected from the candidate configuration parameters read from the setting file path or the second cache space based on the configuration parameter loading strategy. In the embodiment of the present application, it can also be determined based on the startup scenario to read the candidate configuration parameters from the setting file path or the second cache space, select the configuration parameter to be determined from the read candidate configuration parameters based on the configuration parameter loading strategy, and determine whether the configuration parameter to be determined is the target configuration parameter based on the timestamp of the configuration parameter to be determined.

[0123] In the embodiment of the present application, the configuration parameter loading strategy follows at least one of the following principles:

[0124] Principle 1:

[0125] According to the loading order and validity check of the camera's own 5A thermal data file, the adjacent camera's 5A thermal data cache, and the adjacent camera's 5A thermal data file, the first valid configuration parameters obtained are the target configuration parameters.

[0126] Principle 2:

[0127] When camera-x is started, if it needs to use the thermal data of the neighboring camera-y, it will borrow the thermal data of the neighboring camera as its own startup thermal data according to the similarity of the camera type. The priority order of loading neighboring cameras is (not limited to) ultra-wide, wide-angle, telephoto, and macro cameras, and the neighboring ones are prioritized upward.

[0128] In some embodiments, the step of selecting the target configuration parameter from the at least one candidate configuration parameter according to the set configuration parameter loading strategy in S103 includes:

[0129] selecting a first candidate configuration parameter from the at least one candidate configuration parameter according to the configuration parameter loading strategy;

[0130] Determining whether the state of the first candidate configuration parameter is ongoing;

[0131] If the state of the first candidate configuration parameter is not persisting, continue to select the first candidate configuration parameter from the at least one candidate configuration parameter according to the configuration parameter loading strategy, and continue to determine whether the state of the first candidate configuration parameter is persisting, until the selected first candidate configuration parameter is persisting;

[0132] If the state of the first candidate configuration parameter is ongoing, the first candidate configuration parameter is determined to be the target configuration parameter.

[0133] In the embodiment of the present application, after selecting a first candidate configuration parameter based on the configuration parameter loading strategy, it is determined which state of the life cycle the first candidate configuration parameter is in. If it is ongoing, the first candidate configuration parameter is considered valid, and the first candidate configuration parameter is determined as the target configuration parameter. If the state of the first candidate configuration parameter is not stored but expired or defaulted, it is determined that the first candidate configuration parameter is not used as the target configuration parameter, and the next candidate configuration parameter is searched for as the new first candidate configuration parameter until the first candidate configuration parameter with an ongoing state is found.

[0134] In one example, if the candidate configuration parameters of the current camera camera-x exist in the form of a file, after loading and parsing, the validity of the candidate configuration parameters is analyzed and the status is checked. If it is in an ongoing state, the life cycle of the candidate configuration parameters has not expired. When starting this camera, the candidate configuration parameters of camera-x itself stored in the form of a file are loaded as the target configuration parameters.

[0135] In one example, if the candidate configuration parameters corresponding to the current camera camera-x exist in the form of a file, after loading and parsing, the validity status of the candidate configuration parameters is checked. If it is expired, the life cycle of the candidate configuration parameters has ended. Then: (1) If the camera has just switched from the adjacent camera-y to the current camera-x, the candidate configuration parameters of the adjacent camera-y memory cache are loaded when the current camera is started (after the adjacent camera-y is switched, its resources will be maintained for a period of time. If it is not switched back for a long time, it will exit and destroy the resources); (2) If the adjacent camera-y is not running, when the current camera-x is started, it is determined whether the candidate configuration parameters of the adjacent camera-y are expired. If not expired, the candidate configuration parameters of camera-y are used; (3) If the candidate configuration parameters of camera-y are also expired, then the candidate configuration parameters of camera-x continue to be used.

[0136] In one example, if the candidate configuration parameters corresponding to the current camera-x are not found in the form of a file, it is determined that the candidate configuration parameters corresponding to camera-x are in a missing state, and a check is made to see whether it is the type of the adjacent camera y. Combined with its operating conditions, the choice is to extract the target configuration parameters from the candidate configuration parameters in the memory cache of camera-y. The priority order for loading adjacent cameras is (not limited to) ultra-wide, wide-angle, telephoto, and macro cameras, and the adjacent ones are prioritized and depended on upward.

[0137] The following is a further description of the parameter configuration method provided in the embodiment of the present application.

[0138] The embodiment of the present application proposes a Camera system 5A Warm Data collaborative storage and loading solution based on a heterogeneous platform, including: (1) utilizing the N-1 frame image AE / AF algorithm results carried by the N frame image Meta distributed upstream across the system, fusing the N-1 frame AWB / LSC / ATF results and caching them, designing an effective hot data selection strategy to avoid invalid hot data, pipeline asynchronous caching caused by flush instructions and abnormal crashes, locking the selected frame 5A information as hot data, and greatly reducing the complexity of round-trip matching and inter-core communication in the general solution. (2) Based on the hot data life cycle, the operating time of adjacent cameras, and the similar relationship between camera types, a set of hot data loading strategies for the 5A algorithm is constructed to decide whether to load the own hot data or the hot data of adjacent cameras in the current scene, so as to solve the problems of hot data missing and expiration when multiple cameras are started or switched.

[0139] Camera hardware systems based on heterogeneous platforms such as Figure 5 As shown, the camera system (Camera-Sys) in the camera hardware system includes: MCU and ISP. MCU controls ISP through the bus protocol (Advanced eXtensible Interface, AXI). The application system (AP-Sys) in the camera hardware system includes: ACPU. MCU and ACPU exchange information through the mailbox (Mailbox) communication bus. Among them, the control of AE / AF is sinking to the MCU side, and the AE / AF algorithm generates the calculation results. Algorithms such as AW / ALSC / ATF are deployed on the ACPU side. The statistical data calculation of a certain frame of image is run separately on the MCU side and the ACPU side at the same time. Therefore, how to cache the algorithm results such as AE / AF, AWB / ALSC / ATF obtained from the analysis of the same frame of image as subsequent hot data at the same time, how to store them centrally with multiple cameras, and how to load them under various conditions are issues that need to be further solved in the application.

[0140] In the embodiment of the present application, ISP is used to implement the image signal processing ISP algorithm, and the pre-processing of the image includes 5A algorithm, front-end (FE) processing, and middle-end (ME) processing. The ISP processing of the image also includes BE processing. Among them, different ends are used to perform different processing on the image. In addition to ACPU, AP Sys can also run application software programs for image processing.

[0141] The process of obtaining 5A thermal data is as follows: Figure 6 The following 5 steps are shown:

[0142] S1. Generation: The 5A algorithm deployed on different processors generates a calculation result based on the relevant input parameters after each frame of image is generated.

[0143] S2, Acquisition: Acquire the calculation result from the 5A algorithm and transmit it to the ACPU side through the data path.

[0144] S3, Cache: On the ACPU side, the 5A result data of the same camera is matched according to the image frame number and temporarily stored in the cache area.

[0145] S4, Centralized: For one or more cameras, valid data is regularly selected from the cache and stored centrally.

[0146] S5. Storage: Periodically or when the camera is turned off, the centralized 5A valid data is stored in a file in a certain format as hot data.

[0147] In the Camera system based on heterogeneous platforms, the 5A Warm Data software transmission channel is as follows: Figure 7 As shown, in Figure 7 It includes: hardware layer, firmware layer, kernel layer, user layer, and camera hardware abstraction layer (Camera HAL). Among them, the Hardware layer and Firmware layer belong to MCU, and the Kernel layer, User layer, and CameraHAL layer belong to ACPU.

[0148] For the Hardware layer, there are many hardware modules in the ISP hardware. In addition to flowing to the next level of hardware, the image stream output by each sensor will also be input to the 5A statistical module, such as the statistical modules corresponding to AWB, AE, AF, ALSC, and ATF. Each statistical module periodically outputs Stats statistical information according to the sensor frame rate and provides it to the 5A algorithm to calculate the 5A operation results.

[0149] For the Firmware layer, it is the control pipeline for managing the Camera hardware. AE / AF related algorithms and software control are directly controlled by the underlying MCU core. In the following description, N frame represents the current running image frame. N-1 represents the previous frame image.

[0150] The ISP FW in the Firmware layer includes: IQC, module 1: AE control (AE CTL), module 2: AF control (CTL), AWB control (ATL), ALSC control (CTL). Among them, module 1: is used for AEC (automatic exposure control) logic and algorithm, processes each frame of image information, and outputs AE operation results. Module 2: is used for AFC (auto focus control) logic and algorithm, processes each frame of image information, and outputs AF operation results.

[0151] The firmware layer also includes module 3: FE MetaBuffer, which puts the N-1 frame AE / AF operation results into the FE MetaBuffer corresponding to the N frame image. It is uploaded to the AP layer along with the N frame MetaBuffer. The MCU interacts with the AP side for uplink data through Metadata.

[0152] For the Kernel layer: it is the Android software Linux kernel. The Kernel layer includes module 4, which is loaded with driver devices related to ISP, sensor, etc., used to realize the interaction between the Firmware layer and the User layer, and can realize the upload of data such as MetaBuffer.

[0153] For the User layer: its HW Layer layer realizes the interaction between the Kernel and the User layer. Among them, the HW Layer is used for ISP services for information management. The User layer includes: Module 5, which includes software modules such as Message Manager and ISP service, and can realize the upload of data such as MetaBuffer.

[0154] Camera HAL, belonging to AP, is used to control the specific business logic of Camera, including AWB, ALSC, ATF algorithms and data processing.

[0155] Camera HAL includes the following modules 6 to 13. Among them,

[0156] Module 6: Receives MetaBuffer from the MCU through SourceNode 0 and sends it to module 7 in a timely manner. At this time, it will wait until module 7 has finished parsing the Meta data before sending other data to modules 8 to 12. Different cameras may correspond to different source nodes.

[0157] Module 7, MetaDataNode_0: Receives N frames of MetaBuffer, and stores the AE / AF algorithm results of N-1 frames in the shared data space MetaTable after analysis for subsequent use by other modules. AE / AF / AWB / ALSC / ATF Node can access.

[0158] Module 8, i.e. AF node (afNode_0), module 9, i.e. AE node (aeNode_0): As SouceNode dispatches N frame data, AE / AF Node accesses MetaTable, obtains AE / AF algorithm results of N-1 frame respectively, and fills them back into a RingBuffer shared by 5A, corresponding to the data position of N-1 frame, referring to the storage position of frame number N-1AE / AF in RingBuffer of module 13. That is, module 8 and module 9 receive N frame FE MetaBuffer, take out AE / AF algorithm results of N-1 frame, and fill the data into the RingBuffer position corresponding to N-1 frame.

[0159] Module 10 is the ATF node (atfNode_0), module 11 is the ALSC node (alscNode_0), and module 12 is the AWB node (aw bNode_0): As SouceNode dispatches each frame of data, after the AWB / ATF / ALSC algorithm outputs the result: on the one hand, the corresponding image frame number is cached in the 5A public RingBuffer queue, and when the frame is stored, it is cached in real time with the frame; on the other hand, it is sent to the MCU, and the frame number is matched with the AE / AF algorithm result to form a cluster of information, and the ISP hardware pipeline configuration parameters are constructed, and the 5A-related hardware module modules are configured to process the image stream of subsequent frames. When each subsequent frame of image is generated, in the FE stage of the ISP process, the MCU uploads the Meta data and the output result of the previous frame of AE / AF algorithm. Modules 12, 11, and 10 store the calculation results of the AWB / ALSC / ATF algorithm of the Nth frame image in the Ring-Buffer position corresponding to the N frame.

[0160] Module 13: RingBuffer for storing 5A operation results. The calculation results of the AWB / ALSC / ATF algorithm for the Nth frame image are stored in the Ring-Buffer position corresponding to the Nth frame. Therefore, from the perspective of the path, for the same frame image, the calculation results of the AWB / ALSC / ATF algorithm are always stored in the Ring-Buffer earlier than the data results of the AE / AF algorithm.

[0161] The processCaptureRequest in the core provides image information for image processing when the image sensor captures an image.

[0162] Cache 5A calculation results when the camera is running

[0163] like Figure 8 As shown, the camera caches the 5A operation results when a single camera is running.

[0164] like Figure 8 801 is a 5A result data structure corresponding to a frame of image (schematic), 5A warm data validity mark, frame number, timestamp, and calculation result of 5A algorithm for each frame of image.

[0165] The Cameral HAL business software will check whether it can obtain valid Meta, whether the 5A algorithm is processed successfully, that is, whether the calculation results of each algorithm in the 5A algorithm are obtained, etc., and record the validity mark of the 5A warm data.

[0166] In addition, the Flush command is downloaded before the camera is turned off, and data is still sent before the ISP stops. The pipeline on the AP side has begun to prepare for destruction, and it is necessary to judge the integrity of the 5A algorithm result data of subsequent frames. If the MetaBuffer does not contain the AE / AF algorithm result, or the operation result is abnormal, the 5A warm data validity mark will be recorded as false.

[0167] Figure 8 802 is the 5A operation result of continuous multi-frame images. The 5A operation results of continuous m frames of images are recorded in real time. Through this data content, the 5A operation result of each frame of image is clearly recorded as the data source for subsequent thermal data selection.

[0168] Figure 8 803 is the RingBuffer caching the 5A operation result. This RingBuffer is used as a carrier for storing the 5A operation result. For each camera running in real time, it continuously fills the N-1th frame AE / AF data carried by the current Nth frame FE MetaBuffer back into the RingBuffer corresponding to the N-1th frame.

[0169] Effective hot data selection strategy

[0170] 5A warm data contains validity mark, frame number, timestamp, and calculation result of 5A algorithm for each frame. CameralHAL business software will check whether valid Meta can be obtained, whether 5A algorithm is successfully processed, etc., and record the validity mark of 5A warm data. When storing each frame of 5A algorithm result, it will check the complete and valid data results one by one from frame N-1 in reverse order. The first valid 5A algorithm result checked will be marked as Flag-Valid.

[0171] When the camera is turned off or switched away or periodically, the data marked as Flag-Valid is stored in a file or cache as hot data, which can be used as hot data at the next startup and input into the 5A algorithm.

[0172] Multi-camera 5A Warm Data storage method

[0173] like Fig. 9 , when multiple cameras are running, select to store 5A operation results. When multiple cameras are running, each camera, such as camera x and y, stores 5A thermal data. Figure 8 .

[0174] The Camera HAL system will select 5A operation results for all working cameras according to validity, latest time, etc. at the following times. When the camera is turned off or switched away or regularly, it will check the complete and valid data results one by one starting from frame N-1, and select the 5A results in a timely manner and concentrate them in a data structure, and store or update them in a file.

[0175] There are several ways to select and store the 5A algorithm results as hot data:

[0176] (a) When the camera is turned off:

[0177] When the camera is shut down normally, the camera resources are released normally. Before the 5A business program is released, the 5A algorithm result data marked as Flag-Valid is stored in the file as hot data according to the "valid hot data selection strategy".

[0178] (b) Camera switching:

[0179] When camera switching occurs, such as simple switching, SAT scaling conversion, etc., according to the "effective hot data selection strategy", the 5A algorithm result data marked as Flag-V alid position is extracted as hot data to the "multi-camera hot data concentration area" and stored in the file as hot data.

[0180] (c) Regularly:

[0181] Combined with the validity period of the hot data, the 5A algorithm result data marked as Flag-Valid is extracted to the "Multi-camera Hot Data Central Area" as hot data according to the results of the "Effective Hot Data Selection Strategy" and stored in the file as hot data. In order to solve the problem of data not being saved in time when Crash is abnormal, the hot data validity is judged regularly when it is stored and loaded.

[0182] like Fig.10 The following is a simplified block diagram of the 5A thermal data loading process. The 5A thermal data loading method is divided into two methods according to the thermal data storage method:

[0183] Hot data is stored in the form of files

[0184] When a camera is turned on, in the initialization phase before streaming, the camera software reads the thermal data file, loads it into the memory space, and parses the 5A algorithm thermal data through the corresponding data storage structure, corresponding to algorithms such as AE / AF / AWB / ALSC / ATF. The camera ID and thermal data validity check match, and then (input or send) configure it to the algorithm for the initialization process of the 5A algorithm.

[0185] Hot data is stored in cache

[0186] When the camera has multiple cameras, if the thermal data of camera-x is expired after a long time when it is initialized, the adjacent camera-y may still be working or just switched to the background. In this case, the 5A operation stored in the cache of camera-y can be used as the 5A thermal data of camera-x for the initialization processing of its 5A algorithm after loading strategy analysis if it meets the requirements.

[0187] Camera 5A hot data loading strategy

[0188] Based on the hot data life cycle, the operating time of adjacent cameras, and the similarity of camera types, a set of hot data loading strategies for the 5A algorithm is constructed to determine whether to load own hot data or hot data of adjacent cameras in the current scene to solve problems such as missing and expired hot data when multiple cameras are started or switched.) The basic principles followed by the 5A hot data loading strategy are:

[0189] Principle 1:

[0190] The order of loading file selection, according to validity check, is the camera's own 5A thermal data file, the adjacent camera's 5A thermal data cache, and the adjacent camera's 5A thermal data file. The search will stop when a valid value is obtained for the first time.

[0191] Principle 2:

[0192] When camera-x is started, if it needs to use the thermal data of the neighboring camera-y, it will borrow the thermal data of the neighboring camera as its own startup thermal data according to the similarity of the camera type. The priority order of loading neighboring cameras is (not limited to) ultra-wide, wide-angle, telephoto, and macro cameras, and the neighboring ones are prioritized upward.

[0193] (2) 5A hot data loading strategy classification and segmentation

[0194] Before analyzing this strategy, the 5A thermal data corresponding to the camera is divided into three states: existing, missing, and expired. Fig.11 , simplified block diagram of 5A hot data loading strategy, analyzes the camera 5A hot data loading strategy from a process perspective. Fig.11 In the example, the hot data source (1) exists in the form of a file. Fig.11 In the example, the hot data source (2) exists in the form of cache.

[0195] Continuing Status

[0196] For example, referring to the first basic principle of loading strategy, if the thermal data source (1) corresponding to the current camera camera-x exists in the form of a file, after loading and parsing, the loading strategy center analyzes the 5A thermal data validity check status. If it is in the ongoing state, the 5A thermal data life cycle has not expired. When starting this camera, the 5A thermal data stored in the camera-x itself is loaded, that is, the 5A thermal data stored in the thermal data source (1) corresponding to the camera-x.

[0197] Expired status

[0198] For example, referring to the first basic principle of loading strategy, if the current camera camera-x corresponds to the thermal data source (1) in the form of a file, after loading and parsing, the 5A thermal data validity check status is performed. If it is expired, the 5A thermal data life cycle has ended. The loading strategy center will analyze:

[0199] (a) If the camera has just switched from neighboring camera-y to current camera-x, the hot data source (2) in the memory cache of neighboring camera-y is loaded when the current camera is started. (After the neighboring camera-y is switched, its resources will be maintained for a period of time. If it is not switched back for a long time, it will exit and destroy the resources.)

[0200] (b) If the neighboring camera-y is not running, when starting the current camera-x, it is determined whether the heat data source (1) of the neighboring camera-y is expired. If it is not expired, the heat data source (1) of camera-y is used.

[0201] (c) If it is also expired, continue to use camera-x’s own thermal data source (1).

[0202] Missing state

[0203] For example, if the corresponding thermal data source (1) of the current camera-x does not exist in the form of a file, the 5A thermal data corresponding to camera-x is determined to be missing. The second basic principle of the loading strategy will be referred to to check whether the camera y is of the same type and, based on its operation, choose to extract thermal data from the thermal data source (2) in the memory cache of camera-y, or further load the thermal data source (1) corresponding to camera-y.

[0204] The priority order for loading adjacent cameras is (but not limited to) ultra-wide, wide-angle, telephoto, and macro cameras, and the adjacent ones are prioritized upward.

[0205] In the parameter configuration method provided in the embodiment of the present application:

[0206] On the one hand, (1) the output results of the AWB / ATF / ALSC algorithms are cached in the RingBuffer array with the corresponding image frame number, and then the AE / AF algorithm results of the previous frame carried by the Meta are backfilled into the data structure corresponding to 5A in the buffer RingBuffer according to the frame number, and updated in real time with the frame. (2) Valid hot data selection strategy: The 5A business software will check whether valid Metadata can be obtained and whether the algorithm is successfully processed, and remove the configuration parameters of invalid frames that have been cached; download the Flush instruction before the camera is turned off, and there is still data sent before the ISP stops. The pip eline on the AP side has begun to prepare for destruction, and it is necessary to judge the integrity of subsequent frame data; in order to solve the problem of untimely data preservation during Crash exceptions, regular storage is done and judgment is made during loading. Based on this, with the help of the valid hot data selection strategy, the 5A configuration parameters that are closest to the current ISP hardware operation are locked, and the position is marked as Flag-Valid. (3) When the camera is turned off, a cluster of 5A data marked as Flag-Valid in the RingBuffer is Warm Data, which is stored in the specified file together with other hot data.

[0207] On the other hand, (1) the 5A thermal data corresponding to the camera has the following states: existing, missing, expired, etc. (2) The 5A thermal data corresponding to the current camera -x is in the existing state, and its life cycle has not expired. When starting this camera, the 5A thermal data stored in camera -x itself is loaded. (3) The 5A thermal data corresponding to the current camera -x is expired: if you have just switched from the adjacent camera -y to this camera -x, load the thermal data in the memory cache of the adjacent camera -y when starting this camera; if the adjacent camera -y is not running, when starting this camera -x, determine whether the thermal data of the adjacent camera -y is expired (if it is also expired, then continue to use the thermal data of camera -x itself). (4) If the 5A thermal data of the current camera -x is missing, use the thermal data of the adjacent camera -y when starting. (5) The priority order for loading adjacent cameras is (not limited to) ultra-wide, wide-angle, telephoto, and macro cameras, and the adjacent ones are prioritized and depend on upward.

[0208] thereby:

[0209] 1. Utilize the AE / AF algorithm results of N-1 frames of images carried by Meta of N frames of images sent upstream across the system, fuse and cache the AWB / LS C / ATF results of N-1 frames, design an effective hot data selection strategy to avoid pipeline asynchronous caching caused by invalid hot data, flush and abnormal crash, lock the selected frame 5A information as hot data, and greatly reduce the complexity of round-trip matching and inter-core communication in the general solution.

[0210] 2. Based on the hot data life cycle, the operating time of adjacent cameras, and the similarity of camera types, a hot data loading strategy for the 5A algorithm is constructed to decide whether to load its own hot data or the hot data of adjacent cameras in the current scene to solve problems such as missing and expired hot data when starting or switching multiple cameras.

[0211] An electronic device according to an embodiment of the present application, such as Fig.12 As shown, the electronic device 1200 includes:

[0212] The determining unit 1201 is configured to determine, for each image sensor of at least one image sensor, a configuration parameter set corresponding to the image sensor, the configuration parameter set being generated by an image processor based on an image acquired by the corresponding image sensor, the configuration parameter set including at least two configuration parameters, and different configuration parameters corresponding to different frame images;

[0213] A first selection unit 1202 is configured to select, for each configuration parameter set, a candidate configuration parameter from the configuration parameter set to determine at least one candidate configuration parameter; the candidate configuration parameter is a valid configuration parameter in the configuration parameter set;

[0214] The storage unit 1203 is configured to store the at least one candidate configuration parameter;

[0215] The second selection unit 1204 is configured to read the at least one candidate configuration parameter and select a target configuration parameter from the at least one candidate configuration parameter read when the target image sensor is started, wherein the target configuration parameter is used by the image processor to process the target image data collected by the target image sensor, and the target image sensor is one of the at least one image sensor.

[0216] In some embodiments, the determining unit 1201 is further configured to:

[0217] When the image sensor captures a first frame of image, a first configuration parameter corresponding to the first frame of image is stored corresponding to a first frame identifier, and a second configuration parameter corresponding to a second frame of image captured by the image sensor is stored corresponding to the second frame identifier, wherein the first frame identifier is used to identify the first frame of image, and the second frame identifier is used to identify the second frame of image, and the second frame of image is a previous frame of image of the first frame of image;

[0218] When the image sensor captures a third frame of image, the first configuration parameter corresponding to the third frame of image is stored corresponding to the third frame identifier, and the second configuration parameter corresponding to the first frame of image is stored corresponding to the first frame identifier, the third frame of image is a frame of image subsequent to the first frame of image, and the third frame identifier is used to identify the third frame of image;

[0219] The first configuration parameter corresponding to the first frame identifier and the second configuration parameter corresponding to the first frame identifier constitute the configuration parameter corresponding to the first frame image in the configuration parameter set;

[0220] The configuration parameters include: the first configuration parameters and the second configuration parameters, the first configuration parameters are generated by a first image processor, the second configuration parameters are generated by a second image processor, and the configuration parameter set corresponding to the image sensor is determined.

[0221] In some embodiments, the determining unit 1201 is further configured to:

[0222] Processing the second frame image by the second image processor to generate second configuration parameters corresponding to the second frame image;

[0223] Sending the second configuration parameters and metadata of the first frame of image captured by the image sensor to the first image processor;

[0224] Processing the metadata of the first frame of image by the first image processor to generate first configuration parameters corresponding to the second frame of image;

[0225] The second configuration parameter corresponding to the second frame image and the first configuration parameter corresponding to the first frame image are stored in the cache subspace corresponding to the image sensor in the first cache space by the first image processor;

[0226] The first image processor (ACPU) is used to control the at least one image sensor, and the second image processor (M CU) is used to pre-process the data collected by the image sensor.

[0227] In some embodiments, the first selection unit 1202 is further configured to:

[0228] When a first time condition is met, a candidate configuration parameter is selected from the configuration parameter set; wherein the first time condition includes at least one of the following:

[0229] The first image sensor corresponding to the configuration parameter set is turned off;

[0230] The image sensor for image acquisition is switched from the first image sensor to a second image sensor, wherein the first image sensor and the second image sensor are different image sensors in the at least one image sensor;

[0231] A time point determined based on a set cycle is reached.

[0232] In some embodiments, the first selection unit 1202 is further configured to:

[0233] The configuration parameter with the shortest generation time among the valid configuration parameters in the configuration parameter set is determined as a candidate configuration parameter of the configuration parameter set.

[0234] In some embodiments, the first selection unit 1202 is further configured to:

[0235] For each configuration parameter in the configuration parameter set, the validity of the configuration parameter is judged according to the validity flag of the configuration parameter, and the validity flag is related to the validity of the metadata of the image corresponding to the corresponding configuration parameter and whether the configuration parameter is complete.

[0236] In some embodiments, the storage unit 1203 is further configured to:

[0237] Storing the at least one candidate configuration parameter in a configuration parameter file, and storing the configuration parameter file in a set file path;

[0238] The at least one candidate configuration parameter is cached in the second cache space.

[0239] In some embodiments, the second selection unit 1204′ is further configured to:

[0240] Determine a startup scenario of the target image sensor, the startup scenario including a first startup scenario and a second startup scenario, the second cache space is cleared in the first startup scenario, and the second cache space is not cleared in the second startup scenario;

[0241] The at least one candidate configuration parameter is read from the set file path or the second cache space according to the startup scenario.

[0242] In some embodiments, the second selection unit 1204 is further configured to:

[0243] The target configuration parameter is selected from the at least one candidate configuration parameter according to a set configuration parameter loading strategy, wherein the configuration parameter loading strategy is related to at least one of the following information: an operating time of an adjacent image sensor and a sensor type of the image sensor.

[0244] In some embodiments, the second selection unit 1204 is further configured to:

[0245] selecting a first candidate configuration parameter from the at least one candidate configuration parameter according to the configuration parameter loading strategy;

[0246] Determining whether the state of the first candidate configuration parameter is ongoing;

[0247] If the state of the first candidate configuration parameter is not ongoing, continue to select a second candidate configuration parameter from the at least one candidate configuration parameter according to the configuration parameter loading strategy, and use the second candidate configuration parameter as a new first candidate configuration parameter, and continue to determine whether the state of the first candidate configuration parameter is ongoing, until the selected first candidate configuration parameter is ongoing;

[0248] If the state of the first candidate configuration parameter is ongoing, the first candidate configuration parameter is determined to be the target configuration parameter.

[0249] In practical applications, the above-mentioned determination unit, first selection unit, second selection unit, storage unit and other modules can be implemented by a processor located on the electronic device, specifically a combination of one or more of a central processing unit (CPU), a microprocessor (MPU), an application processor (AP), a digital signal processor (DSP), and a field programmable gate array (FPGA).

[0250] Those skilled in the art should understand that the relevant description of the above-mentioned electronic device in the embodiment of the present application can be understood by referring to the relevant description of the parameter configuration method in the embodiment of the present application.

[0251] Fig.13 An optional implementation of the present application is provided as a schematic diagram of the structure of an electronic device, such as Fig.13 As shown, the embodiment of the present application provides an electronic device 1300, including an electronic chip 1301, which can be implemented as a parameter configuration method as described in one or more embodiments above. The electronic chip can be implemented as a processor of an electronic device or as a microprocessor of a fast charging module.

[0252] An embodiment of the present application provides an electronic device, Fig.14 Another optional implementation provided in the embodiment of the present application is a structural diagram of an electronic device, such as Fig.14 As shown, an embodiment of the present application provides an electronic device 1400, including:

[0253] A processor 1401 and a storage medium 1402 storing executable instructions of the processor 801, wherein the storage medium 1402 relies on the processor 1401 to perform operations through the communication bus 803, and when the instructions are executed by the processor 1401, the parameter configuration method performed in one or more of the above embodiments is executed.

[0254] It should be noted that in actual application, the various components in the terminal are coupled together through the communication bus 1403. It is understandable that the communication bus 1403 is used to realize the connection and communication between these components. In addition to the data bus, the communication bus 1403 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, Fig.14 Various buses are labeled as communication buses 1403.

[0255] An embodiment of the present application provides a computer storage medium, wherein the computer-readable storage medium is used to store a computer program, and the computer program enables a computer to execute the steps of the parameter configuration method described in one or more of the above embodiments.

[0256] An embodiment of the present application provides a schematic structural diagram of an electronic device 1500. Fig.15 The electronic device 1500 shown includes a processor 1510. The processor 1510 is configured to:

[0257] For each image sensor in at least one image sensor, determine a configuration parameter set corresponding to the image sensor, the configuration parameter set is generated by an image processor based on an image acquired by the corresponding image sensor, the configuration parameter set includes at least two configuration parameters, and different configuration parameters correspond to different frame images;

[0258] For each configuration parameter set, selecting a candidate configuration parameter from the configuration parameter set to determine at least one candidate configuration parameter, and storing the at least one candidate configuration parameter; the candidate configuration parameter is a valid configuration parameter in the configuration parameter set;

[0259] When the target image sensor is started, the at least one candidate configuration parameter is read, and a target configuration parameter is selected from the at least one candidate configuration parameter read, the target configuration parameter being used by the image processor to process the target image data acquired by the target image sensor, the target image sensor being one of the at least one image sensor.

[0260] In the embodiment of the present application, the processor 1510 can call and run a computer program from the memory to implement the parameter configuration method in the embodiment of the present application.

[0261] Alternatively, if Fig.15 As shown, the electronic device 1500 may further include a memory 1520. The processor 1510 may call and run a computer program from the memory 1520 to implement the parameter configuration method in the embodiment of the present application.

[0262] The memory 1520 may be a separate device independent of the processor 1510 , or may be integrated into the processor 1510 .

[0263] Alternatively, if Fig.15 As shown, the electronic device 1500 may further include a transceiver 1530, and the processor 1510 may control the transceiver 1530 to communicate with other devices, specifically, to receive signals sent by other devices. Here, the transceiver may include at least two antennas.

[0264] It is understandable that the transceiver includes multiple physical paths for receiving or transmitting signals, and the physical elements on one or more physical paths for transmitting signals constitute a transmitter, wherein the MPR values ​​of each physical path in the transmitter are independent.

[0265] Understandable, Figures 13 to 15 The electronic device shown in any of the figures may also include an image sensor and an image processor. The image sensor is used to capture images, and the image processor is used to process the images captured by the image sensor.

[0266] Optionally, the electronic device 1500 can implement the corresponding processes implemented by the electronic device in each method of the embodiment of the present application, and for the sake of brevity, it will not be repeated here. It should be understood that the processor of the embodiment of the present application may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method embodiment can be completed by the hardware integrated logic circuit or software instructions in the processor. The above processor can be a general processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Arra y, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to execute, or the hardware and software modules in the decoding processor are combined to execute. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0267] It can be understood that the memory in the embodiments of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus random access memory (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0268] It should be understood that the above-mentioned memory is exemplary but not restrictive. For example, the memory in the embodiments of the present application may also be static random access memory (static RAM, SRAM), dynamic random access memory (dynamic RAM, DRAM), synchronous dynamic random access memory (synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (double data rate S DRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (enhanced SDRAM, ESDRAM), synchronous link dynamic random access memory (synch link DRAM, SLDRAM) and direct RAM bus random access memory (Direct RAM, DR RAM), etc. That is to say, the memory in the embodiments of the present application is intended to include but not limited to these and any other suitable types of memory.

[0269] An embodiment of the present application also provides a computer-readable storage medium for storing a computer program.

[0270] Optionally, the computer-readable storage medium can be applied to the electronic device in the embodiments of the present application, and the computer program enables the computer to execute the corresponding processes implemented by the electronic device in the various methods of the embodiments of the present application. For the sake of brevity, they are not repeated here.

[0271] An embodiment of the present application also provides a computer program product, including computer program instructions.

[0272] Optionally, the computer program can be applied to the electronic device in the embodiments of the present application. When the computer program runs on a computer, the computer executes the corresponding processes implemented by the electronic device in the various methods of the embodiments of the present application. For the sake of brevity, they are not described here.

[0273] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0274] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0275] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0276] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0277] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0278] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0279] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A parameter configuration method, characterized in that: The method comprises: For each image sensor in at least one image sensor, determine a configuration parameter set corresponding to the image sensor, the configuration parameter set is generated by an image processor based on an image acquired by the corresponding image sensor, the configuration parameter set includes at least two configuration parameters, and different configuration parameters correspond to different frame images; For each configuration parameter set, selecting a candidate configuration parameter from the configuration parameter set to determine at least one candidate configuration parameter, and storing the at least one candidate configuration parameter; the candidate configuration parameter is a valid configuration parameter in the configuration parameter set; When the target image sensor is started, the at least one candidate configuration parameter is read, and a target configuration parameter is selected from the at least one candidate configuration parameter read, the target configuration parameter being used by the image processor to process the target image data acquired by the target image sensor, the target image sensor being one of the at least one image sensor.

2. The method according to claim 1, characterized in that The configuration parameters include: a first configuration parameter and a second configuration parameter, the first configuration parameter is generated by a first image processor, and the second configuration parameter is generated by a second image processor, and the determining of the configuration parameter set corresponding to the image sensor includes: When the image sensor captures a first frame of image, a first configuration parameter corresponding to the first frame of image is stored corresponding to a first frame identifier, and a second configuration parameter corresponding to a second frame of image captured by the image sensor is stored corresponding to the second frame identifier, wherein the first frame identifier is used to identify the first frame of image, and the second frame identifier is used to identify the second frame of image, and the second frame of image is a previous frame of image of the first frame of image; When the image sensor captures a third frame of image, the first configuration parameter corresponding to the third frame of image is stored corresponding to the third frame identifier, and the second configuration parameter corresponding to the first frame of image is stored corresponding to the first frame identifier, the third frame of image is a frame of image subsequent to the first frame of image, and the third frame identifier is used to identify the third frame of image; The first configuration parameter corresponding to the first frame identifier and the second configuration parameter corresponding to the first frame identifier constitute the configuration parameter corresponding to the first frame image in the configuration parameter set.

3. The method according to claim 2, characterized in that The first image processor (ACPU) is used to control the at least one image sensor, and the second image processor (MCU) is used to pre-process the data collected by the image sensor; The determining of the configuration parameter set corresponding to the image sensor further includes: Processing the second frame image by the second image processor to generate second configuration parameters corresponding to the second frame image; Sending the second configuration parameters and metadata of the first frame of image captured by the image sensor to the first image processor; Processing the metadata of the first frame of image by the first image processor to generate first configuration parameters corresponding to the second frame of image; The first image processor stores the second configuration parameters corresponding to the second frame image and the first configuration parameters corresponding to the first frame image in the cache subspace corresponding to the image sensor in the first cache space.

4. The method according to claim 1, characterized in that: The selecting a candidate configuration parameter from the configuration parameter set includes: When a first time condition is met, a candidate configuration parameter is selected from the configuration parameter set; wherein the first time condition includes at least one of the following: The first image sensor corresponding to the configuration parameter set is turned off; The image sensor for image acquisition is switched from the first image sensor to a second image sensor, wherein the first image sensor and the second image sensor are different image sensors in the at least one image sensor; A time point determined based on a set cycle is reached.

5. The method according to claim 1, characterized in that The selecting a candidate configuration parameter from the configuration parameter set includes: The configuration parameter with the shortest generation time among the valid configuration parameters in the configuration parameter set is determined as a candidate configuration parameter of the configuration parameter set.

6. The method according to claim 5, characterized in that The method further comprises: For each configuration parameter in the configuration parameter set, the validity of the configuration parameter is judged according to the validity flag of the configuration parameter, and the validity flag is related to the validity of the metadata of the image corresponding to the corresponding configuration parameter and whether the configuration parameter is complete.

7. The method according to claim 1, characterized in that The storing of the at least one candidate configuration parameter comprises: Storing the at least one candidate configuration parameter in a configuration parameter file, and storing the configuration parameter file in a set file path; The at least one candidate configuration parameter is cached in the second cache space.

8. The method according to claim 7, characterized in that The reading of the at least one candidate configuration parameter comprises: Determine a startup scenario of the target image sensor, the startup scenario including a first startup scenario and a second startup scenario, the second cache space is cleared in the first startup scenario, and the second cache space is not cleared in the second startup scenario; The at least one candidate configuration parameter is read from the set file path or the second cache space according to the startup scenario.

9. The method according to any one of claims 1 to 7, characterized in that: Selecting a target configuration parameter for configuring a target image sensor from the at least one candidate configuration parameter comprises: The target configuration parameter is selected from the at least one candidate configuration parameter according to a set configuration parameter loading strategy, wherein the configuration parameter loading strategy is related to at least one of the following information: an operating time of an adjacent image sensor and a sensor type of the image sensor.

10. The method according to claim 9, characterized in that The selecting the target configuration parameter from the at least one candidate configuration parameter according to the set configuration parameter loading strategy includes: selecting a first candidate configuration parameter from the at least one candidate configuration parameter according to the configuration parameter loading strategy; Determining whether the state of the first candidate configuration parameter is ongoing; If the state of the first candidate configuration parameter is not ongoing, continue to select a second candidate configuration parameter from the at least one candidate configuration parameter according to the configuration parameter loading strategy, and use the second candidate configuration parameter as a new first candidate configuration parameter, and continue to determine whether the state of the first candidate configuration parameter is ongoing, until the selected first candidate configuration parameter is ongoing; If the state of the first candidate configuration parameter is ongoing, the first candidate configuration parameter is determined to be the target configuration parameter.

11. An electronic device, characterized in that: The electronic device comprises: a determining unit configured to determine, for each image sensor of at least one image sensor, a configuration parameter set corresponding to the image sensor, wherein the configuration parameter set is generated by an image processor based on an image acquired by the corresponding image sensor, and the configuration parameter set includes at least two configuration parameters, and different configuration parameters correspond to different frame images; A first selection unit is configured to select, for each configuration parameter set, a candidate configuration parameter from the configuration parameter set to determine at least one candidate configuration parameter; the candidate configuration parameter is a valid configuration parameter in the configuration parameter set; a storage unit configured to store the at least one candidate configuration parameter; The second selection unit is configured to read the at least one candidate configuration parameter and select a target configuration parameter from the at least one candidate configuration parameter read when the target image sensor is started, wherein the target configuration parameter is used by the image processor to process the target image data collected by the target image sensor, and the target image sensor is one of the at least one image sensor.

12. An electronic device, comprising a processor, characterized in that: The processor is configured to: For each image sensor in at least one image sensor, determine a configuration parameter set corresponding to the image sensor, the configuration parameter set is generated by an image processor based on an image acquired by the corresponding image sensor, the configuration parameter set includes at least two configuration parameters, and different configuration parameters correspond to different frame images; For each configuration parameter set, selecting a candidate configuration parameter from the configuration parameter set to determine at least one candidate configuration parameter, and storing the at least one candidate configuration parameter; the candidate configuration parameter is a valid configuration parameter in the configuration parameter set; When the target image sensor is started, the at least one candidate configuration parameter is read, and a target configuration parameter is selected from the at least one candidate configuration parameter read, the target configuration parameter being used by the image processor to process the target image data acquired by the target image sensor, the target image sensor being one of the at least one image sensor.

13. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps in the parameter configuration method according to any one of claims 1 to 10 are implemented.

14. A storage medium storing an executable program, characterized in that: When the executable program is executed by a processor, the parameter configuration method according to any one of claims 1 to 10 is implemented.

15. A chip, comprising a processor, characterized in that: The processor is configured to execute the parameter configuration method according to any one of claims 1 to 10.

Images