Camera module initialization method and device, computer readable medium and electronic device

Abstract

The present disclosure provides a camera module initialization method and device, a computer-readable medium and an electronic device, and relates to the field of data processing technology. The method is executed by a terminal device integrated with a camera module and a camera platform architecture CamX‑CHI, including: responding to a start-up instruction to start the camera module, obtaining identification information corresponding to the camera module, and determining the corresponding photo mode of the camera module according to the identification information; determining the function driver corresponding to the photo mode; the function driver includes a preview function driver and a photo function driver; before receiving a photo request, the camera module in the photo mode is initialized by the created preview function driver, and a preview image is displayed. The present disclosure can reduce the number of function drivers created when the camera module is started and initialized without affecting the display of the preview image of the camera module, effectively improve the efficiency of the camera module initialization, speed up the generation of preview images, and improve the response speed of the camera module.

Description

Background Art

[0002] With the continuous improvement of people's living standards, taking photos, selfies, and recording videos have become an indispensable form of entertainment in people's lives. Therefore, most existing terminal devices are integrated with camera modules. In order to ensure the refined control of the camera module by the terminal device, a camera platform architecture CamX-CHI is proposed for terminal devices using the Android operating system. All functions of the camera module are integrated in different function-driven Features.

[0003] At present, in the relevant technical solutions, the function drivers corresponding to all the functions of the camera module are created during the initialization process, but some function drivers will not be used when generating preview images, resulting in unnecessary waste of system resources, reducing the initialization speed of the camera module, reducing the efficiency of generating preview images, and causing the camera module to respond slowly. Summary of the invention

[0004] The purpose of the present disclosure is to provide a camera module initialization method, a camera module initialization device, a computer-readable medium and an electronic device, thereby improving the initialization speed of the camera module, improving the generation efficiency of the preview image, and improving the response speed of the camera module at least to a certain extent.

[0005] According to a first aspect of the present disclosure, a camera module initialization method is provided, which is executed by a terminal device integrated with a camera module and a camera platform architecture CamX-CHI, and the method includes:

[0006] In response to a start-up instruction for starting the camera module, obtaining identification information corresponding to the camera module, and determining a photographing mode corresponding to the camera module according to the identification information;

[0007] Determine a function driver corresponding to the photographing mode; the function driver includes a preview function driver and a photographing function driver;

[0008] Before receiving a photo request, the camera module in the photo mode is initialized by the created preview function driver, and a preview image is displayed.

[0009] According to a second aspect of the present disclosure, a camera module initialization device is provided, which is provided in a terminal device integrated with a camera module and a camera platform architecture CamX-CHI, and the device includes:

[0010] a photographing mode determining module, configured to respond to a start-up instruction for starting the camera module, obtain identification information corresponding to the camera module, and determine a photographing mode corresponding to the camera module according to the identification information;

[0011] A function driver determination module, used to determine the function driver corresponding to the photographing mode; the function driver includes a preview function driver and a photographing function driver;

[0012] The camera module initialization module is used to initialize the camera module in the photographing mode by using the created preview function driver before receiving a photographing request, and to display a preview image.

[0013] According to a third aspect of the present disclosure, a computer-readable medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the above method is implemented.

[0014] According to a fourth aspect of the present disclosure, there is provided an electronic device, characterized in that it includes:

[0015] Processor; and

[0016] The memory is used to store one or more programs. When the one or more programs are executed by one or more processors, the one or more processors implement the above method.

[0017] A camera module initialization method provided by an embodiment of the present disclosure can respond to the start-up instruction of the camera module, obtain the identification information of the camera module, and determine the corresponding shooting mode of the camera module according to the identification information, and then determine the preview function driver and the shooting function driver corresponding to the shooting mode, and before receiving the shooting request, the camera module in the shooting mode is initialized by the created preview function driver, so as to quickly display the preview image. On the one hand, the function driver corresponding to the shooting mode of the camera module is divided, and before receiving the shooting request, since the camera module only needs to implement the preview function, only the preview function driver is created to implement the initialization of the camera module, avoiding the creation of unnecessary shooting function drivers, effectively reducing the number of function drivers that need to be created when the camera module is initialized, improving the initialization speed of the camera module, improving the generation efficiency of the preview image, and improving the response speed of the camera module; on the other hand, since the function drivers corresponding to the same shooting mode of different types of camera modules are not the same, the corresponding shooting mode is determined by the identification information of the camera module, which can not only ensure the accuracy of the determined function driver, but also improve the efficiency of the determined function driver, and further ensure the response speed of the camera module.

[0018] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present disclosure, and together with the specification, are used to explain the principles of the present disclosure. Obviously, the drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0020] Figure 1 A schematic diagram showing an overall system architecture to which the embodiments of the present disclosure can be applied;

[0021] Figure 2 A schematic diagram showing an electronic device to which the embodiments of the present disclosure can be applied;

[0022] Figure 3 A schematic diagram of a camera platform architecture CamX-CHI to which the embodiments of the present disclosure can be applied is shown.

[0023] Figure 4 A schematic diagram schematically illustrates a flow chart of a camera module initialization method in an exemplary embodiment of the present disclosure;

[0024] Figure 5 A structural diagram schematically showing the relationship between basic components of a camera platform architecture CamX-CHI in an exemplary embodiment of the present disclosure;

[0025] Figure 6 A schematic diagram schematically illustrates a time-consuming process of starting a camera in an exemplary embodiment of the present disclosure;

[0026] Figure 7 A schematic diagram of a process for triggering the creation of a camera function driver in an exemplary embodiment of the present disclosure is schematically shown;

[0027] Figure 8 A schematic diagram schematically illustrates a startup process before optimization in an exemplary embodiment of the present disclosure;

[0028] Fig. 9 A schematic diagram schematically illustrates an optimized startup process in an exemplary embodiment of the present disclosure;

[0029] Fig.10 The following is a schematic diagram schematically showing the composition of a camera module initialization device in an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0030] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0031] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0032] Figure 1 A schematic diagram of an overall system architecture to which the embodiments of the present disclosure can be applied is shown.

[0033] like Figure 1 As shown, the overall architecture of the camera module operation method in the present disclosure may include a development framework Frameworks 110, a CamX architecture 120, a CHI-CDK 130, and a Kernel 140. The development framework Frameworks 110 is the basic environment for application (APP) development. For example, a camera application that displays images captured by the camera module can be developed through the development framework Frameworks 110. When a user triggers a photo request through a shooting button provided by the camera application, the development framework Frameworks 110 can send the photo request Request to the CamX architecture 120, and the CamX architecture 120 generates an image Result and feeds it back to the development framework Frameworks 110 to display the image in the camera application.

[0034] The CamX architecture 120 represents a set of code implementations of general functional interfaces, and the CHI-CDK 130 represents a set of code implementations of customizable requirements. The CamX architecture 120 and the CHI-CDK 130 constitute the camera platform architecture CamX-CHI in this example embodiment.

[0035] The CamX architecture 120 maintains communication with the Kernel kernel 140 through the V4L2 framework, and maintains interaction with the CHI-CDK 130 by dlopening the so library and obtaining the operation interface of the other party.

[0036] Below Figure 2 Taking the electronic device 200 in the example, the structure of the terminal device in the present disclosure is exemplarily described. Figure 2 The electronic device 200 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0037] like Figure 2 As shown, the electronic device 200 is in the form of a general computing device. The components of the electronic device 200 may include, but are not limited to: at least one processing unit 210, at least one storage unit 220, a bus 230 connecting different system components (including the storage unit 220 and the processing unit 210), and a display unit 240.

[0038] The storage unit 220 stores program codes, and the program codes can be executed by the processing unit 210, so that the processing unit 210 executes the motion posture determination method in this specification.

[0039] The storage unit 220 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 221 and / or a cache memory unit 222 , and may further include a read-only memory unit (ROM) 223 .

[0040] The storage unit 220 may also include a program / utility 224 having a set (at least one) of program modules 225, such program modules 225 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0041] Bus 230 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0042] The electronic device 200 may also communicate with one or more external devices 270 (e.g., sensor devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 200, and / or communicate with any device that enables the electronic device 200 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed through an input / output (I / O) interface 250. In addition, the electronic device 200 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) through a network adapter 260. As shown, the network adapter 260 communicates with other modules of the electronic device 200 through a bus 230. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 200, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, data backup storage systems, and sensor modules (e.g., gyroscope sensors, magnetic sensors, acceleration sensors, distance sensors, proximity light sensors, etc.).

[0043] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the embodiment of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiment of the present disclosure.

[0044] Figure 3 A schematic diagram of a camera platform architecture CamX-CHI to which the embodiments of the present disclosure can be applied is shown.

[0045] refer to Figure 3 As shown in the figure, the CamX part is implemented as the HAL3 interface at the top, and communicates with the kernel through the V4L2 framework at the bottom. In the middle, it maintains interaction with CHI by dlopening the so library and obtaining the other party's operation interface.

[0046] The CamX part can include the following main directory structures: Core: used to store the core implementation module of CamX, which also includes the Hal directory mainly used to implement the HAL3 interface, and the CHI directory responsible for interacting with CHI; Csl: used to store the communication module mainly responsible for CamX and Camera driver, providing a unified Camera driver control interface for CamX; Hwl: used to store hardware Nodes with independent computing capabilities, which are managed by Csl; Swl: used to store Nodes that do not have independent computing capabilities and must rely on the CPU to be implemented.

[0047] CHI-CDK can include the following main directory structures: Chioverride: used to store the core module of CHI implementation, responsible for interacting with CamX and realizing the overall framework of CHI and specific business processing; Bin: used to store configuration items related to the camera platform architecture; Topology: used to store user-defined Usecase XML configuration files; Node: used to store Nodes of user-defined functions; Module: used to store configuration files of different Sensors, which are needed when initializing the Sensor; Tuning: used to store configuration files of effect parameters under different scenes; Sensor: used to store private information and register configuration parameters of different Sensors; Actuator: used to store configuration information of different focus modules; Ois: used to store configuration information of the anti-shake module; Flash: stores configuration information of the flash module; Eeprom: stores configuration information of the Eeprom external storage module; Fd: stores configuration information of the face recognition module.

[0048] The camera module initialization method and the camera module initialization device according to the exemplary embodiments of the present disclosure are described in detail below.

[0049] Figure 4 The flowchart of a camera module initialization method in this exemplary embodiment is shown, which may include the following steps S410 to S430:

[0050] In step S410, in response to a start instruction of starting the camera module, identification information corresponding to the camera module is acquired, and a photographing mode corresponding to the camera module is determined according to the identification information.

[0051] In an exemplary embodiment, an on instruction refers to an instruction to call the camera module by launching a camera application when taking a photo. For example, the user clicks on an icon to trigger the camera application in the image user interface of the terminal device to generate an on instruction, and calls and turns on the camera module through the on instruction. Of course, the user can also trigger and generate an on instruction to start the camera module in the form of a shortcut key combination, and this exemplary embodiment does not specifically limit this.

[0052] The camera module refers to a software module in the terminal device that processes image information based on the image acquisition device. For example, the image acquisition device can be composed of an optical lens, an image sensor, a flash, and a control circuit. The camera module can process the sensor data collected and transmitted by the image acquisition device to generate a displayed image.

[0053] The identification information of the camera module refers to the data used to identify and distinguish different camera modules, namely, CameraID. Since the processing of the collected images is different when using a multi-camera camera module or a single-camera camera module, or a single-camera camera module using different camera models, or a multi-camera camera module using different camera models and a combination of different types of cameras, differentiating different camera modules by CameraID helps to ensure the accuracy of image processing.

[0054] The photo mode refers to the image processing method adopted by the camera module when it is in different application scenarios. For example, for ordinary scenes with sufficient lighting, the photo mode can adopt the front photo mode or the rear photo mode; for dark scenes with insufficient lighting, the photo mode can adopt the front night scene mode or the rear night scene mode; for scenes of taking pictures of people, the photo mode can adopt the front portrait mode. Of course, other types of photo modes are also possible, and this example embodiment does not specifically limit this.

[0055] In step S420, a function driver corresponding to the photographing mode is determined; the function driver includes a preview function driver and a photographing function driver.

[0056] In an exemplary embodiment, function driver refers to the basic component Feature in the camera platform architecture CamX-CHI. Function driver Feature represents a specific function. For example, for the artificial intelligence photo mode, it may include function driver FeatureASD. Function driver FeatureASD is used to implement AI (Artificial Intelligence) functions. During preview, each frame of data is received, and the AI ​​recognition output result of the current scene is analyzed, and it is given to the upper layer through methods such as metadata for subsequent processing. Generally speaking, one function driver Feature or multiple function driver Features can be set for a photo mode. Of course, the function driver Feature may not be set for the photo mode. Specifically, it can be customized according to user needs in the CHI-CDK corresponding to the camera platform architecture CamX-CHI. This exemplary embodiment does not specifically limit this.

[0057] Figure 5 A structural diagram schematically illustrates the relationship between basic components of a camera platform architecture CamX-CHI in an exemplary embodiment of the present disclosure.

[0058] refer to Figure 5 As shown, the camera platform architecture CamX-CHI may include at least an application instance Usecase component 510, a function driver Feature component 520, a camera pipeline control Session component 530, and a camera pipeline Pipeline component 540.

[0059] Among them, the application instance Usecase component 510 is the largest abstract concept in the camera platform architecture CamX-CHI, which contains multiple Pipelines for implementing specific functions. The specific implementation is completed in CHI through the Usecase class, which is mainly responsible for business processing and resource management.

[0060] The function-driven Feature component 520 represents a specific function, which requires multiple Pipelines to be combined to implement. It is uniformly managed by Usecase and implemented through the Feature class in CHI. There is no corresponding definition in XML. The specific Feature selection work is completed in Usecase. By passing in the instance of Usecase when creating Feature, the respective resources of Usecase can be accessed.

[0061] The camera pipeline control Session component 530 is used to manage the abstract control unit of the Pipeline. A Session has at least one Pipeline and controls all hardware resources, the flow of requests for each internal Pipeline, and the input and output of data. It has no customizable parts, so the Session is not defined as an independent unit in the XML file in CHI.

[0062] The camera pipeline Pipeline component 540 is a collection of all resources that provide a single specific function. It maintains the flow of all hardware resources and data. Each Pipeline can include Node nodes 550 and Link node connection relationships 560. It is implemented in CamX through the Pipeline class, which is responsible for the maintenance of the software and hardware resources of the entire Pipeline and the processing of business logic.

[0063] In simple terms, one photo-taking mode can correspond to an application instance Usecase. An application instance Usecase can manage and select multiple function-driven Features. A function-driven Feature can be composed of multiple Pipelines to implement a specific function. Pipelines can be managed by Session.

[0064] Of course, this is only an illustrative example and should not impose any special limitations on this exemplary embodiment.

[0065] The preview function driver refers to a function driver Feature that implements preview image display. For example, the preview function driver can be FeatureHDR (High-Dynamic Range, high dynamic range rendering), or it can be FormatConvertor (image processing class, used to convert digital images into matrix form. Stored in the form of one-dimensional and two-dimensional arrays). Of course, it can also be other function drivers that implement preview image display. This example embodiment does not make any special limitations on this.

[0066] The camera function driver refers to a function driver Feature that realizes the display of camera images. For example, the camera function driver can be FeatureMFSR (Multi-frame super resolution) or AnchorSync (a method for face detection). Of course, it can also be other function drivers that realize the display of camera images. This example embodiment does not make any special limitation on this.

[0067] Continue to refer Figure 4In step S430, before receiving a photo request, the camera module in the photo mode is initialized by the created preview function driver, and a preview image is displayed.

[0068] In an exemplary embodiment, a photo request refers to a request instruction to process and store image data collected by a camera module. The photo request may include an image capture request and a video recording request. For example, a trigger control for image capture or video recording may be provided by a camera application running in a terminal device, and an image capture request and a video recording request may be generated by the trigger control.

[0069] Before receiving a photo request, it can be considered that the camera module is always in the preview stage. It is only necessary to create a preview function driver that supports displaying preview images, without the need to additionally create a photo function driver that can only be used when taking photos. This effectively reduces the number of function drivers that need to be created when the camera module is started, reduces the consumption of system resources, improves system performance, and increases the startup response speed of the camera module.

[0070] Steps S410 to S430 are described in detail below.

[0071] In an exemplary embodiment, the camera platform architecture CamX-CHI may include a camera main process, which may include a camera device startup phase, a camera data stream configuration phase, and a photo request response phase.

[0072] Figure 6 A schematic diagram schematically illustrates a time-consuming process of starting a camera in an exemplary embodiment of the present disclosure.

[0073] refer to Figure 6 As shown, the underlying time-consuming part of the camera main process Camera Provider 610 in the entire camera opening process is composed of the camera device opening Open stage 620, the camera data stream configuration Configure_stream stage 630 and the photo request response Request->Result stage 640. Among them:

[0074] When the system starts, the camera main process Camera Provider 610 will be run. During the entire program initialization process, the get_number_of_camera interface of the acquired camera_module_t will be called to obtain the number of cameras supported by the underlying layer. Since it is the first time to obtain, there will be many initialization actions in the camera platform architecture CamX-CHI.

[0075] In the camera device opening Open phase 620, once the user opens the camera application, the App will call the openCamera method of CameraManager, which will eventually call the CameraService::connectDevice method in the Camera Service, and then notify the Provider through the ICameraDevice::open() HIDL interface, and then inside the Provider, a Camera device is obtained by calling the open method of methods in the previously obtained camera_module_t, corresponding to the camera3_device_t structure in the HAL, and then the initialize method of the obtained camera3_device_t will continue to be called in the Provider for initialization.

[0076] In the camera data stream configuration Configure_stream stage 630, when opening the camera application, after obtaining and opening the camera device, the camera application will call CameraDevice.createCaptureSession to obtain CameraDeviceSession, and notify the Camera Service through the Camera api v2 standard interface, calling its CameraDeviceClient.endConfigure method. In this method, the Provider will be notified through the HIDL interface ICameraDeviceSession::configureStreams_3_4 to start processing this configuration requirement. Inside the Provider, the data stream configuration will be passed to CamX-CHI through the configure_streams method of the camera3_device_t structure obtained in the call open process, and then CamX-CHI will complete the configuration of the data stream. The camera data stream configuration Configure_stream stage 630 mainly includes two stages: selecting UsecaseID and creating Usecase according to the selected UsecaseId. Different UsecaseIds correspond to different application scenarios.

[0077] The main process in the camera data stream configuration Configure_stream stage 630 is: 1. Get the Usecase configuration information in the XML file; 2. Create a Feature; 3. Save the data stream and rebuild the Usecase configuration information; 4. Call the initialize method of the parent class CameraUsecaseBase to perform some routine initialization work.

[0078] In the photo request response Request->Result stage 640, when the user opens the camera application for preview or clicks a photo operation, a photo request is triggered, that is, Request. This action first sends the request to the Camera Service through the Camera apiv2 interface through the capture or setRepeatingRequest method of CameraDeviceSession, and then sends the request to the CameraDevice::RequestThread thread inside the Camera Service for processing. Once entering the thread, it will eventually send the request to the Provider through the HIDL interface ICameraCaptureSession:processCaptureRequest_3_4. After that, when the Provider receives the request, it will call the process_capture_request of the camera3_device_t structure to start the HAL processing of this Request, and this processing is implemented by CamX-CHI.

[0079] After the user opens the camera application, the camera platform architecture CamX-CHI will start processing a request after receiving it. Once image data is generated, it will be returned to the application layer for display through layers of callbacks. Each Request corresponds to three Results, namely partial metadata, metadata and imagedata. For each Result, the upload process can be roughly divided into the following two stages: 1. The image data is processed within the Session and the result is sent to the Usecase; 2. The Usecase receives the data from the Session, uploads it to the camera main process Camera Provider 610, and displays the image.

[0080] In an exemplary embodiment, a preview function driver corresponding to the photographing mode may be created in the camera data stream configuration phase. The camera data stream configuration phase is before the photographing request response phase on the timeline, that is, before receiving the photographing request, the camera module is initialized, and only the preview function driver needs to be created to realize the display of the preview image.

[0081] Specifically, a camera function driver corresponding to the camera mode can be created in the camera request response phase, and the sensor data cached by the camera module can be processed by the camera function driver to generate a camera image. When the camera main process CameraProvider is in the camera request response phase, it can be considered that a camera request is currently received, that is, after receiving the camera request, since a camera image corresponding to the camera mode needs to be generated, a camera function driver corresponding to the camera mode needs to be created at this time, so that the sensor data collected by the camera module can be processed by the created camera function driver to generate a camera image.

[0082] Before the camera module starts the initialization process and receives a photo request, the camera application only needs to display a preview image. At this time, only a preview function driver needs to be created to process the preview image, and no photo function driver that is not needed at this time needs to be created. This effectively reduces the number of function drivers that need to be created during the camera module initialization process, saves system resources, increases the initialization speed of the camera module, increases the efficiency of generating preview images, and improves the response speed of the camera module.

[0083] In an exemplary embodiment, the photographing request response phase may include a photographing request receiving step and a photographing result feedback step, wherein the photographing request receiving step is the Request phase, and the photographing result feedback step is the Result phase. The delayed creation driven by the photographing function may be realized through the steps in 7, as shown in FIG. Figure 7 As shown, it may specifically include:

[0084] Step S710, creating a time-delay photography trigger thread;

[0085] Step S720, triggering and creating a photographing function driver corresponding to the photographing mode based on the delayed photographing triggering thread; wherein the delayed photographing triggering thread is triggered after receiving the photographing request and before feeding back the photographing result.

[0086] Among them, the delayed photography trigger thread refers to the trigger thread that implements the creation of the photography function driver in the photography request response phase. After the preview function driver is created in the camera data stream configuration phase, the delayed photography trigger thread is created. The delayed photography trigger thread can trigger the creation of the photography function driver after receiving the photography request and before the photography result is fed back.

[0087] By creating a delayed photo trigger thread, it is possible to ensure that after receiving a photo request, the photo function driver corresponding to the photo mode is created in time, ensuring timely processing of the photo image and improving the processing efficiency of the photo image.

[0088] In an exemplary embodiment, the camera module may include a multi-camera camera module and a single-camera camera module, wherein a single-camera camera module refers to a camera module including one camera, and the image captured by the single-camera camera module is only the recorded image of the camera; a multi-camera camera module refers to a camera module including two or more cameras with different functions, for example, a multi-camera camera module may be a camera module composed of a wide-angle camera combined with a telephoto camera, or a camera module composed of a wide-angle camera combined with a depth of field camera, and this exemplary embodiment does not specifically limit this. The multi-camera camera module can synthesize images captured by different cameras by superposition, extraction, etc., so as to obtain clearer images with higher resolution and better effects. Since the multi-camera camera module and the single-camera camera module have great differences in composition, the processing flow of the captured images is also different.

[0089] It is easy to understand that due to the structural limitations of terminal devices such as smart phones, the image acquisition device associated with the multi-camera module is generally set on the back of the terminal device as a rear-mounted shooting module for image acquisition, and the image acquisition device associated with the single-camera module is generally set in front of the terminal device as a front-mounted shooting module for image acquisition. Of course, in special scenarios, the image acquisition device associated with the multi-camera module can also be set in the front of the terminal device, and the image acquisition device associated with the single-camera module can also be set on the back of the terminal device, and this example embodiment is not limited to this.

[0090] It is understandable that different shooting modes may correspond to different function drive sets. Specifically, different shooting modes may correspond to different function drive sets, including: different shooting modes in a camera module may correspond to different function drive sets. For example, in a single-camera camera module, shooting mode 1, shooting mode 2, and shooting mode 3 may be included, wherein shooting mode 1 may include function drive 2, function drive 3, and function drive 5, shooting mode 2 may include function drive 6, function drive 7, and function drive 8, and shooting mode 3 may include function drive 1, function drive 3, and function drive 4. Of course, this is only an illustrative example and should not impose any special limitations on this exemplary embodiment.

[0091] In this example embodiment, the multi-camera camera module may include at least a rear photo mode and a rear night scene mode, wherein the rear photo mode may include at least an MFNR function driver and a RawHDR function driver, the MFNR (Multi Frame Noise Reduction) function driver is mainly used to perform multi-frame noise reduction processing on image data, and the RawHDR function driver is mainly used to perform HDR (High-Dynamic Range) synthesis on image data based on the original data Raw format to respectively display the details of highlights, midtones and shadows in the image data; the rear night scene mode may include at least an HDR function driver and a Demux Serializer function driver, the HDR function driver is mainly used to synthesize a final HDR image according to LDR (Low-Dynamic Range) with different exposure times and using the LDR image with the best details corresponding to each exposure time. Compared with ordinary images, HDR images can provide more dynamic range and image details, and the Demux Serializer function driver is mainly used to perform demultiplexing and serialization processing on image data to improve the transmission and storage efficiency of image data.

[0092] The single-camera camera module may include at least a front photo mode and a front night scene mode, wherein the front photo mode may include at least a Y2YPostMFNR function driver and an Anchor function driver, the Y2YPostMFNR function driver is mainly used to convert the color space of the image data into the YUV color space and perform multi-frame noise reduction, effectively reducing the noise data in the image data, and the Anchor function driver is mainly used to perform target detection on the image data; the front night scene mode may include at least a FormatConvertor function driver and a RealTimeSerializer function driver, the FormatConvertor function driver is mainly used to convert the digital image into a matrix form and store it in the form of one-dimensional and two-dimensional arrays, and the RealTimeSerializer function driver is mainly used to perform real-time serialization processing on the image data to improve transmission efficiency.

[0093] It is easy for those skilled in the art to understand that the above-mentioned photographing modes and the functional drivers corresponding to the photographing modes are only illustrative, so as to facilitate those skilled in the art to understand the relationship between the camera module and the photographing modes and functional drivers in the present disclosure. In actual application, the photographing modes corresponding to the camera module are not limited to the above-mentioned illustrative examples, and the functional drivers corresponding to the various photographing modes are not limited to the above-mentioned illustrative examples, and should not impose any special limitations on this exemplary embodiment.

[0094] Figure 8A schematic diagram schematically illustrates a startup process before optimization in an exemplary embodiment of the present disclosure.

[0095] refer to Figure 8 As shown, the startup process 810 before optimization can be divided into a process 820 of starting a multi-camera module and a process 830 of starting a single-camera module.

[0096] Among them, in the process 820 of starting the multi-camera camera module, the shooting mode 840 corresponding to the multi-camera camera module may include multiple shooting modes such as rear photo mode and rear night scene mode, and the function driver corresponding to the shooting mode of the multi-camera camera module can be divided into a preview function driver and a shooting function driver; when the multi-camera camera module is started and initialized, all function drivers 860 corresponding to the multi-camera camera module are created in the Configure_stream stage 850 in the startup process 810 before optimization.

[0097] In the process 830 of starting the single-camera module, the photo mode 870 corresponding to the single-camera module may include multiple photo modes such as rear photo mode and rear night scene mode, and the function driver corresponding to the photo mode of the single-camera module may be divided into a preview function driver and a photo function driver; when the single-camera module is started and initialized, all function drivers 890 corresponding to the single-camera module are created in the Configure_stream stage 880 in the startup process 810 before optimization.

[0098] Assume that a multi-camera module (or a single-camera module) only sets the rear-camera mode (or the front-camera mode) at startup, the number of preview function driver Features corresponding to the rear-camera mode (or the front-camera mode) is 5, and the number of camera function driver Features is 10, that is, only these 5 preview function driver Features are created in the Configure_stream stage to enable the multi-camera module (or the single-camera module) to display the preview image in the rear-camera mode (or the front-camera mode); while the startup process 810 before optimization will create all 15 function driver Features corresponding to the multi-camera camera module (or the single-camera camera module) in the Configure_stream stage, however, the 10 camera function drivers are not used in the Configure_stream stage, so creating the camera function driver in the Configure_stream stage will cause a waste of system resources, reduce system performance, and reduce the startup speed of the camera module.

[0099] Fig. 9 A schematic diagram schematically illustrates an optimized startup process in an exemplary embodiment of the present disclosure.

[0100] refer to Fig. 9 As shown, the optimized startup process 901 can be divided into a process 902 for starting a multi-camera module and a process 903 for starting a single-camera module.

[0101] Among them, in the process 902 of starting the multi-camera module, the shooting mode 904 corresponding to the multi-camera module can include multiple shooting modes such as rear photo mode and rear night scene mode, and the function driver corresponding to the shooting mode of the multi-camera module can be divided into a preview function driver and a shooting function driver; when the multi-camera module is started and initialized, in the optimized startup process 910, only the preview function driver 906 corresponding to the multi-camera module will be created in the Configure_stream stage 905 to realize the display of the preview image; and after receiving the shooting request and before the shooting result is fed back, the shooting function driver 908 is created in the Request stage 907 by delaying the shooting to trigger the thread, so as to realize the generation and display of the shooting image.

[0102] In the process 903 of starting the single-camera module, the shooting mode 909 corresponding to the single-camera module may include multiple shooting modes such as front photo mode and front night scene mode, and the function driver corresponding to the shooting mode of the single-camera module may be divided into a preview function driver and a shooting function driver; when the single-camera module is started and initialized, in the optimized startup process 901, only the preview function driver 911 corresponding to the single-camera module will be created in the Configure_stream stage 910 to realize the display of the preview image; and after receiving the shooting request and before the shooting result is fed back, the shooting function driver 913 is created in the Request stage 912 by delaying the shooting to trigger the thread, so as to realize the generation and display of the shooting image.

[0103] Assume that the multi-camera camera module (or single-camera camera module) only sets the rear photo mode (or front photo mode) at startup, the number of preview function driver Features corresponding to the rear photo mode (or front photo mode) is 5, and the number of photo function driver Features is 10, that is, only these 5 preview function driver Features are created in the Configure_stream stage to realize the multi-camera camera module (or single-camera camera module) to display the preview image in the rear photo mode (or front photo mode). Compared with creating 15 function drivers, the number of function drivers that need to be created is greatly reduced, saving system resources and improving the startup speed of the multi-camera camera module (or single-camera camera module); and after receiving the photo request and before feeding back the photo result, the creation of the photo function driver is triggered in time in the Request stage by delaying the photo trigger thread, so as to ensure the generation efficiency and response speed of the photo image, and realize the generation and display of the photo image.

[0104] In summary, in this exemplary embodiment, the identification information of the camera module can be obtained, and the photographing mode corresponding to the camera module can be determined according to the identification information, and then the preview function driver and the photographing function driver corresponding to the photographing mode can be determined, and before receiving the photographing request, the camera module in the photographing mode is initialized by the created preview function driver, so as to quickly display the preview image. On the one hand, the function driver corresponding to the photographing mode of the camera module is divided, and before receiving the photographing request, since the camera module only needs to implement the preview function, only the preview function driver is created to implement the initialization of the camera module, avoiding the creation of unnecessary photographing function drivers, effectively reducing the number of function drivers that need to be created when the camera module is initialized, improving the initialization speed of the camera module, improving the efficiency of generating preview images, and improving the response speed of the camera module; on the other hand, since the function drivers corresponding to the same photographing mode of different types of camera modules are not the same, the corresponding photographing mode is determined by the identification information of the camera module, which can not only ensure the accuracy of the determined function driver, but also improve the efficiency of determining the function driver, and further ensure the response speed of the camera module.

[0105] It should be noted that the above figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0106] For further reference, Fig.10As shown, in the implementation of this example, a camera module initialization device 1000 is also provided, which may include a photographing mode determination module 1010, a function drive determination module 1020, and a camera module initialization module 1030. Among them:

[0107] A photographing mode determining module 1010, configured to respond to a start instruction of the camera module, obtain identification information corresponding to the camera module, and determine a photographing mode corresponding to the camera module according to the identification information;

[0108] A function driver determination module 1020 is used to determine a function driver corresponding to the photographing mode; the function driver includes a preview function driver and a photographing function driver;

[0109] The camera module initialization module 1030 is used to initialize the camera module in the photographing mode by using the created preview function driver before receiving a photographing request, and to display a preview image.

[0110] In an exemplary embodiment, the camera platform architecture CamX-CHI may include a camera main process, which may include a camera device startup phase, a camera data stream configuration phase, and a photo request response phase.

[0111] In an exemplary embodiment, the camera module initialization module 1030 may be used to:

[0112] A preview function driver corresponding to the photographing mode is created during the camera data stream configuration phase.

[0113] In an exemplary embodiment, the camera module initialization device 1000 may include a camera module photographing module, and the camera module photographing unit may include:

[0114] A photographing function driver creating unit, used for creating a photographing function driver corresponding to the photographing mode in the photographing request response phase;

[0115] A photographic image generating unit is used to drive the processing of the sensor data collected by the camera module through the photographic function and generate a photographic image.

[0116] In an exemplary embodiment, the photographing request response phase may include a photographing request receiving step and a photographing result feedback step;

[0117] The camera driver creation unit can be used for:

[0118] Create a time-lapse photography trigger thread;

[0119] Triggering and creating a photo function driver corresponding to the photo mode based on the delayed photo triggering thread;

[0120] The delayed photography triggering thread is triggered after receiving the photography request and before feeding back the photography result.

[0121] In an exemplary embodiment, the camera module may include at least one of a multi-camera camera module and a single-camera camera module.

[0122] In an exemplary embodiment, the multi-camera camera module may include a rear-facing photo mode and a rear-facing night view mode, and the single-camera camera module may include a front-facing photo mode and a front-facing night view mode.

[0123] In an exemplary embodiment, the rear photo mode includes MFNR function driving and RawHDR function driving;

[0124] The rear night scene mode includes an HDR function driver and a Demux Serializer function driver;

[0125] The front photo mode includes a Y2YPostMFNR function driver and an Anchor function driver;

[0126] The front night scene mode includes a FormatConvertor function driver and a RealTimeSerializer function driver.

[0127] The specific details of each module in the above device have been described in detail in the implementation method of the method part. The undisclosed details can be found in the implementation method of the method part, so they will not be repeated here.

[0128] Those skilled in the art will appreciate that various aspects of the present disclosure may be implemented as systems, methods or program products. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as "circuits", "modules" or "systems".

[0129] The exemplary embodiments of the present disclosure also provide a computer-readable storage medium on which a program product capable of implementing the above-mentioned method of the present specification is stored. In some possible implementations, various aspects of the present disclosure may also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary embodiments of the present disclosure described in the above-mentioned "Exemplary Method" section of the present specification.

[0130] It should be noted that the computer-readable medium shown in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0131] In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, apparatus, or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0132] In addition, program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0133] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and examples are to be considered as exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0134] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A camera module initialization method, characterized in that: The method is performed by a terminal device integrated with a camera module and a camera platform architecture CamX-CHI, and includes: In response to a start-up instruction for starting the camera module, obtaining identification information corresponding to the camera module, and determining a photographing mode corresponding to the camera module according to the identification information; Determine a function driver corresponding to the photographing mode; the function driver includes a preview function driver and a photographing function driver; Before receiving a photo request, initializing the camera module in the photo mode through the created preview function driver and displaying a preview image; The camera platform architecture CamX-CHI includes a camera main process, which includes a camera device startup phase, a camera data stream configuration phase, and a photo request response phase; Before receiving the photographing instruction, initializing the camera module in the photographing mode by using the created preview function driver includes: Creating a preview function driver corresponding to the photographing mode in the camera data stream configuration phase; The method further comprises: Creating a camera function driver corresponding to the camera mode in the camera request response phase; The sensor data collected by the camera module is processed by the camera function driver to generate a photographic image.

2. The method according to claim 1, characterized in that The photographing request response phase includes a photographing request receiving step and a photographing result feedback step; The step of creating a photographing function driver corresponding to the photographing mode in the photographing request response phase includes: Create a time-lapse photography trigger thread; Triggering and creating a photo function driver corresponding to the photo mode based on the delayed photo triggering thread; The delayed photography triggering thread is triggered after receiving the photography request and before feeding back the photography result.

3. The method according to claim 1, characterized in that The camera module includes at least one of a multi-camera camera module and a single-camera camera module.

4. The method according to claim 3, characterized in that The multi-camera module includes a rear photo mode and a rear night scene mode, and the single-camera module includes a front photo mode and a front night scene mode.

5. The method according to claim 4, characterized in that: The rear photo mode includes MFNR function driving and RawHDR function driving; The rear night scene mode includes an HDR function driver and a Demux Serializer function driver; The front photo mode includes a Y2YPostMFNR function driver and an Anchor function driver; The front night scene mode includes a FormatConvertor function driver and a RealTimeSerializer function driver.

6. A camera module initialization device, characterized in that: The device is provided in a terminal device integrated with a camera module and a camera platform architecture CamX-CHI, and the device includes: a photographing mode determining module, configured to respond to a start-up instruction for starting the camera module, obtain identification information corresponding to the camera module, and determine a photographing mode corresponding to the camera module according to the identification information; A function driver determination module, used to determine the function driver corresponding to the photographing mode; the function driver includes a preview function driver and a photographing function driver; A camera module initialization module, used for initializing the camera module in the photographing mode by using the created preview function driver before receiving a photographing request, and displaying a preview image; The camera platform architecture CamX-CHI includes a camera main process, which includes a camera device startup phase, a camera data stream configuration phase, and a photo request response phase; Before receiving the photographing instruction, initializing the camera module in the photographing mode by using the created preview function driver includes: Creating a preview function driver corresponding to the photographing mode in the camera data stream configuration phase; A photographing function driver creating unit, used for creating a photographing function driver corresponding to the photographing mode in the photographing request response phase; A photographic image generating unit is used to drive the processing of the sensor data collected by the camera module through the photographic function and generate a photographic image.

7. A computer readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

8. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to perform the method of any one of claims 1 to 5 by executing the executable instructions.

Images