Method for generating image simulation data, electronic device and storage medium

Abstract

The present application discloses a method for generating image simulation data, an electronic device, and a storage medium, which can conveniently and accurately obtain image simulation data corresponding to target exposure parameters without relying on an algorithm. The method for generating image simulation data includes: an application program in an application layer sends a request for generating image simulation data to a hardware abstraction layer; the hardware abstraction layer responds to the generation request, generates a request for obtaining image simulation data, and sends the obtainment request to a control layer; the control layer responds to the obtainment request, reads and writes at least one exposure parameter into an image sensor; the control layer sends a notification message to the hardware abstraction layer before the exposure of the image sensor takes effect, the notification message is used to trigger the hardware abstraction layer to obtain an image frame at the exposure taking effect; the hardware abstraction layer obtains the target image frame output by the image sensor at the exposure taking effect according to the notification message, and uses the target image frame as the image simulation data.

Description

Technical field

[0001] The embodiments of the present application relate to the field of algorithm simulation technology, and in particular to a method for generating image simulation data, an electronic device, and a storage medium. [Background Technology]

[0002] In some scenarios, it is necessary to obtain images with different exposure parameters. In the existing technology, the exposure parameters are written into the algorithm, and then the algorithm is used to obtain the required image simulation data. This method is highly dependent on the algorithm and has poor versatility in different application scenarios. [Summary of the invention]

[0003] The embodiments of the present application provide a method for generating image simulation data, an electronic device, and a storage medium, which can conveniently and accurately obtain image simulation data corresponding to target exposure parameters without relying on algorithms, thereby improving the versatility of obtaining image simulation data.

[0004] In a first aspect, an embodiment of the present application provides a method for generating image simulation data, the method being applied to an electronic device, wherein the operating system of the electronic device includes a hardware abstraction layer, an application layer, and a control layer, the method comprising:

[0005] The application program of the application layer sends a generation request for image simulation data to the hardware abstraction layer;

[0006] The hardware abstraction layer generates an acquisition request for the image simulation data in response to the generation request, and sends the acquisition request to the control layer;

[0007] The control layer reads at least one exposure parameter in a pre-stored configuration file in response to the acquisition request, and writes the at least one exposure parameter into the image sensor;

[0008] The control layer sends a notification message to the hardware abstraction layer before the exposure effective timing of the image sensor, wherein the notification message is used to trigger the hardware abstraction layer to obtain an image frame at the exposure effective timing, and the exposure effective timing is used to indicate the timing when the image sensor generates a corresponding image frame based on the written target exposure parameters;

[0009] The hardware abstraction layer obtains the target image frame output by the image sensor at the exposure effective time according to the notification message, and uses the target image frame as the image simulation data.

[0010] In an embodiment of the present application, when image simulation data needs to be generated, the application program of the application layer can send a generation request for the image simulation data to the hardware abstraction layer, the hardware abstraction layer can respond to the generation request, generate and send an acquisition request for the image simulation data to the control layer, the control layer can respond to the acquisition request, thereby writing at least one exposure parameter into the image sensor in sequence, and the control layer can send a notification message to the hardware abstraction layer before the target exposure parameter is written into the image sensor and the image sensor generates a corresponding target image frame based on the target exposure parameter, so that the hardware abstraction layer can accurately obtain the target image frame when the target image frame is generated, and use the target image frame as image simulation data. In this method, the target image frame corresponding to the target exposure parameter can still be accurately obtained without relying on an algorithm, that is, it is more versatile when obtaining the target image frame corresponding to the target exposure parameter.

[0011] Optionally, the acquisition request carries a notification timing parameter, where the notification timing parameter is determined by the hardware abstraction layer based on the pre-stored exposure effective timing and is used to indicate a timing at which the control layer notifies the hardware abstraction layer to acquire an image frame after the control layer writes the exposure parameter to the image sensor. The control layer sending the notification message to the hardware abstraction layer before the exposure effective timing of the image sensor includes:

[0012] After the control layer writes the target exposure parameter into the image sensor, the control layer sends a notification message to the hardware abstraction layer based on the notification timing parameter, which is used to notify the hardware abstraction layer to obtain a target image frame corresponding to the target exposure parameter.

[0013] In an embodiment of the present application, the acquisition request sent by the hardware abstraction layer to the control layer may carry a notification timing parameter indicating that after the control layer writes the exposure parameters to the image sensor, the control layer notifies the hardware abstraction layer to obtain the image frame. Then, after the control layer writes the target exposure parameters to the image sensor, the control layer may send a notification message to the hardware abstraction layer based on the above notification timing parameter to notify the hardware abstraction layer to obtain the target image frame corresponding to the target exposure parameters, so that the hardware abstraction layer can accurately obtain the above target image frame.

[0014] Optionally, the acquisition request carries the exposure effective timing, and the control layer sends a notification message to the hardware abstraction layer before the exposure effective timing of the image sensor, including:

[0015] The control layer determines a notification timing parameter based on the exposure validation timing, where the notification timing parameter is used to represent a timing at which the control layer notifies the hardware abstraction layer to acquire an image frame after the control layer writes the exposure parameter into the image sensor;

[0016] After the control layer writes the target exposure parameter into the image sensor, the control layer sends a notification message to the hardware abstraction layer based on the notification timing parameter, which is used to notify the hardware abstraction layer to obtain a target image frame corresponding to the target exposure parameter.

[0017] In an embodiment of the present application, the acquisition request sent by the hardware abstraction layer to the control layer may carry the exposure effective timing, and the control layer may determine the notification timing parameter based on the exposure effective timing, that is, after the control layer writes the exposure parameter into the image sensor, the control layer notifies the hardware abstraction layer of the timing to obtain the image frame. Then, after the control layer writes the target exposure parameter into the image sensor, the control layer may send a notification message to the hardware abstraction layer based on the above notification timing parameter to notify the hardware abstraction layer to obtain the target image frame corresponding to the target exposure parameter, so that the hardware abstraction layer can accurately obtain the above target image frame.

[0018] Optionally, before the control layer sends a notification message to the hardware abstraction layer before the exposure of the image sensor takes effect, the method further includes:

[0019] The control layer responds to the acquisition request and reads the exposure validity timing in the configuration file;

[0020] The control layer sending a notification message to the hardware abstraction layer before the exposure of the image sensor takes effect includes:

[0021] The control layer determines a notification timing parameter based on the exposure validation timing, where the notification timing parameter is used to represent a timing at which the control layer notifies the hardware abstraction layer to acquire an image frame after the control layer writes the exposure parameter into the image sensor;

[0022] After the control layer writes the target exposure parameter into the image sensor, the control layer sends a notification message to the hardware abstraction layer based on the notification timing parameter, which is used to notify the hardware abstraction layer to obtain a target image frame corresponding to the target exposure parameter.

[0023] In an embodiment of the present application, the exposure effective timing can be stored in a configuration file. The control layer reads the exposure effective timing in the configuration file by responding to the acquisition request of the hardware abstraction layer, and the control layer can determine the notification timing parameter based on the exposure effective timing. That is, after the control layer writes the exposure parameter into the image sensor, the control layer notifies the hardware abstraction layer of the timing to obtain the image frame. Then, after the control layer writes the target exposure parameter into the image sensor, the control layer can send a notification message to the hardware abstraction layer based on the above-mentioned notification timing parameter to notify the hardware abstraction layer to obtain the target image frame corresponding to the target exposure parameter, so that the hardware abstraction layer can accurately obtain the above-mentioned target image frame.

[0024] Optionally, the control layer sending a notification message to the hardware abstraction layer before the exposure of the image sensor takes effect includes:

[0025] The control layer sends a notification message to the hardware abstraction layer indicating that the target exposure parameter has been written into the image sensor;

[0026] The hardware abstraction layer acquiring, according to the notification message, a target image frame output by the image sensor at the exposure effective time, includes:

[0027] The hardware abstraction layer obtains the target image frame output by the image sensor at the exposure validation timing according to the pre-stored exposure validation timing and the notification message.

[0028] In an embodiment of the present application, the control layer can send a notification message indicating that the target exposure parameters have been written into the image sensor to the hardware abstraction layer, so that the hardware abstraction layer can accurately obtain the target image frame based on the pre-stored exposure effectiveness timing and the above notification message.

[0029] Optionally, the operating system of the electronic device further includes a driver layer, the configuration file further includes a target quantity of image simulation data to be generated, and before the control layer sends a notification message to the hardware abstraction layer before the exposure of the image sensor takes effect, the method further includes:

[0030] The control layer responds to the acquisition request and reads the target quantity in the configuration file;

[0031] The control layer sends the target quantity to the hardware abstraction layer;

[0032] The hardware abstraction layer applies for an equal number of buffer areas based on the target number, and sends the address of each buffer area to the driver layer.

[0033] In an embodiment of the present application, a target number of image simulation data to be acquired can be stored in a configuration file. The control layer reads the target number from the configuration file by responding to an acquisition request from the hardware abstraction layer and sends the target number to the hardware abstraction layer. The hardware abstraction layer can separately apply for an equal number of buffer areas based on the target number and send the addresses of the buffer areas to the driver layer, so that image frames generated by the image sensor can be separately stored in the buffer areas via the driver layer.

[0034] Optionally, the target image frame is stored in a target buffer area via the driver layer, and the hardware abstraction layer obtains the target image frame corresponding to the exposure effective timing of the image sensor according to the notification message, including:

[0035] The hardware abstraction layer sends an acquisition instruction to the driver layer, where the acquisition instruction includes the address of the target buffer area;

[0036] The hardware abstraction layer receives the target image frame fed back by the driver layer based on the address of the target buffer area.

[0037] In the embodiment of the present application, the hardware abstraction layer can accurately obtain the target image frame stored in the target buffer area from the driver layer according to the address of the target buffer area.

[0038] In a second aspect, an embodiment of the present application provides an electronic device, the electronic device including a hardware abstraction layer, an application layer, and a control layer, and the electronic device also including an image sensor:

[0039] The application program of the application layer sends a generation request for image simulation data to the hardware abstraction layer;

[0040] The hardware abstraction layer generates an acquisition request for the image simulation data in response to the generation request, and sends the acquisition request to the control layer;

[0041] The control layer reads at least one exposure parameter in a pre-stored configuration file in response to the acquisition request, and writes the at least one exposure parameter into the image sensor;

[0042] The control layer sends a notification message to the hardware abstraction layer before the exposure effective timing of the image sensor, wherein the notification message is used to trigger the hardware abstraction layer to obtain an image frame at the exposure effective timing, and the exposure effective timing is used to indicate the timing when the image sensor generates a corresponding image frame based on the written target exposure parameters;

[0043] The hardware abstraction layer obtains the target image frame output by the image sensor at the exposure effective time according to the notification message, and uses the target image frame as the image simulation data.

[0044] Optionally, the acquisition request carries a notification timing parameter, which is determined by the hardware abstraction layer based on the pre-stored exposure validity timing and is used to indicate a timing at which the control layer notifies the hardware abstraction layer to acquire an image frame after the control layer writes the exposure parameter to the image sensor. The control layer is specifically configured to:

[0045] After the target exposure parameter is written into the image sensor, a notification message for notifying the hardware abstraction layer to obtain a target image frame corresponding to the target exposure parameter is sent to the hardware abstraction layer based on the notification timing parameter.

[0046] Optionally, the acquisition request carries the exposure validity timing, and the control layer is specifically configured to:

[0047] determining a notification timing parameter based on the exposure validation timing, wherein the notification timing parameter is used to represent a timing at which the control layer notifies the hardware abstraction layer to acquire an image frame after the control layer writes the exposure parameter into the image sensor;

[0048] After the target exposure parameter is written into the image sensor, a notification message for notifying the hardware abstraction layer to obtain a target image frame corresponding to the target exposure parameter is sent to the hardware abstraction layer based on the notification timing parameter.

[0049] Optionally, the control layer is further configured to:

[0050] In response to the acquisition request, reading the exposure effective timing in the configuration file;

[0051] The control layer is specifically used for:

[0052] determining a notification timing parameter based on the exposure validation timing, wherein the notification timing parameter is used to represent a timing at which the control layer notifies the hardware abstraction layer to acquire an image frame after the control layer writes the exposure parameter into the image sensor;

[0053] After the target exposure parameter is written into the image sensor, a notification message for notifying the hardware abstraction layer to obtain a target image frame corresponding to the target exposure parameter is sent to the hardware abstraction layer based on the notification timing parameter.

[0054] Optionally, the control layer is specifically used to:

[0055] Sending a notification message to the hardware abstraction layer indicating that the target exposure parameter has been written into the image sensor;

[0056] The hardware abstraction layer is specifically used for:

[0057] The target image frame output by the image sensor at the exposure effective timing is acquired according to the pre-stored exposure effective timing and the notification message.

[0058] Optionally, the electronic device further includes a driver layer, the configuration file further includes a target quantity of image simulation data to be generated, and the control layer is further configured to:

[0059] In response to the acquisition request, reading the target quantity in the configuration file;

[0060] The control layer is further configured to: send the target quantity to the hardware abstraction layer;

[0061] The hardware abstraction layer is further configured to apply for an equal number of buffer areas based on the target number, and send the address of each buffer area to the driver layer.

[0062] Optionally, the target image frame is stored in a target buffer area via the driver layer, and the hardware abstraction layer is specifically used to:

[0063] Sending an acquisition instruction to the driver layer, wherein the acquisition instruction includes the address of the target buffer area;

[0064] The target image frame fed back by the driving layer based on the address of the target buffer area is received.

[0065] In a third aspect, an embodiment of the present invention provides an electronic device, comprising a processor and a memory, wherein the processor is configured to implement the steps of the method described in any embodiment of the first aspect when executing a computer program stored in the memory.

[0066] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any embodiment of the first aspect.

[0067] It should be understood that the second to fourth aspects of the embodiments of the present invention are consistent with the technical solutions of the first aspect of the embodiments of the present invention, and the beneficial effects achieved by each aspect and the corresponding feasible implementation methods are similar and will not be repeated here.

Brief Description of the Drawings

[0068] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0069] Figure 1 A schematic diagram of a flow chart of a method for generating image simulation data provided in an embodiment of the present application;

[0070] Figure 2 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0071] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. [Specific implementation method]

[0072] In order to better understand the technical solutions of this specification, the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0073] It should be clear that the embodiments described are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this specification.

[0074] The terms used in the examples of this application are for the purpose of describing specific embodiments only and are not intended to limit this specification. The singular forms "a," "an," "the," and "the" used in the examples of this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0075] The inventors of this application have discovered through research that, in the process of image algorithm simulation, images with different exposure parameters are often required. However, in the prior art, different exposure parameters are usually written into a specific algorithm, and then images with different exposure parameters are obtained based on the specific algorithm. This method is highly dependent on the algorithm and has poor versatility in different application scenarios.

[0076] In view of this, an embodiment of the present application provides a method for generating image simulation data. In this method, when image simulation data needs to be generated, the application program of the application layer can send a generation request for the image simulation data to the abstract layer, the hardware abstraction layer can respond to the generation request, generate and send an acquisition request for the image simulation data to the control layer, the control layer can respond to the acquisition request, thereby writing at least one exposure parameter into the image sensor in sequence, and the control layer can send a notification message to the hardware abstraction layer before the target exposure parameter is written into the image sensor and the image sensor generates a corresponding target image frame based on the target exposure parameter, so that the hardware abstraction layer can accurately obtain the target image frame when the target image frame is generated, and use the target image frame as image simulation data. That is, the target image frame corresponding to the target exposure parameter can still be accurately obtained without relying on an algorithm, that is, it is more versatile when obtaining the target image frame corresponding to the target exposure parameter.

[0077] The following is an introduction to the technical solutions provided by the embodiments of this application with reference to the accompanying drawings. Figure 1 The present invention provides a method for generating image simulation data in an embodiment. The method is applied to an electronic device, which may be a smart phone, a tablet computer, a digital camera, etc. There is no particular limitation on the type of electronic device. The process of the method is described as follows:

[0078] Step 101: An application program of an application layer sends a request for generating image simulation data to a hardware abstraction layer.

[0079] In an embodiment of the present application, when image simulation data needs to be generated, if the electronic device is a smart phone or a tablet computer, the user can click on the photo-taking option of the application installed in the smart phone or tablet computer, thereby triggering the application to start the photo-taking function. For example, the application can be a camera application installed by default when the smart phone or tablet computer leaves the factory, or it can be an installed third-party camera application; if the electronic device is a digital camera, the user can click on the physical button for taking pictures in the digital camera, thereby triggering the application in the digital camera to start the photo-taking function. It should be understood that whether it is an application in a smart phone, a tablet computer or a digital camera, it belongs to the application layer of the operating system it runs. That is to say, when the application in the application layer of the operating system of the electronic device is triggered to take a photo, the application in the application layer can send a request to the hardware abstraction layer to generate image simulation data.

[0080] Step 102: The hardware abstraction layer generates an acquisition request for image simulation data in response to the generation request, and sends the acquisition request to the control layer.

[0081] In an embodiment of the present application, after the hardware abstraction layer receives a request for generating image simulation data sent by an application program of the application layer, since the hardware abstraction layer cannot directly control the image sensor to generate image frames, the hardware abstraction layer can indirectly use the control layer to control the image sensor to generate image frames.

[0082] As a possible implementation, the hardware abstraction layer can respond to a request for generating image simulation data sent by an application program of the application layer, thereby generating a request for obtaining image simulation data, and sending the request to the control layer, thereby indirectly controlling the image sensor to generate image frames based on the control layer.

[0083] Step 103: The control layer responds to the acquisition request, reads at least one exposure parameter in a pre-stored configuration file, and writes the at least one exposure parameter into the image sensor.

[0084] In an embodiment of the present application, after receiving a request from the hardware abstraction layer to obtain image simulation data, the control layer can trigger the control layer to write exposure parameters into the image sensor so that the image sensor can generate a corresponding image frame.

[0085] As a possible implementation method, the control layer can respond to the hardware abstraction layer's request to obtain image simulation data, read at least one exposure parameter from a pre-stored configuration file, and write the at least one exposure parameter into the image sensor in sequence, so that the image sensor can generate corresponding image frames in sequence based on the writing order of the at least one exposure parameter.

[0086] Step 104: The control layer sends a notification message to the hardware abstraction layer before the exposure timing of the image sensor takes effect. The notification message is used to trigger the hardware abstraction layer to obtain an image frame at the exposure timing. The exposure timing is used to indicate the timing when the image sensor generates the corresponding target image frame based on the written target exposure parameters.

[0087] Any exposure parameter does not take effect immediately after being written to the image sensor. The image sensor itself has an exposure effective time, which can be considered as the time when the image sensor generates the corresponding target image frame based on the written target exposure parameters. Since the hardware abstraction layer cannot know the time when the control layer writes the target exposure parameters, it is unable to accurately obtain the target image frame at the exposure effective time. Therefore, in an embodiment of the present application, the control layer can notify the hardware abstraction layer to obtain the target image frame before the exposure effective time, so that the hardware abstraction layer can obtain the target image frame at the exposure effective time. The following is a detailed explanation of how the control layer notifies the hardware abstraction layer to obtain the target image frame before the exposure effective time.

[0088] Method 1: The hardware abstraction layer informs the control layer of the notification timing parameters in advance.

[0089] In an embodiment of the present application, the hardware abstraction layer pre-stores an exposure effective timing, which can be considered as the exposure effective timing sent by the image sensor to the hardware abstraction layer during the initialization phase of startup. The hardware abstraction layer can determine a notification timing parameter based on the exposure effective timing, and the notification timing parameter is used to represent the timing when the control layer needs to notify the hardware abstraction layer to obtain the image frame after the control layer writes the exposure parameter to the image sensor. The acquisition request sent by the hardware abstraction layer to the control layer can carry the above-mentioned notification timing parameter. Then, after the control layer writes the target exposure parameter to the image sensor, the control layer can send a notification message to the hardware abstraction layer based on the notification timing parameter to notify the hardware abstraction layer to obtain the target image frame corresponding to the target exposure parameter, so that the hardware abstraction layer can obtain the target image frame.

[0090] For example, if the exposure-validating timing is that the image frame corresponding to the exposure parameters can be generated in the third frame after the exposure parameters are written to the image sensor, then the notification timing parameter determined by the hardware abstraction layer based on the exposure-validating timing may be: sending a notification message to the hardware abstraction layer to notify the hardware abstraction layer to obtain the image frame in the second frame after the exposure parameters are written to the image sensor. When the exposure parameters are written to the image sensor and the second image frame is generated, the control layer sends a notification message to the hardware abstraction layer to notify the hardware abstraction layer to obtain the image frame, so that the hardware abstraction layer can accurately obtain the third image frame.

[0091] Method 2: The hardware abstraction layer informs the control layer in advance of the exposure effective time.

[0092] In an embodiment of the present application, the hardware abstraction layer pre-stores an exposure effective timing, and the exposure effective timing here can be considered as the exposure effective timing sent by the image sensor to the hardware abstraction layer during the initialization phase of startup. The acquisition request sent by the hardware abstraction layer to the control layer can carry the above-mentioned exposure effective timing. After receiving the exposure effective timing, the control layer can determine the notification timing parameter based on the exposure effective timing. The notification timing parameter is used to characterize the timing when the control layer needs to notify the hardware abstraction layer to obtain the image frame after writing the exposure parameter to the image sensor. Then, after the control layer writes the target exposure parameter to the image sensor, the control layer can send a notification message to the hardware abstraction layer based on the notification timing parameter to notify the hardware abstraction layer to obtain the target image frame corresponding to the target exposure parameter, so that the hardware abstraction layer can obtain the target image frame.

[0093] For example, if the exposure effective timing is that the image frame corresponding to the exposure parameters can be generated in the third frame after the exposure parameters are written to the image sensor, then after receiving the above exposure effective timing, the control layer can determine the notification timing parameter based on the exposure effective timing, that is, send a notification message to the hardware abstraction layer to notify the hardware abstraction layer to obtain the image frame in the second frame after the exposure parameters are written to the image sensor. When the exposure parameters are written to the image sensor and the second frame of image is generated, the control layer sends a notification message to the hardware abstraction layer to notify the hardware abstraction layer to obtain the image frame, so that the hardware abstraction layer can accurately obtain the third frame of image.

[0094] Method 3: The control layer reads the exposure effective timing from the configuration file and determines the notification timing parameters.

[0095] In an embodiment of the present application, in addition to including at least one exposure parameter, the configuration file may also include an exposure effective time. The control layer can respond to a request from the hardware abstraction layer to retrieve the exposure effective time from the configuration file. The control layer can then determine a notification timing parameter based on the exposure effective time. This notification timing parameter indicates the timing at which the control layer needs to notify the hardware abstraction layer to acquire an image frame after writing the exposure parameter to the image sensor. Once the control layer writes the target exposure parameter to the image sensor, the control layer can, based on the notification timing parameter, send a notification message to the hardware abstraction layer to notify the hardware abstraction layer to acquire the target image frame corresponding to the target exposure parameter, so that the hardware abstraction layer can acquire the target image frame.

[0096] For example, if the control layer reads from the configuration file that the exposure effective timing is that the image frame corresponding to the exposure parameters can be generated in the third frame after the exposure parameters are written to the image sensor, the control layer can determine the notification timing parameter based on the exposure effective timing, that is, send a notification message to the hardware abstraction layer to notify the hardware abstraction layer to obtain the image frame in the second frame after the exposure parameters are written to the image sensor. When the exposure parameters are written to the image sensor and the second frame of image is generated, the control layer sends a notification message to the hardware abstraction layer to notify the hardware abstraction layer to obtain the image frame, so that the hardware abstraction layer can accurately obtain the third frame of image.

[0097] It should be understood that in the above-mentioned methods 1, 2 and 3, whether the hardware abstraction layer determines the notification timing parameters based on the exposure effective timing, or the control layer determines the notification timing parameters based on the exposure effective timing, the time consumed by the control layer and the hardware abstraction layer in information transmission is taken into account, so that the hardware abstraction layer can accurately obtain the target image frame corresponding to the target exposure parameter.

[0098] Method 4: The control layer informs the hardware abstraction layer of the writing timing of the target exposure parameters.

[0099] In this embodiment of the present application, the control layer can respond to the hardware abstraction layer's acquisition request and sequentially write at least one exposure parameter read from the configuration file to the image sensor. When the control layer writes the target exposure parameter (any of the at least one exposure parameter) to the image sensor, the control layer can send a notification message to the hardware abstraction layer indicating that the target exposure parameter has been written to the image sensor. Based on the pre-stored exposure validity timing and the notification message, the hardware abstraction layer can accurately obtain the target image frame corresponding to the target exposure parameter.

[0100] For example, if a configuration file contains three sets of exposure parameters, the control layer can respond to the hardware abstraction layer's acquisition request and write the first, second, and third sets of exposure parameters to the image sensor, respectively. Taking the first set of exposure parameters as an example, upon writing the first set of exposure parameters to the image sensor, the control layer can send a notification message to the hardware abstraction layer indicating that the first set of exposure parameters has been written to the image sensor. After obtaining the write timing of the first set of exposure parameters and combining it with the pre-stored exposure activation timing, the hardware abstraction layer can accurately obtain the image frame corresponding to the first set of exposure parameters.

[0101] It should be understood that when the control layer in the above-mentioned method 4 informs the hardware abstraction layer that the target exposure parameters have been written into the image sensor, the time consumed by the control layer and the hardware abstraction layer in information transmission has been taken into account, so that the hardware abstraction layer can accurately obtain the target image frame corresponding to the target exposure parameters based on the exposure effective timing and the received notification message.

[0102] Step 105: The hardware abstraction layer obtains the target image frame output by the image sensor when the exposure is effective according to the notification message, and uses the target image frame as image simulation data.

[0103] In an embodiment of the present application, after the control layer sends a notification message to the hardware abstraction layer, the hardware abstraction layer can obtain the target image frame output by the image sensor when the exposure takes effect according to the notification message, and use the obtained target image frame as image simulation data for use by the image simulation algorithm. There is no special restriction on the type of image simulation algorithm that needs to use the image simulation data obtained by the hardware abstraction layer.

[0104] For example, the operating system of an electronic device also includes a driver layer, and the configuration file may also include a target number of image simulation data to be acquired. The control layer reads the target number in the configuration file by responding to the acquisition request of the hardware abstraction layer and sends the target number to the hardware abstraction layer. The hardware abstraction layer can separately apply for an equal number of buffer areas based on the target number and send the addresses of the buffer areas to the driver layer so that the image frames generated by the image sensor can be separately stored in the buffer areas via the driver layer.

[0105] It should be understood that image frames generated by the image sensor are stored sequentially in the buffers. For example, if the application provides three buffers, including a first buffer, a second buffer, and a third buffer, then when storing the same number of image frames, the image frame generated by the first exposure parameter written is written to the first buffer, the image frame generated by the second exposure parameter written is written to the second buffer, and the image frame generated by the last exposure parameter written is written to the third buffer.

[0106] Because the hardware abstraction layer knows which buffers store the image frames it needs, it can sequentially retrieve image frames from the driver layer based on the buffer addresses. As a possible implementation, the hardware abstraction layer can send a retrieval instruction to the driver layer, which can include the address of the target buffer where the image frame is located. The driver layer then feeds back the target image frame stored at the target buffer address to the hardware abstraction layer based on the target buffer address.

[0107] See Figure 2 Based on the same inventive concept, an embodiment of the present application provides an electronic device, which includes a hardware abstraction layer 201, an application layer 202, and a control layer 203. The electronic device also includes an image sensor:

[0108] The application program of the application layer 202 sends a request for generating image simulation data to the hardware abstraction layer 201;

[0109] The hardware abstraction layer 201 generates an acquisition request for the image simulation data in response to the generation request, and sends the acquisition request to the control layer 203;

[0110] The control layer 203 responds to the acquisition request, reads at least one exposure parameter in a pre-stored configuration file, and writes the at least one exposure parameter into the image sensor;

[0111] The control layer 203 sends a notification message to the hardware abstraction layer 201 before the exposure timing of the image sensor. The notification message is used to trigger the hardware abstraction layer 201 to obtain an image frame at the exposure timing. The exposure timing is used to indicate the timing when the image sensor generates a corresponding image frame based on the written target exposure parameters.

[0112] The hardware abstraction layer 201 obtains the target image frame output by the image sensor at the exposure effective timing according to the notification message, and uses the target image frame as image simulation data.

[0113] Optionally, the acquisition request carries a notification timing parameter. The notification timing parameter is determined by the hardware abstraction layer 201 based on a pre-stored exposure validity timing and is used to indicate the timing at which the control layer 203 notifies the hardware abstraction layer 201 to acquire an image frame after the control layer writes the exposure parameters to the image sensor. The control layer 203 is specifically used to:

[0114] After the target exposure parameters are written into the image sensor, a notification message for notifying the hardware abstraction layer 201 to obtain a target image frame corresponding to the target exposure parameters is sent to the hardware abstraction layer 201 based on the notification timing parameters.

[0115] Optionally, the acquisition request carries the exposure validity timing, and the control layer 203 is specifically configured to:

[0116] Determining a notification timing parameter based on the exposure effective timing, where the notification timing parameter is used to represent the timing at which the control layer 203 notifies the hardware abstraction layer 201 to obtain an image frame after the control layer 203 writes the exposure parameters into the image sensor;

[0117] After the target exposure parameters are written into the image sensor, a notification message for notifying the hardware abstraction layer 201 to obtain a target image frame corresponding to the target exposure parameters is sent to the hardware abstraction layer 201 based on the notification timing parameter.

[0118] Optionally, the control layer 203 is further configured to:

[0119] Respond to the acquisition request and read the exposure effective time in the configuration file;

[0120] The control layer 203 is specifically used for:

[0121] Determining a notification timing parameter based on the exposure effective timing, where the notification timing parameter is used to represent the timing at which the control layer 203 notifies the hardware abstraction layer 201 to obtain an image frame after the control layer 203 writes the exposure parameters into the image sensor;

[0122] After the target exposure parameters are written into the image sensor, a notification message for notifying the hardware abstraction layer 201 to obtain a target image frame corresponding to the target exposure parameters is sent to the hardware abstraction layer 201 based on the notification timing parameter.

[0123] Optionally, the control layer 203 is specifically configured to:

[0124] Sending a notification message to the hardware abstraction layer 201 indicating that the target exposure parameters have been written into the image sensor;

[0125] The hardware abstraction layer 201 is specifically used for:

[0126] The target image frame output by the image sensor at the exposure effective timing is acquired according to the pre-stored exposure effective timing and the notification message.

[0127] Optionally, the electronic device further includes a driver layer, the configuration file further includes a target quantity of image simulation data to be generated, and the control layer 203 is further configured to:

[0128] Respond to the get request and read the target quantity in the configuration file;

[0129] The control layer 203 is further configured to: send a target quantity to the hardware abstraction layer 201;

[0130] The hardware abstraction layer 201 is further configured to apply for an equal number of buffer areas based on the target number, and send the address of each buffer area to the driver layer.

[0131] Optionally, the target image frame is stored in the target buffer area via the driver layer, and the hardware abstraction layer 201 is specifically used for:

[0132] Send an acquisition instruction to the driver layer, which includes the address of the target buffer area;

[0133] The receiving driver layer feeds back a target image frame based on the address of the target buffer area.

[0134] See Figure 3 Based on the same inventive concept, the embodiment of the present application further provides an electronic device 100, which may include at least one processor, the at least one processor being configured to execute a computer program stored in a memory to implement the embodiment of the present application. Figure 1 The steps of the method for generating image simulation data are shown.

[0135] Optionally, the processor may specifically be a central processing unit, a specific ASIC, or one or more integrated circuits for controlling program execution.

[0136] Optionally, the electronic device 100 may further include a memory connected to at least one processor, and the memory may include ROM, RAM, and disk storage. The memory is used to store data required by the processor when it is running, that is, it stores instructions that can be executed by at least one processor. At least one processor executes the instructions stored in the memory to execute the following operations: Figure 1 The method shown in FIG.

[0137] The physical devices corresponding to the hardware abstraction layer 201, the application layer 202 and the control layer 203 can all be the aforementioned processors. Figure 1 Therefore, for the functions that can be realized by each functional module in the electronic device, please refer to Figure 1 The corresponding description in the illustrated embodiment will not be repeated here.

[0138] The electronic device 100 may be a smart phone, tablet computer or other intelligent electronic device. This embodiment does not limit the form of the electronic device.

[0139] Instance, Figure 3 Taking a smart phone as an example, a schematic diagram of the structure of the electronic device 100 is shown. Figure 3 As shown, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0140] It should be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0141] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0142] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0143] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0144] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0145] The charging management module 140 is configured to receive charging input from a charger.

[0146] The power management module 141 is used to connect the battery 142 , the charging management module 140 and the processor 110 .

[0147] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0148] The electronic device 100 implements the display function through a GPU, a display screen 194 , and an application processor.

[0149] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel.

[0150] The ISP is used to process data fed back by the camera 193 .

[0151] The camera 193 is used to capture still images or videos.

[0152] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0153] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. This allows electronic device 100 to play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, and MPEG4.

[0154] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 via the external memory interface 120 to implement data storage functions. For example, files such as music and videos can be stored on the external memory card.

[0155] The internal memory 121 can be used to store computer executable program codes, which include instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device 100 by running instructions stored in the internal memory 121 and / or instructions stored in a memory provided in the processor.

[0156] The electronic device 100 can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0157] The audio module 170 is used to convert digital audio information into analog audio signals for output, and is also used to convert analog audio input into digital audio signals.

[0158] The speaker 170A, also called a "horn", is used to convert audio electrical signals into sound signals.

[0159] The receiver 170B, also called a "handset", is used to convert audio electrical signals into sound signals.

[0160] Microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals.

[0161] The headphone jack 170D is used to connect a wired headphone and can be the USB interface 130 or a 3.5mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0162] The buttons 190 include a power button, a volume button, and the like.

[0163] Motor 191 can generate vibration prompts.

[0164] The indicator 192 may be an indicator light, which may be used to indicate the charging status, power level changes, messages, missed calls, notifications, etc.

[0165] The SIM card interface 195 is used to connect a SIM card. In some embodiments, the electronic device 100 uses an eSIM, or embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.

[0166] The embodiment of the present application also provides a computer storage medium, wherein the computer storage medium stores computer instructions, which, when executed on a computer, cause the computer to execute the following Figure 1 The method described.

[0167] The above description is only a preferred embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of this specification should be included in the scope of protection of this specification.

Claims

1. A method for generating image simulation data, characterized in that: Applied to an electronic device, wherein the operating system of the electronic device includes a hardware abstraction layer, an application layer, and a control layer, the method includes: The application program of the application layer sends a generation request for image simulation data to the hardware abstraction layer; The hardware abstraction layer generates an acquisition request for the image simulation data in response to the generation request, and sends the acquisition request to the control layer; The control layer reads at least one exposure parameter in a pre-stored configuration file in response to the acquisition request, and writes the at least one exposure parameter into the image sensor; The control layer sends a notification message to the hardware abstraction layer before the exposure effective timing of the image sensor, wherein the notification message is used to trigger the hardware abstraction layer to obtain an image frame at the exposure effective timing, and the exposure effective timing is used to indicate the timing when the image sensor generates a corresponding target image frame based on the written target exposure parameters; The hardware abstraction layer obtains the target image frame output by the image sensor at the exposure effective time according to the notification message, and uses the target image frame as the image simulation data.

2. The method according to claim 1, characterized in that The acquisition request carries a notification timing parameter, which is determined by the hardware abstraction layer based on the pre-stored exposure validity timing and is used to represent a timing at which the control layer notifies the hardware abstraction layer to acquire an image frame after the control layer writes the exposure parameter to the image sensor. The control layer sends a notification message to the hardware abstraction layer before the exposure validity timing of the image sensor, including: After the control layer writes the target exposure parameter into the image sensor, the control layer sends a notification message to the hardware abstraction layer based on the notification timing parameter, which is used to notify the hardware abstraction layer to obtain a target image frame corresponding to the target exposure parameter.

3. The method according to claim 1, characterized in that The acquisition request carries the exposure effective timing, and the control layer sends a notification message to the hardware abstraction layer before the exposure effective timing of the image sensor, including: The control layer determines a notification timing parameter based on the exposure validation timing, where the notification timing parameter is used to represent a timing at which the control layer notifies the hardware abstraction layer to acquire an image frame after the control layer writes the exposure parameter into the image sensor; After the control layer writes the target exposure parameter into the image sensor, the control layer sends a notification message to the hardware abstraction layer based on the notification timing parameter, which is used to notify the hardware abstraction layer to obtain a target image frame corresponding to the target exposure parameter.

4. The method according to claim 1, wherein Before the control layer sends a notification message to the hardware abstraction layer before the exposure of the image sensor takes effect, the method further includes: The control layer responds to the acquisition request and reads the exposure validity timing in the configuration file; The control layer sending a notification message to the hardware abstraction layer before the exposure of the image sensor takes effect includes: The control layer determines a notification timing parameter based on the exposure validation timing, where the notification timing parameter is used to represent a timing at which the control layer notifies the hardware abstraction layer to acquire an image frame after the control layer writes the exposure parameter into the image sensor; After the control layer writes the target exposure parameter into the image sensor, the control layer sends a notification message to the hardware abstraction layer based on the notification timing parameter, which is used to notify the hardware abstraction layer to obtain a target image frame corresponding to the target exposure parameter.

5. The method according to claim 1, wherein The control layer sending a notification message to the hardware abstraction layer before the exposure of the image sensor takes effect includes: The control layer sends a notification message to the hardware abstraction layer indicating that the target exposure parameter has been written into the image sensor; The hardware abstraction layer acquiring, according to the notification message, a target image frame output by the image sensor at the exposure effective time, includes: The hardware abstraction layer obtains the target image frame output by the image sensor at the exposure validation timing according to the pre-stored exposure validation timing and the notification message.

6. The method according to claim 1, characterized in that The operating system of the electronic device further includes a driver layer, the configuration file further includes a target amount of image simulation data to be generated, and before the control layer sends a notification message to the hardware abstraction layer before the exposure of the image sensor takes effect, the method further includes: The control layer responds to the acquisition request and reads the target quantity in the configuration file; The control layer sends the target quantity to the hardware abstraction layer; The hardware abstraction layer applies for an equal number of buffer areas based on the target number, and sends the address of each buffer area to the driver layer.

7. The method according to claim 6, characterized in that The target image frame is stored in the target buffer area via the driver layer, and the hardware abstraction layer obtains the target image frame corresponding to the exposure effective timing of the image sensor according to the notification message, including: The hardware abstraction layer sends an acquisition instruction to the driver layer, where the acquisition instruction includes the address of the target buffer area; The hardware abstraction layer receives the target image frame fed back by the driver layer based on the address of the target buffer area.

8. An electronic device, characterized in that: The electronic device includes a hardware abstraction layer, an application layer, and a control layer, and further includes an image sensor: The application program of the application layer sends a generation request for image simulation data to the hardware abstraction layer; The hardware abstraction layer generates an acquisition request for the image simulation data in response to the generation request, and sends the acquisition request to the control layer; The control layer reads at least one exposure parameter in a pre-stored configuration file in response to the acquisition request, and writes the at least one exposure parameter into the image sensor; The control layer sends a notification message to the hardware abstraction layer before the exposure effective timing of the image sensor, wherein the notification message is used to trigger the hardware abstraction layer to obtain an image frame at the exposure effective timing, and the exposure effective timing is used to indicate the timing when the image sensor generates a corresponding image frame based on the written target exposure parameters; The hardware abstraction layer obtains the target image frame output by the image sensor at the exposure effective time according to the notification message, and uses the target image frame as the image simulation data.

9. The electronic device according to claim 8, wherein: The acquisition request carries a notification timing parameter, which is determined by the hardware abstraction layer based on the pre-stored exposure validity timing and is used to indicate the timing at which the control layer notifies the hardware abstraction layer to acquire an image frame after the control layer writes the exposure parameter to the image sensor. The control layer is specifically configured to: After the target exposure parameter is written into the image sensor, a notification message for notifying the hardware abstraction layer to obtain a target image frame corresponding to the target exposure parameter is sent to the hardware abstraction layer based on the notification timing parameter.

10. The electronic device according to claim 8, wherein The acquisition request carries the exposure validity timing, and the control layer is specifically used to: determining a notification timing parameter based on the exposure validation timing, wherein the notification timing parameter is used to represent a timing at which the control layer notifies the hardware abstraction layer to acquire an image frame after the control layer writes the exposure parameter into the image sensor; After the target exposure parameter is written into the image sensor, a notification message for notifying the hardware abstraction layer to obtain a target image frame corresponding to the target exposure parameter is sent to the hardware abstraction layer based on the notification timing parameter.

11. The electronic device according to claim 8, wherein The control layer is also used to: In response to the acquisition request, reading the exposure effective timing in the configuration file; The control layer is specifically used for: determining a notification timing parameter based on the exposure validation timing, wherein the notification timing parameter is used to represent a timing at which the control layer notifies the hardware abstraction layer to acquire an image frame after the control layer writes the exposure parameter into the image sensor; After the target exposure parameter is written into the image sensor, a notification message for notifying the hardware abstraction layer to obtain a target image frame corresponding to the target exposure parameter is sent to the hardware abstraction layer based on the notification timing parameter.

12. The electronic device according to claim 8, wherein The control layer is specifically used for: Sending a notification message to the hardware abstraction layer indicating that the target exposure parameter has been written into the image sensor; The hardware abstraction layer is specifically used for: The target image frame output by the image sensor at the exposure effective timing is acquired according to the pre-stored exposure effective timing and the notification message.

13. The electronic device according to claim 8, wherein The operating system of the electronic device further includes a driver layer, the configuration file further includes a target number of image simulation data to be generated, and the control layer is further configured to: respond to the acquisition request and read the target number in the configuration file; The control layer is further configured to: send the target quantity to the hardware abstraction layer; The hardware abstraction layer is further configured to apply for an equal number of buffer areas based on the target number, and send the address of each buffer area to the driver layer.

14. The electronic device according to claim 13, wherein: The target image frame is stored in the target buffer area via the driver layer, and the hardware abstraction layer is specifically used for: Sending an acquisition instruction to the driver layer, wherein the acquisition instruction includes the address of the target buffer area; The target image frame fed back by the driving layer based on the address of the target buffer area is received.

15. An electronic device, characterized in that: The electronic device includes at least one processor and a memory connected to the at least one processor, and the at least one processor is configured to implement the steps of the method according to any one of claims 1 to 7 when executing a computer program stored in the memory.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images