Image data acquisition method and device, terminal and storage medium

Abstract

The embodiment of the application discloses an image data acquisition method and device, a terminal and a storage medium, and belongs to the technical field of image processing. The method comprises the following steps: acquiring image data through a first image sensor, the first image sensor operates in a first working mode, and a second image sensor is in a non-working mode; in response to determining that a current image recognition scene does not match the first working mode, configuring the second image sensor into a second working mode based on the current image recognition scene; and in response to completing the working mode configuration of the second image sensor, acquiring image data through the second image sensor. The phenomenon of image data flow interruption caused by mode switching is avoided, and the continuity and timeliness of subsequent image recognition are ensured.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of image processing, and particularly relate to an image data acquisition method and device, a terminal, and a storage medium. BACKGROUND

[0002] With the development of Internet technology, terminal devices need to have image acquisition functions for different image recognition scenarios, such as face recognition scenarios, gesture recognition scenarios, and the like. Different image recognition scenarios have different requirements for images acquired by the terminal.

[0003] Currently, image sensors with multiple working modes are configured in the terminal for different recognition scenarios, so that the working mode of the image sensor can be switched to adapt to different image recognition scenarios. However, the image sensor will cause the image data stream to be interrupted during the working mode switching process, resulting in no image data stream output for a certain period of time, and thus continuous image recognition processing cannot be performed. SUMMARY

[0004] Embodiments of the present application provide an image data acquisition method and device, a terminal, and a storage medium. The technical solution is as follows:

[0005] In one aspect, an image data acquisition method is provided. The method is applied to a terminal, and the terminal is provided with a first image sensor and a second image sensor. The method comprises the following steps:

[0006] acquiring image data by the first image sensor, the first image sensor operating in a first working mode, and the second image sensor being in a non-working mode;

[0007] in response to determining that a current image recognition scenario does not match the first working mode, configuring the second image sensor into a second working mode based on the current image recognition scenario, wherein the first working mode and the second working mode correspond to different image recognition scenarios respectively, and the second working mode matches the current image recognition scenario;

[0008] in response to the configuration of the working mode of the second image sensor being completed, acquiring image data by the second image sensor.

[0009] In another aspect, an image data acquisition device is provided. The device is applied to a terminal, and the terminal is provided with a first image sensor and a second image sensor. The device comprises the following modules:

[0010] a first acquisition module, configured to acquire image data by the first image sensor, the first image sensor operating in a first working mode, and the second image sensor being in a non-working mode;

[0011] The configuration module is configured to, in response to determining that the current image recognition scene does not match the first working mode, configure the second image sensor into a second working mode based on the current image recognition scene, wherein the first working mode and the second working mode correspond to different image recognition scenes respectively, and the second working mode matches the current image recognition scene.

[0012] The second acquisition module is configured to, in response to the configuration of the working mode of the second image sensor being completed, acquire image data through the second image sensor.

[0013] In another aspect, an embodiment of the present application provides a terminal, which comprises a processor and a memory, and the memory stores at least one program, which is loaded and executed by the processor to implement the image data acquisition method according to the above aspect.

[0014] In another aspect, an embodiment of the present application provides a computer readable storage medium, which stores at least one instruction, and the at least one instruction is used to be executed by a processor to implement the image data acquisition method according to the above aspect.

[0015] According to another aspect of the present application, a computer program product or computer program is provided, which comprises computer instructions stored in a computer readable storage medium. A processor of a terminal reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to enable the terminal to perform the image data acquisition method provided in the optional implementation manner.

[0016] The technical scheme provided by the embodiment of the present application has at least the following beneficial effects:

[0017] By arranging at least two image sensors, such as a first image sensor and a second image sensor, in the terminal and keeping only one image sensor in a working mode at the same time, when the terminal determines that the current image recognition scene does not match the working mode of the first image sensor, the second image sensor can be configured into a second working mode matching the current image recognition scene while keeping the first image sensor continuously outputting image data, and then the image data output by the second image sensor is used for image content recognition. This makes it possible to avoid the phenomenon of image data flow interruption caused by mode switching, and ensures the continuity and timeliness of subsequent image recognition. At the same time, the image sensor can be kept in a working mode matching the current image recognition scene, further improving the accuracy of subsequent image content recognition. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 A workflow diagram of a normally open image sensor in the related art is shown;

[0019] Figure 2 A workflow diagram of an image data acquisition method according to an example embodiment of the present application is shown;

[0020] Figure 3 A workflow diagram of an image data acquisition method according to another example embodiment of the present application is shown;

[0021] Figure 4 A workflow diagram of an image data acquisition method according to another example embodiment of the present application is shown;

[0022] Figure 5 A workflow diagram of an image data acquisition method according to another example embodiment of the present application is shown;

[0023] Figure 6 A workflow diagram of an image data acquisition method according to an example embodiment of the present application is shown;

[0024] Figure 7 A schematic diagram of a processing procedure of image data acquired by an image sensor in a terminal is shown;

[0025] Figure 8 A structural block diagram of an image data acquisition apparatus according to an example embodiment of the present application is shown;

[0026] Figure 9 A structural block diagram of a terminal according to an example embodiment of the present application is shown. DETAILED DESCRIPTION

[0027] For the purpose, technical solutions and advantages of the present application to be clearer, the embodiments of the present application will be further described in detail below with reference to the drawings.

[0028] In the present document, "a plurality of" refers to two or more. "And / or" describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent the following three cases: A exists alone, A and B exist together, and B exists alone. The character " / " generally represents that the associated objects before and after it are in an "or" relationship.

[0029] With the increase of terminal image recognition scenarios, the image information concerned by the image sensor is different in different recognition scenarios. For example, in a face recognition scenario, more attention is paid to facial features, and in a gesture recognition scenario, more attention is paid to finger movement trend features. Therefore, in order to adapt to different image recognition scenarios, a Always ON sensor is developed. The Always ON sensor has two working modes. One is an MD (Motion Detection) mode, which can be used to detect the relative motion change of an object, such as for a gesture recognition scenario. The other is an HE (Human Existence) mode, which can be used to detect target object features, such as detecting whether a face exists in an image. For illustration, please refer to Figure 1 which shows a working flowchart of the Always ON sensor in the related art. In the working flowchart of the Always ON sensor, the main modules involved are the chip hardware abstraction layer, the chip driver layer, the Always ON sensor driver layer, and the Always ON sensor. As shown in Figure 1 , it can be seen that the working flow of the Always ON sensor involves the power-on / power-off process, the configuration process, and the flow-on / flow-off process of the image sensor. When the mode switching process of the Always ON sensor is needed, since the configuration parameters of the image sensor are different in different working modes, the Always ON sensor needs to be reconfigured and restarted during the corresponding mode switching process. Each step will consume a certain amount of time, resulting in no image data flow output during the mode switching process, so that continuous image recognition processing cannot be realized, and thus the accuracy of image recognition is affected.

[0030] Based on the problems in the related art, the embodiments of the present application provide a new image data acquisition method. Two image sensors are set in the terminal, such as a first image sensor and a second image sensor, and only one image sensor is kept in working mode at the same time. The first image sensor and the second image sensor can be the Always ON sensor described above, or other types of image sensors. When the terminal determines that the current image recognition scenario does not match the working mode of the first image sensor, the second image sensor can be configured to a second working mode that matches the current image recognition scenario while keeping the first image sensor continuously outputting image data, and then the image data output by the second image sensor is used for image content recognition. This makes it possible to avoid the phenomenon of image data flow interruption caused by mode switching, ensures the continuity and timeliness of subsequent image recognition, and makes the image sensor work in a working mode that matches the current image recognition scenario, further improving the accuracy of subsequent image content recognition.

[0031] Reference is made to Figure 2 which shows a flow chart of an image data collection method according to an example embodiment of the present application. The example embodiment is exemplarily described by taking the method applied to a terminal as an example, and the method comprises the following steps:

[0032] In step 201, image data is collected by a first image sensor, the first image sensor operates in a first working mode, and a second image sensor is in a non-working mode.

[0033] In the example embodiment, at least two image sensors, i.e. the first image sensor and the second image sensor, are provided in the terminal, which are different from the front camera in the terminal. The image sensor is generally in an always-on state, and the image data collected by the image sensor is used for image content recognition in the software end. Illustratively, the image data collected by the image sensor is used for target object recognition (such as face recognition), or for motion trend recognition (such as hand gesture recognition), etc.

[0034] Optionally, the first image sensor and the second image sensor can be image sensors with the same working mode. Illustratively, the first image sensor and the second image sensor can both be always-on image sensors, or can be other types of image sensors.

[0035] The first image sensor and the second image sensor each have at least two working modes, and different working modes correspond to different image recognition scenarios. Illustratively, when the image sensor is an always-on image sensor, it has MD and HE two working modes. The MD working mode can be used to detect the relative motion change of the object, i.e. the motion trend recognition scenario, and the HE working mode can be used to detect whether there is a person in the image, i.e. the target object recognition scenario.

[0036] In a possible implementation, when the terminal is provided with the first image sensor and the second image sensor, when the terminal is turned on, the first image sensor is started by default to collect image data, and the first image sensor operates in a default working mode (first working mode), and the second image sensor is in a non-working mode, so as to reduce the power consumption of the terminal. That is, only one image sensor is in the working mode at the same time.

[0037] Optionally, the first image sensor can be any one of the at least two image sensors in the terminal, and the first working mode can be any one of the working modes possessed by the first image sensor, which is not limited in the example embodiment.

[0038] Optionally, the first working mode can also be a historical working mode of the first image sensor. Illustratively, if the first image sensor is in the MD working mode before the current is turned off, MD is taken as the first working mode; or if the first image sensor is in the HE working mode before the power is turned off, HE is taken as the first working mode.

[0039] In step 202, in response to determining that the current image recognition scene does not match the first working mode, the second image sensor is configured to a second working mode based on the current image recognition scene, wherein the first working mode and the second working mode respectively correspond to different image recognition scenes, and the second working mode matches the current image recognition scene.

[0040] Optionally, since there are various image recognition scenes, and the required image data under different image recognition scenes has different requirements for the image acquisition parameters of the image sensor, in order to adapt to different image recognition scenes, the image sensor has at least two working modes, and the configuration parameters of the image sensor are different under different working modes, and different working modes correspond to different image recognition scenes.

[0041] If a single image sensor is used to adapt to different image recognition scenes, if the working mode of the image sensor does not match the current image recognition scene, the image sensor needs to be switched to the working mode, and the working mode switching process involves the processes of reconfiguring parameters, turning off the current, and restarting the current, so that there is no image data stream output in the working mode switching process, which further affects the subsequent image content recognition process. Therefore, in order to avoid the interruption of the image data stream in the working mode switching process, at least two image sensors are provided in the terminal, so that when it is determined that the current image recognition scene does not match the first working mode, the first image sensor still operates in the first working mode, and the second image sensor is configured to a second working mode that matches the current image recognition scene based on the current image recognition scene. That is, the second working mode and the first working mode respectively correspond to different image recognition scenes, so that the first image sensor can still output image data in the process of reconfiguring parameters for subsequent image content recognition (that is, in the process of configuring the second image sensor to the second working mode, the first image sensor maintains the first working mode to output image data).

[0042] Illustratively, when the second image sensor is configured to the working mode, the configuration parameters involved can include the related parameters of the second image sensor, the storage location of the image data output by the second image sensor, etc. Optionally, it can also include the resolution of the acquired image data, the frame rate, etc. The embodiments of the present application do not constitute a limitation on the configuration parameters.

[0043] Optionally, if the second image sensor is in the powered-off state, when the terminal determines that the current image recognition scenario does not match the working mode (the first working mode) of the first image sensor, the terminal performs the power-on processing on the second image sensor while maintaining the continuous image output of the first image sensor, configures the second image sensor to the second working mode based on the current image recognition scenario, and then performs the flow-on processing.

[0044] Optionally, if the second image sensor is in the flow-off state (a state in which the second image sensor is not powered off but does not output image data flow), when the terminal determines that the current image recognition scenario does not match the working mode (the first working mode) of the first image sensor, the terminal configures the second image sensor to the second working mode while maintaining the continuous image output of the first image sensor, and performs the flow-on processing on the second image sensor.

[0045] In step 203, in response to the completion of the configuration of the working mode of the second image sensor, the terminal acquires image data through the second image sensor.

[0046] In a possible implementation, after the configuration of the working mode of the second image sensor is completed, the terminal can perform the flow-on processing on the second image sensor, so that the terminal can acquire image data through the second image sensor.

[0047] Optionally, after the second image sensor can output the image data flow, in order to reduce the power consumption of the terminal, the terminal can perform the flow-off processing or the power-off processing on the first image sensor, so as to stop acquiring image data through the first image sensor.

[0048] Optionally, after the second image sensor can output the image data flow, in order to avoid the inaccuracy of the image recognition result due to the sudden switching of the working mode, the terminal can temporarily not close the first image sensor, simultaneously use the first image sensor and the second image sensor to acquire image data within a period of time, cross-read the two image data flows acquired by the first image sensor and the second image sensor, gradually reduce the reading proportion of the image data acquired by the first image sensor, so that the image data can gradually transit to the image data acquired by the second image sensor completely, and then perform the flow-off processing or the power-off processing on the first image sensor, so as to stop acquiring image data through the first image sensor.

[0049] In the mode switching process caused by the image recognition scene, unlike the operation of reconfiguring parameters and restarting flow of a single image sensor in the related art, the mode switching process is completed by two image sensors in the embodiment of the application. Since the original image sensor (the first image sensor) is still in the working mode during the mode switching process, the output of image data can be maintained, thereby avoiding the flow interruption of image data in the subsequent image content recognition process. When the new image sensor operates in the second working mode, the current working mode can be matched with the current image recognition scene.

[0050] Optionally, after the second image sensor can output image data, the image data is processed by an image processing algorithm at the software end. Illustratively, the image processing algorithm can be an image recognition algorithm, an image edge detection algorithm, an image segmentation algorithm, an image enhancement algorithm, etc.

[0051] Illustratively, taking the image processing algorithm as an image recognition algorithm as an example, the specific recognition content of the image content recognition is related to the current image recognition scene. If the current image recognition scene is a gesture recognition scene, the image content recognition needs to recognize whether a gesture is contained in the image frame corresponding to the output image data and whether the gesture matches a preset gesture, thereby determining the recognition result. If the current image recognition scene is a face recognition scene, the image content recognition needs to recognize whether a face is contained in the image frame corresponding to the output image data and whether a face feature matches a preset face feature, etc.

[0052] To sum up, in the embodiment of the application, at least two image sensors, such as the first image sensor and the second image sensor, are arranged in the terminal, and only a single image sensor is kept in the working mode at the same time. When the terminal determines that the current image recognition scene does not match the working mode of the first image sensor, the second image sensor can be configured to the second working mode matching the current image recognition scene while keeping the first image sensor continuously outputting image data, and then the image data output by the second image sensor is used for image content recognition. Therefore, during the mode switching process, the image data is not output for a period of time due to the parameter configuration and the flow interruption and starting operation, thereby avoiding the phenomenon of image data flow interruption caused by mode switching, ensuring the continuity and timeliness of the subsequent image recognition. At the same time, the image sensor can be in the working mode matching the current image recognition scene, thereby further improving the accuracy of the subsequent image content recognition.

[0053] Since the image sensor in the embodiment of the present application has two working modes, and the image sensor itself has the image detection function, in order to improve the timeliness of the image recognition scene detection, and further make the image sensor switch to the adaptive working mode to collect image data as soon as possible, in a possible implementation, the image recognition scene detection is performed by the image sensor currently in the working mode, and then the working mode switching process is triggered by the image sensor.

[0054] Please refer to Figure 3 which shows a flowchart of an image data collection method according to another exemplary embodiment of the present application. The embodiment is exemplarily described taking the case that the method is applied to a terminal. The method comprises:

[0055] In step 301, image data is collected by a first image sensor, the first image sensor operates in a first working mode, and a second image sensor is in a non-working mode.

[0056] Optionally, the first image sensor or the second image sensor comprises a first working mode and a second working mode, the first working mode corresponds to a motion trend recognition scene, and the motion trend recognition scene can be gesture recognition, gait recognition, head action recognition, etc. illustratively; the second working mode corresponds to a target object recognition scene; and the target object recognition scene can be a face recognition scene, an iris recognition scene, an eye feature recognition scene, a fingerprint recognition scene, etc. illustratively.

[0057] In step 302, the collected image data is subjected to scene recognition by the first image sensor, to determine that the current image recognition scene does not match the first working mode.

[0058] In the embodiment of the present application, the first image sensor and the second image sensor both have the function of image detection on image data, and further can determine the current image recognition scene based on the image detection result; in a possible implementation, when the first image sensor operates in the first working mode, the first image sensor can perform scene recognition on the collected image data, to determine the current image recognition scene indicated by the image data, and further determine whether the first working mode matches the current image recognition scene.

[0059] Optionally, when the first image sensor determines that the current image recognition scene does not match the first working mode, the first image sensor triggers the subsequent configuration of the second image sensor into the second working mode; otherwise, when the first image sensor determines that the current image recognition scene matches the first working mode, the first image sensor continues to collect image data.

[0060] Illustratively, if the first image sensor includes both MD and HE working modes, and the first image sensor operates in the MD working mode, after the first image sensor performs image detection based on the acquired image data, it is determined that the image data corresponds to an image frame containing a face, and the face position in the adjacent image frame does not change significantly, that is, there is no motion trend, it is determined that the current image recognition scene is a target object recognition scene (a face recognition scene), which does not match the current MD working mode, and a subsequent mode switching process is required; if the first image sensor performs image detection based on the acquired image data, it is determined that the image data corresponds to an image frame not containing a face, and it is detected that there is a gesture motion trend in the adjacent image frame, it is determined that the current image recognition scene is a motion trend recognition scene, which matches the current MD working mode, and there is no need to perform mode switching, and the first image sensor continues to output image data.

[0061] Optionally, in order to reduce the power consumption of the terminal, the first image sensor or the second image sensor can perform scene recognition every preset image frame number or every preset time period when performing scene recognition.

[0062] Step 303, in response to determining that the current image recognition scene does not match the first working mode, triggering the second image sensor to be configured to the second working mode through the first image sensor.

[0063] In a possible implementation, after the first image sensor performs scene recognition on the collected image data and obtains the current image recognition scene, it can be directly judged whether the current image recognition scene matches the first working mode, if it matches, it means that there is no need to perform mode switching, and then the first image sensor continues to operate in the first working mode; otherwise, if it does not match, the subsequent process of configuring the second image sensor to the second working mode needs to be performed.

[0064] Since the scene recognition is performed by the first image sensor currently in the working mode, and it is judged whether the current image recognition scene matches the first working mode, in order to timely notify the second image sensor to adapt to the current image recognition scene, in a possible implementation, the first image sensor directly triggers the second image sensor to be configured to the second working mode, the mode switching is triggered by the lower layer, and there is no need for manual triggering, and then the mode switching instruction is issued by the upper layer.

[0065] If the second image sensor needs to be triggered to adapt to the current image recognition scene, in a possible implementation, the first image sensor needs to send the current image recognition scene to the second image sensor or the control module of the second image sensor, so that the second image sensor can perform the working mode configuration process based on the current image recognition scene. In an exemplary example, step 303 can further include step 303A and step 303B.

[0066] In response to determining that the current image recognition scene does not match the first working mode, a first configuration instruction is sent to the second image sensor by the first image sensor, and the first configuration instruction contains the current image recognition scene.

[0067] In order to enable the second image sensor to timely configure the working mode, in a possible implementation, the first image sensor can directly send a first configuration instruction to the second image sensor, and the first configuration instruction contains the current image recognition scene, so that the second image sensor can configure the working mode based on the current image recognition scene, and implement adaptation of the working mode to the current image recognition scene.

[0068] Optionally, the second image sensor can not need to directly send the second configuration instruction to the second image sensor, but can send the second configuration instruction to the control chip, and the control chip controls the second image sensor to configure the mode.

[0069] In response to the second image sensor receiving the first configuration instruction, the second image sensor is configured to the second working mode.

[0070] Correspondingly, after the second image sensor receives the first configuration instruction, the second working mode that needs to be configured can be determined based on the current image recognition scene contained in the first configuration instruction, and then the second image sensor is configured to the second working mode.

[0071] Optionally, after the control chip receives the first configuration instruction, the second image sensor can be configured to the second working mode based on the second working mode corresponding to the first configuration instruction.

[0072] In response to the second image sensor completing the configuration of the working mode, image data is collected by the second image sensor.

[0073] The implementation of step 304 can refer to the foregoing embodiments, and will not be described here in detail.

[0074] Step 305: stopping collecting image data by the first image sensor.

[0075] In a possible implementation, when the first image sensor determines that the current image recognition scene does not match the first working mode, the second image sensor is triggered to be configured to the second working mode, and in the process of configuring the working mode of the second image sensor, in order to avoid image data interruption, image data is still collected by the first image sensor, and until the second image sensor completes the configuration of the second working mode, in order to reduce terminal power consumption, collecting image data by the first image sensor can be stopped.

[0076] In response to the scene switching frequency being less than the frequency threshold, performing a power-off process on the first image sensor, the scene switching frequency being used to indicate a switching frequency of the image recognition scene.

[0077] Optionally, when the image data acquisition by the first image sensor is stopped, that is, the first image sensor does not need to output the image data stream in a subsequent period of time, the first image sensor can be subjected to a current-off process or a power-off process to reduce the power consumption of the first image sensor.

[0078] When the switching frequency of the image recognition scene is low, if the first image sensor is always kept in the powered-on state, that is, only the current-off process is performed on the first image sensor, the terminal power consumption will obviously be increased. Therefore, in a possible implementation, a frequency threshold is set, and when the terminal determines that the scene switching frequency is less than the frequency threshold, the first image sensor can be directly subjected to a power-off process to reduce the terminal power consumption.

[0079] Illustratively, the frequency threshold can be set by the developer, for example, the frequency threshold can be 2s -1 .

[0080] In response to the scene switching frequency being higher than the frequency threshold, performing a current-off process on the first image sensor.

[0081] When the image recognition scene is frequently switched, if the first image sensor is directly subjected to a power-off process, when the image recognition scene changes, the first image sensor needs to be re-powered, which increases the working mode configuration time to some extent. Therefore, in order to improve the timeliness of the working mode switching, in a possible implementation, when the terminal determines that the scene switching frequency is higher than the frequency threshold, only the current-off process is performed on the first image sensor, thereby saving the time for re-powering and powering off, and further improving the accuracy of subsequent image content recognition.

[0082] In this embodiment, the first image sensor in the working mode is used for scene recognition to obtain the current image recognition scene, and the second image sensor is triggered by the first image sensor to perform the configuration of the second working mode, so that the mode switching instruction does not need to be issued by the upper layer, thereby improving the timeliness of the mode adaptation. In addition, when the mode switching of the second image sensor is completed, whether the first image sensor is subjected to a power-off process can be selected based on the high or low of the scene switching frequency. If the scene switching frequency is low, the first image sensor is directly subjected to a power-off process, which can reduce the terminal power consumption. If the scene frequency is high, only the current-off process is performed on the first image sensor, which can avoid the need for re-powering and powering off in the subsequent mode configuration, thereby affecting the timeliness of the mode switching.

[0083] In a possible application scenario, image content processing on the image data output by the image sensor is also needed, the image content processing is more detailed than image detection of the image sensor, and the matching software algorithm also differs in different image recognition scenarios, therefore, in a possible implementation, when it is determined that the current image recognition scenario does not match the current working mode, the backend software algorithm also needs to be updated synchronously based on the current image recognition scenario.

[0084] Reference is made to Figure 4 which shows a flowchart of an image data acquisition method according to another example embodiment of the present application. This embodiment is exemplarily described taking that the method is applied to a terminal as an example, and the method comprises:

[0085] In step 401, image data is acquired by a first image sensor, the first image sensor operates in a first working mode, and a second image sensor is in a non-working mode.

[0086] The implementation of step 401 can refer to the foregoing embodiments, and will not be described here in detail.

[0087] In step 402, the acquired image data is processed by a first image processing algorithm.

[0088] Optionally, the requirement for the backend software recognition algorithm also differs in different image recognition scenarios, that is, different image processing algorithms are needed in different image recognition scenarios.

[0089] Illustratively, the image processing algorithm can be an image recognition algorithm, an image edge detection algorithm, an image segmentation algorithm, an image enhancement algorithm, and the like, and the specific algorithm of the image processing algorithm is not limited in the embodiments of the present application.

[0090] In a possible implementation, when the terminal starts the first working mode of the first image sensor by default, the backend image processing algorithm is also configured correspondingly, so that the image processing algorithm matches the first working mode of the first image sensor, and image content processing on the image recognition scenario corresponding to the first working mode is realized; that is, when the first image sensor acquires image data in the first working mode, the backend first image processing algorithm is used to process the image data output by the first image sensor.

[0091] Optionally, different image processing algorithms are provided in the terminal, and in order to improve the accuracy of different image recognition scenarios, different image recognition scenarios are provided with different image processing algorithms. Illustratively, if the image recognition scenario is a face recognition scenario, the corresponding image processing algorithm can be a face recognition algorithm; if the image recognition scenario is a motion trend recognition scenario, the corresponding image processing algorithm can be a target tracking algorithm, and the embodiments of the present application do not constitute a limitation on the image processing algorithm.

[0092] In step 403, in response to determining that the current image recognition scenario does not match the first working mode, the second image sensor is configured to the second working mode based on the current image recognition scenario, wherein the first working mode and the second working mode correspond to different image recognition scenarios respectively, and the second working mode matches the current image recognition scenario.

[0093] In a possible implementation, the current image recognition scenario can be identified by the first image sensor, and then when it is determined that the current image recognition scenario does not match the first working mode, the first image sensor triggers the configuration of the second image sensor to the second working mode.

[0094] Optionally, the scene recognition of the image data output by the first image sensor can also be performed by the first image processing algorithm of the software end (backend), and then whether the working mode switching is needed is determined based on the scene recognition result.

[0095] Illustratively, the process of triggering the mode switching by the first image processing algorithm (software end) can include the following steps:

[0096] I. Based on the image content processing result of the first image processing algorithm, the current image recognition scenario is determined.

[0097] In a possible implementation, when the first image processing algorithm performs the image content processing on the image data output by the first image sensor, the content contained in the image frame corresponding to the image data can also be identified, the image content processing result is obtained, and then the current image recognition scenario is determined based on the image content processing result.

[0098] Illustratively, if the first image processing algorithm identifies that the image frame corresponding to the image data contains a face, and the face position in the adjacent image frame does not change significantly, that is, there is no motion trend, it is determined that the current image recognition scenario is a target object recognition scenario (face recognition scenario); if the first image processing algorithm identifies that the image frame corresponding to the image data does not contain a face, and detects that there is a gesture motion trend in the adjacent image frame, it is determined that the current image recognition scenario is a motion trend recognition scenario.

[0099] II. In response to determining that the current image recognition scene does not match the first working mode, issuing a second configuration instruction to the second image sensor, the second configuration instruction containing the current image recognition scene.

[0100] Based on the relationship between the image processing algorithm, the image recognition scene and the working mode, if the terminal determines that the current image recognition scene does not match the first working mode, a configuration instruction can be issued from the upper layer to the lower layer, that is, a configuration instruction is issued from the software layer to the image sensor to control the second image sensor to perform the configuration process of the second working mode.

[0101] Illustratively, the second configuration instruction can also contain the current image recognition scene.

[0102] III. In response to the second image sensor receiving the second configuration instruction, configuring the second image sensor into the second working mode.

[0103] After the second image sensor receives the second configuration instruction, the configuration process of the second working mode can be performed according to the current image recognition scene indicated by the second configuration instruction.

[0104] Step 404, in response to determining that the current image recognition scene does not match the first working mode, switching the first image processing algorithm to a second image processing algorithm, and performing image content processing on the collected image data through the second image processing algorithm.

[0105] In a possible implementation, when it is determined that the current image recognition scene does not match the first working mode, it is determined that the first image processing algorithm of the backend also does not match the current image recognition scene, and in order to ensure the accuracy of image processing, the first image processing algorithm needs to be switched to a second image processing algorithm that matches the current image recognition scene, so that the collected image data can be processed through the second image processing algorithm.

[0106] Since the second image sensor is still outputting image data collected in the first working mode by the first image sensor during the configuration into the second working mode, the image data does not match the current image recognition scene, and in order to enable subsequent recognition of features required by the current image recognition scene, in a possible implementation, when it is determined that the current image recognition scene does not match the first working mode, the first image processing algorithm is first switched to a second image processing algorithm that matches the current image recognition scene, and then the image data output by the first image sensor is processed by the second image processing algorithm for image content recognition, so that the backend image content recognition module can still correctly recognize the image content required by the current scene.

[0107] Step 405, in response to the completion of the configuration of the working mode of the second image sensor, collecting image data by the second image sensor.

[0108] The implementation of the present step can refer to the above embodiments, which will not be repeated here.

[0109] In one possible implementation, when the second image sensor can output image data, the image content of the image data output by the second image sensor is recognized by the second image processing algorithm.

[0110] In the present embodiment, by updating the image processing algorithm for image content processing in the backend during the configuration of the second working mode of the second image sensor, the image features required in the current image recognition scenario are recognized by the image processing algorithm in the case of mismatching of the working modes, further ensuring the accuracy of image processing during the configuration of the working modes. In addition, by updating the backend image processing algorithm, the backend image processing algorithm is consistent with the image content recognition scenario, ensuring the accuracy of subsequent image processing.

[0111] In order to avoid the sudden switching of the image data stream and the jump or inaccurate result of the recognition effect, after the completion of the configuration of the working mode of the second image sensor, the image data can be output by the first image sensor and the second image sensor at the same time for a period of time.

[0112] In one exemplary example, as shown in Figure 5 FIG. 2 shows a flowchart of an image data collection method according to another exemplary embodiment of the present application. The present embodiment takes the terminal as an example to illustrate the method, which comprises the following steps:

[0113] Step 501, collecting image data by the first image sensor, the first image sensor operates in a first working mode, and the second image sensor is in a non-working mode.

[0114] Step 502, in response to determining that the current image recognition scenario does not match the first working mode, configuring the second image sensor to a second working mode based on the current image recognition scenario, wherein the first working mode and the second working mode correspond to different image recognition scenarios respectively, and the second working mode matches the current image recognition scenario.

[0115] Step 503, in response to the completion of the configuration of the working mode of the second image sensor, collecting image data by the second image sensor.

[0116] The implementation of steps 501-503 can refer to the above embodiments, which will not be repeated here.

[0117] Step 504, acquiring a first image data stream output by the first image sensor and a second image data stream output by the second image sensor.

[0118] In a possible implementation, when the working mode configuration of the second image sensor is completed, the first image sensor is not temporarily subjected to the current-off processing, and the first image sensor and the second image sensor simultaneously output the image data streams, that is, the terminal can simultaneously acquire the first image data stream output by the first image sensor and the second image data stream output by the second image sensor.

[0119] Step 505, reading a first proportion of the first image data from the first image data stream and a second proportion of the second image data from the second image data stream, the first proportion decreasing over time and the second proportion increasing over time.

[0120] The sum of the first proportion and the second proportion is 1 or 100%. Illustratively, at the beginning stage, the first proportion can be 50% and the second proportion can be 50%, the first proportion decreases over time and the second proportion increases over time, and after a preset time period, the first proportion can be 20% and the second proportion can be 80%.

[0121] In a possible implementation, after the terminal acquires the two image data streams, the terminal can cross-read the two image data streams, that is, read a first proportion of the first image data from the first image data stream and a second proportion of the second image data from the second image data stream, and the image frame rate remains unchanged, so as to gradually reduce the proportion of reading the first image data stream and gradually increase the proportion of reading the second image data stream, so that the image data can gradually transition to the image data collected in the second working mode, thereby avoiding the situation that the sudden change causes the identification effect to jump or the result to be inaccurate.

[0122] Step 506, performing image content processing based on the first image data and the second image data.

[0123] In a possible implementation, the first image data and the second image data can be subjected to image content processing based on a second image processing algorithm; when the second proportion reaches 100%, that is, the image data collected in the second working mode is completely transitioned, the second image data is subjected to image content processing based on the second image processing algorithm, and the collection of the image data by the first image sensor is stopped.

[0124] When the collection of the image data by the first image sensor is stopped, the first image sensor can be subjected to the power-off processing or the current-off processing.

[0125] In this embodiment, after the second image sensor is configured, the first image sensor can be temporarily powered off or current flow is not processed, and the first image sensor and the second image sensor output image data streams at the same time. Through cross reading and updating of image data reading ratio, the image data gradually transitions to the corresponding image data in the second working mode, thereby avoiding the sudden change of image data and the jump change of image content recognition effect or the inaccurate result.

[0126] In one exemplary example, as shown in FIG. 1, a flowchart of an image data acquisition method according to an exemplary embodiment of the present application is shown, which includes the following steps: Figure 6

[0127] Step 601, the image sensor module is started.

[0128] When the terminal is powered on, the image sensor is initialized and configured.

[0129] Step 602, the default image sensor working mode is set.

[0130] The image sensor is started according to the default working mode configuration, and the image sensor works according to the configuration parameters and outputs image data streams.

[0131] Step 603, the image sensor continuously performs scene detection.

[0132] The output image data is detected or recognized by the current image sensor to obtain the current image recognition scene.

[0133] Step 604, whether the image recognition scene matches the working mode of the current image sensor is detected.

[0134] If the current image recognition scene does not match the working mode of the current image sensor, step 605 is entered, the standby image sensor is started, and the working mode of the standby image sensor is configured to match the working mode of the current image recognition scene.

[0135] If the current image recognition scene matches the working mode of the current image sensor, step 610 is entered, and the image data is continuously output by the current image sensor.

[0136] Step 605, the standby image sensor is started, and the standby image sensor is configured to match the current image recognition scene.

[0137] Step 606, the back-end image recognition algorithm is updated.

[0138] When it is determined that the current image recognition scene does not match the working mode of the current image sensor, the back-end image recognition algorithm needs to be updated so that the back-end can obtain image features that match the current image recognition scene.​

[0139] Step 607, the backup image sensor outputs image data.

[0140] Step 608, the original image sensor is powered off or current is cut off.

[0141] When the backup image sensor can output image data, in order to reduce the power consumption of the terminal, the original image sensor can be powered off or current can be cut off.

[0142] Step 609, the image data output by the backup image sensor is taken as the input of the image recognition algorithm.

[0143] Step 610, the image data output by the current image sensor is continued.

[0144] Step 611, the image content of the image data is recognized by the image recognition algorithm.

[0145] Step 612, the image sensor module continues to work.

[0146] As shown in the image data acquisition method shown in Figure 6 As shown in the image data acquisition method shown in Figure 7 As shown in the image data acquisition method shown in

[0147] Please refer to Figure 8 , which shows the structure block diagram of the image data acquisition device provided by an exemplary embodiment of the present application. The device can be realized by software, hardware or a combination of both to become all or part of the terminal. The device comprises:

[0148] The first acquisition module 801 is used for acquiring image data by the first image sensor, the first image sensor runs in the first working mode, and the second image sensor is in the non-working mode.

[0149] The configuration module 802 is configured to, in response to determining that the current image recognition scene does not match the first working mode, configure the second image sensor into a second working mode based on the current image recognition scene, wherein the first working mode and the second working mode correspond to different image recognition scenes respectively, and the second working mode matches the current image recognition scene.

[0150] The second acquisition module 803 is configured to acquire image data through the second image sensor in response to completion of configuration of the working mode of the second image sensor.

[0151] Optionally, the configuration module 802 comprises:

[0152] The determination unit is configured to perform scene recognition on the acquired image data through the first image sensor, to determine that the current image recognition scene does not match the first working mode.

[0153] Optionally, the configuration module 802 further comprises:

[0154] The first configuration unit is configured to, in response to determining that the current image recognition scene does not match the first working mode, trigger configuration of the second image sensor into the second working mode through the first image sensor.

[0155] Optionally, the first configuration unit is further configured to:

[0156] in response to determining that the current image recognition scene does not match the first working mode, send a first configuration instruction to the second image sensor through the first image sensor, the first configuration instruction containing the current image recognition scene;

[0157] in response to the second image sensor receiving the first configuration instruction, configure the second image sensor into the second working mode.

[0158] Optionally, the apparatus further comprises:

[0159] The first image processing module is configured to perform image content processing on the acquired image data through a first image processing algorithm.

[0160] The algorithm switching module is configured to, in response to determining that the current image recognition scene does not match the first working mode, switch the first image processing algorithm to a second image processing algorithm, and perform image content processing on the acquired image data through the second image processing algorithm.

[0161] Optionally, the apparatus further comprises:

[0162] determining, based on the image content processing result of the first image processing algorithm, a current image recognition scenario;

[0163] The configuration module 802 includes:

[0164] The instruction issuing unit is configured to, in response to determining that the current image recognition scenario does not match the first working mode, issue a second configuration instruction to the second image sensor, the second configuration instruction containing the current image recognition scenario.

[0165] The second configuration unit is configured to, in response to the second image sensor receiving the second configuration instruction, configure the second image sensor to the second working mode.

[0166] Optionally, the apparatus further includes:

[0167] The stopping acquisition module is configured to stop acquiring image data by the first image sensor.

[0168] Optionally, the apparatus further includes:

[0169] The acquisition module is configured to acquire a first image data stream output by the first image sensor and a second image data stream output by the second image sensor.

[0170] The reading module is configured to read a first proportion of first image data from the first image data stream and a second proportion of second image data from the second image data stream, the first proportion decreasing over time and the second proportion increasing over time.

[0171] The second image processing module is configured to perform image content processing based on the first image data and the second image data.

[0172] Optionally, the apparatus further includes:

[0173] The power-down module is configured to, in response to a scene switching frequency being less than a frequency threshold, perform power-down processing on the first image sensor, the scene switching frequency being used to indicate a switching frequency of an image recognition scenario.

[0174] The current-off module is configured to, in response to the scene switching frequency being higher than the frequency threshold, perform current-off processing on the first image sensor.

[0175] Optionally, the first working mode corresponds to a motion trend recognition scenario.

[0176] The second working mode corresponds to a target object recognition scenario.

[0177] In the embodiments of the present application, at least two image sensors are arranged in the terminal, such as a first image sensor and a second image sensor, and only one image sensor is kept in the working mode at the same time. When the terminal determines that the current image recognition scene does not match the working mode of the first image sensor, the second image sensor can be configured to the second working mode matching the current image recognition scene while keeping the first image sensor continuously outputting image data, and then the image data output by the second image sensor is used for image content recognition. Therefore, during the mode switching process, no image data is output for a period of time due to parameter configuration and flow start and stop operations, so that the phenomenon of image data flow interruption caused by mode switching is avoided, and the continuity and timeliness of subsequent image recognition are ensured. At the same time, the image sensor can be in the working mode matching the current image recognition scene, and the accuracy of subsequent image content recognition is further improved.

[0178] Please refer to Figure 9 which shows a structural block diagram of a terminal 900 provided by an exemplary embodiment of the present application. The terminal 900 can be a smart phone, a tablet computer, an electronic book, a portable personal computer, or other electronic devices provided with an image sensor. The terminal 900 in the present application can include one or more of the following components: a memory 901, an AP 902, a coprocessor 903, a first image sensor 904, and a second image sensor 905.

[0179] The memory 901 can include a random access memory (RAM) and a read-only memory (ROM). Optionally, the memory 901 includes a non-transitory computer-readable storage medium. The memory 901 can be used to store instructions, programs, codes, code sets, or instruction sets. The memory 901 can include a program storage area and a data storage area, wherein the program storage area can store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the above-mentioned various method embodiments, etc., and the operating system can be an Android system (including a system developed based on the Android system), an IOS system developed by Apple Inc. (including a system developed based on the IOS system), or other systems. The data storage area can also store data created by the terminal 900 in use (such as a phonebook, audio and video data, chat record data, etc.). Optionally, in the embodiments of the present application, the memory 901 can be used to store image data generated in the image acquisition process.

[0180] The AP 902 is configured with image processing algorithms for image content processing of image data acquired by the terminal. In an embodiment of the present application, the AP 902 can receive image data sent from the coprocessor 903 and perform image content processing on the image data. Optionally, the AP 902 runs a plurality of image processing algorithms, and different image processing algorithms correspond to different image recognition scenarios.

[0181] The coprocessor 903 is configured with an ISP for image preprocessing of image data output by the image sensor, and the preprocessed image data is fed back to the AP 902 for image content recognition. Optionally, the coprocessor 903 is also used to control the first image sensor 904 and the second image sensor 905 to configure working modes, switch flows, and power on and off.

[0182] The first image sensor 904 and the second image sensor 905 are sensors with image data acquisition functions. The first image sensor 904 and the second image sensor 905 have at least two working modes, and different working modes correspond to different image recognition scenarios.

[0183] In addition, those skilled in the art can understand that the structure of the terminal 900 shown in the above-mentioned drawings does not constitute a limitation on the terminal 900. The terminal can include more or fewer components than shown, or combine certain components, or different component arrangements. For example, the terminal 900 also includes radio frequency circuitry, a camera assembly, a sensor, audio circuitry, a wireless fidelity (WiFi) component, a power supply, a Bluetooth component, and the like, which are not described here.

[0184] An embodiment of the present application also provides a computer-readable storage medium, which stores at least one instruction. The at least one instruction is used to be executed by a processor to implement the image data acquisition method described in the above-mentioned various embodiments.

[0185] According to another aspect of the present application, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a terminal reads the computer instructions from the computer-readable storage medium. The processor executes the computer instructions, so that the terminal performs the image data acquisition method provided in the above-mentioned optional implementation manners.

[0186] Those skilled in the art should be aware that, in the above one or more examples, the functions described in the embodiments of the present application can be implemented in hardware, software, firmware or any combination thereof. When implemented in software, the functions can be stored in a computer readable medium or transmitted as one or more instructions or code on a computer readable medium. The computer readable medium includes computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. The storage media can be any available media that can be accessed by a general purpose or special purpose computer.

[0187] The above merely provides the optional embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. An image data acquisition method, characterized in that, The method is applied to a terminal, which is equipped with a first image sensor and a second image sensor. The first image sensor and the second image sensor have the same operating mode, and each has at least two operating modes. Different operating modes correspond to different image recognition scenarios, and the configuration parameters of the image sensors are different in different operating modes. The method includes: Image data is acquired through the first image sensor, which operates in a first working mode, while the second image sensor is in a non-working mode. In response to determining that the current image recognition scene does not match the first working mode, the second image sensor is configured to a second working mode based on the current image recognition scene, wherein the first working mode and the second working mode correspond to different image recognition scenes, and the second working mode matches the current image recognition scene; In response to the completion of the operating mode configuration of the second image sensor, image data is acquired through the second image sensor.

2. The method according to claim 1, characterized in that, The step of determining that the current image recognition scene does not match the first working mode includes: The first image sensor performs scene recognition on the acquired image data to determine that the current image recognition scene does not match the first working mode.

3. The method according to claim 2, characterized in that, In response to determining that the current image recognition scene does not match the first operating mode, configuring the second image sensor to the second operating mode based on the current image recognition scene includes: In response to determining that the current image recognition scene does not match the first working mode, the second image sensor is configured to the second working mode by triggering the first image sensor.

4. The method according to claim 3, characterized in that, The step of responding to determining that the current image recognition scene does not match the first working mode, and triggering the configuration of the second image sensor to the second working mode via the first image sensor, includes: In response to determining that the current image recognition scene does not match the first working mode, a first configuration instruction is sent from the first image sensor to the second image sensor, the first configuration instruction containing the current image recognition scene; In response to the second image sensor receiving the first configuration instruction, the second image sensor is configured to the second operating mode.

5. The method according to any one of claims 1 to 4, characterized in that, The method further includes: The acquired image data is processed using the first image processing algorithm; In response to determining that the current image recognition scene does not match the first working mode, the first image processing algorithm is switched to the second image processing algorithm, and the image content is processed by the second image processing algorithm on the acquired image data.

6. The method according to claim 5, characterized in that, The method further includes: Based on the image content processing results of the first image processing algorithm, the current image recognition scene is determined; The step of responding to determining that the current image recognition scene does not match the first working mode, and configuring the second image sensor to a second working mode based on the current image recognition scene, includes: In response to determining that the current image recognition scene does not match the first working mode, a second configuration instruction is sent to the second image sensor, the second configuration instruction containing the current image recognition scene; In response to the second image sensor receiving the second configuration command, the second image sensor is configured to the second operating mode.

7. The method according to any one of claims 1 to 4, characterized in that, The method further includes, after the image data is acquired via the second image sensor in response to the completion of the operating mode configuration of the second image sensor: Stop acquiring image data through the first image sensor.

8. The method according to any one of claims 1 to 4, characterized in that, The method further includes, after the image data is acquired via the second image sensor in response to the completion of the operating mode configuration of the second image sensor: Acquire the first image data stream output by the first image sensor and the second image data stream output by the second image sensor; Read first image data at a first ratio from the first image data stream, and read second image data at a second ratio from the second image data stream, wherein the first ratio decreases over time and the second ratio increases over time; Image content processing is performed based on the first image data and the second image data.

9. The method according to claim 7, characterized in that, The method further includes, after ceasing the acquisition of image data through the first image sensor: In response to the scene switching frequency being less than a frequency threshold, the first image sensor is powered down, wherein the scene switching frequency is used to indicate the switching frequency of the image recognition scene; In response to a scene switching frequency exceeding the frequency threshold, the first image sensor is subjected to flow-off processing.

10. The method according to any one of claims 1 to 4, characterized in that, The first working mode corresponds to the motion trend recognition scenario; The second working mode corresponds to the target object recognition scenario.

11. An image data acquisition device, characterized in that, The device is applied to a terminal, which is equipped with a first image sensor and a second image sensor. The first image sensor and the second image sensor have the same operating mode and each has at least two operating modes. Different operating modes correspond to different image recognition scenarios, and the configuration parameters of the image sensors are different in different operating modes. The device includes: The first acquisition module is used to acquire image data through the first image sensor, wherein the first image sensor operates in a first working mode and the second image sensor is in a non-working mode. A configuration module is configured to, in response to determining that the current image recognition scene does not match the first working mode, configure the second image sensor to a second working mode based on the current image recognition scene, wherein the first working mode and the second working mode correspond to different image recognition scenes, and the second working mode matches the current image recognition scene; The second acquisition module is used to acquire image data through the second image sensor in response to the completion of the working mode configuration of the second image sensor.

12. A terminal, characterized in that, The terminal includes a processor and a memory, the memory storing at least one program, which is loaded and executed by the processor to implement the image data acquisition method as described in any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that, The storage medium stores at least one instruction, which is executed by a processor to implement the image data acquisition method as described in any one of claims 1 to 10.

Images