Data processing method and device

Through the data processing method and device, the target application requests data from the acquisition device and obtains different data streams, solving the problem of single use of the acquisition device function, and achieving diversification of functions and improving user experience.

CN115633118BActive Publication Date: 2024-11-26LENOVO (BEIJING) LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202211219777.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-09-30
Publication Date
2024-11-26
Estimated Expiration
2042-09-30

AI Technical Summary

Technical Problem

In the prior art, third-party applications or local applications have relatively single use of the functions of the acquisition device, and cannot effectively utilize the additional functions of the acquisition device, which limits the functional diversity and user experience of the electronic device.

Method used

Provide a data processing method and device, through the target application request of the data of the designated acquisition device, obtain different data streams formed from the acquisition device, and process them in response, so as to realize the use of additional functions of the acquisition device, such as camera function. switch and beauty treatment.

Benefits of technology

It realizes the diversified use of the functions of the acquisition device by third-party applications or local applications, improves the user experience and the functional richness of electronic devices, provides different visual effects and more flexible functional operations.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115633118B_ABST
    Figure CN115633118B_ABST
Patent Text Reader

Abstract

The present application discloses a data processing method and a data processing device, wherein the method comprises: obtaining a data request from a target application to a designated acquisition device; obtaining a target data stream based on the data request; using the target data stream as a response to the data request so that the target application processes the target data stream, wherein the target data stream is different from a data stream formed by acquisition by the designated acquisition device.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of data processing, and in particular to a data processing method and a data processing device. Background Art

[0002] The acquisition device in the electronic device, such as the image acquisition device (camera) and the audio acquisition device (microphone), as a device in the electronic device that can acquire images and audio of the external environment, can be called by the inherent application of the electronic device, i.e., the local application, and can also be called by the third-party application in the electronic device, so as to better realize the inherent functions of the third-party application or the local application based on such calling. At present, the use of the functions of the acquisition device by the third-party application or the local application is relatively single. Summary of the invention

[0003] The present application provides a data processing method and device to at least solve the above technical problems existing in the prior art.

[0004] According to a first aspect of the present application, there is provided a data processing method, comprising:

[0005] Obtaining a data request from a target application to a specified acquisition device;

[0006] Based on the data request, obtaining a target data stream;

[0007] The target data stream is used as a response to the data request so that the target application processes the target data stream, wherein the target data stream is different from the data stream formed by the acquisition of the designated acquisition device.

[0008] According to a second aspect of the present application, there is provided a data processing device, comprising:

[0009] A first obtaining unit is used to obtain a data request from a target application to a designated collection device;

[0010] A second obtaining unit, configured to obtain a target data stream based on the data request;

[0011] The third obtaining unit is used to use the target data stream as a response to the data request so that the target application processes the target data stream, wherein the target data stream is different from the data stream formed by the designated acquisition device.

[0012] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] By reading the detailed description below with reference to the accompanying drawings, the above and other purposes, features and advantages of the exemplary embodiments of the present application will become readily understood. In the accompanying drawings, several embodiments of the present application are shown in an exemplary and non-limiting manner, wherein:

[0014] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts.

[0015] Figure 1 The following is a schematic diagram showing the implementation process of the data processing method of the present application embodiment. Figure 1 ;

[0016] Figure 2 The following is a schematic diagram showing the implementation process of the data processing method of the present application embodiment. Figure 2 ;

[0017] Figure 3 The following is a schematic diagram showing the implementation process of the data processing method of the present application embodiment. Figure 3 ;

[0018] Figure 4 The following is a schematic diagram showing the implementation process of the data processing method of the present application embodiment. Figure 4 ;

[0019] Figure 5 The following is a schematic diagram showing the implementation process of the data processing method of the present application embodiment. Figure 5 ;

[0020] Figure 6 A display effect diagram of screen sharing in an embodiment of the present application is shown;

[0021] Figure 7 The following is a schematic diagram showing the implementation process of the data processing method of the present application embodiment. Figure 6 ;

[0022] Figure 8 A schematic diagram showing the framework structure of an electronic device for implementing a data processing method according to an embodiment of the present application is shown;

[0023] Fig. 9 A schematic diagram showing the structure of a data processing device according to an embodiment of the present application is shown;

[0024] Fig.10 A schematic diagram of the structure of an electronic device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0025] In order to make the purpose, features, and advantages of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0026] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0027] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it can be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0028] In the following description, the terms "first\second" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0029] It can be understood that in practical applications, taking the image acquisition device (camera) as an example, in addition to its own function of acquiring images, it also has additional functions, such as processing the acquired images such as beautification, filters, exposure parameter adjustment, etc. In the related art, the use of the functions of the image acquisition device (or audio acquisition device) by third-party applications or local applications is limited to: using the camera (or microphone) to acquire images (or audio) of the external environment of the electronic device. It can be seen that the use of the functions of the image acquisition device (or audio acquisition device) in the related art is relatively single.

[0030] Among them, the additional functions of the camera include not only those shown above, but also other reasonable functions mentioned in the following scheme, which are not listed one by one.

[0031] The processing logic of the data processing method of the embodiment of the present application can be deployed in any electronic device with a camera and / or a microphone. The electronic device can be a terminal or a server. Among them, the terminal includes but is not limited to a smart phone, a wearable device, a tablet computer, an all-in-one machine, a desktop computer, etc. The server includes a common server, a cloud server, a server for a special field, etc. Exemplarily, the processing logic of the data processing method is deployed in a smart phone or a wearable device.

[0032] The present application provides an embodiment of a data processing method, such as Figure 1 As shown, the method includes:

[0033] S101: Obtaining a data request from a target application to a designated collection device;

[0034] In this step, the target application (referred to as the target application) is any application in the electronic device that has the ability to call the image acquisition device and / or the ability to call the audio acquisition device. The target application can be a third-party application (program) of the electronic device or a local application. Exemplarily, the target application is a video playback or call application, or an audio playback or audio call application.

[0035] The acquisition device may be an image acquisition device, i.e., a camera, or an audio acquisition device, i.e., a microphone. The number of cameras and audio acquisition devices in an electronic device may be one, or two, or more than three. Taking a camera as an example, the camera in an electronic device may include, but is not limited to, two, three, or more of the following: a front camera, a rear camera, a wide-angle camera, a depth of field camera, a telephoto camera, a macro camera, or a monocular camera.

[0036] The designated acquisition device may be the default image / audio acquisition device of the target application among one or two or more image / audio acquisition devices of the electronic device, for example, the designated image acquisition device is the default front camera of the target application.

[0037] It can be understood that the target application can realize one function of itself without calling the designated acquisition device, and can also realize another function of itself when calling the designated acquisition device. For example, the video call software WX application can communicate with the other user through voice or text without calling the camera. When the camera is called, the communication with the other user is realized through video. That is, the target application can call the designated acquisition device or not. Based on this, it is possible to monitor whether the target application has a call to the designated acquisition device. If it is monitored that the target application has a call to the designated acquisition device, a data request of the target application to the designated acquisition device is generated, thereby obtaining the data request of the target application to the designated acquisition device.

[0038] Taking the image acquisition device as an example, the target application can start the designated acquisition device, and the designated acquisition device started can collect external image streams or audio streams. These collected information can be output on the electronic device side through the target application and transmitted by the electronic device to another electronic device to make the other electronic device output. It can be understood that the aforementioned functions of the designated acquisition device are its own functions. In the process of the target application using the designated acquisition device, if the user has the need to use the additional functions of the designated acquisition device, it can operate the function button displayed in the display interface of the target application. When the electronic device detects that there is a click, press, etc. operation on the function button, it can generate a data request from the target application to the designated acquisition device, thereby obtaining a data request from the target application to the designated acquisition device. That is, the data request can be generated when the target application needs to use the designated acquisition device's own functions and wants to use the additional functions of the designated acquisition device, and can also be generated when the target application starts the designated acquisition device (want to call the designated acquisition device).

[0039] S102: Obtaining a target data stream based on the data request;

[0040] The electronic device responds to the data request and obtains a data stream different from the image data stream (or audio data stream) formed by the image data (or audio data) collected by the designated collection device, and the different data stream is the target data stream. If the image data stream (or audio data stream) formed by the image data (or audio data) collected by the designated collection device is regarded as the first data stream, then the target data stream can be regarded as the second data stream.

[0041] S103: Using the target data stream as a response to the data request so that the target application processes the target data stream, wherein the target data stream is different from the data stream formed by the designated acquisition device.

[0042] In this step, the electronic device provides the target data stream to the target application as a response or request result of the target application's data request to the designated acquisition device. The target application can process the response or request result, such as outputting it. Alternatively, the second data stream is processed and outputted in the same manner as the first data stream.

[0043] Among them, the provision of the second data stream to the target application realizes the target application's unaware reception of the data stream, that is, the target application does not need to know or has no need to know whether the data stream provided to it is the second data stream or the first data stream, and can process the data stream provided to it in the same way as the first data stream. It can be understood that if the target application is a third-party application, the present application solution can realize the use of additional functions of the acquisition device in the third-party application without the third-party application being aware of it.

[0044] In S101 to S103, based on the data request generated by the target application to the designated collection device, a second data stream different from the first data stream collected by the designated collection device can be obtained, and the second data stream can be used as a response result or a request result in response to the data request, so that the target application processes the second data stream. Among them, the first data stream obtained by the designated collection device is the data stream collected when the designated collection device implements the existing functions of the collection device, and the second data stream is different from the first data stream. The processing of the second data stream by the target application realizes the use of the additional functions of the collection device by the target application. The use of the functions of the collection device by third-party applications or local applications of electronic devices is no longer single.

[0045] From a visual perspective, third-party or local applications, based on the use of additional functions of the acquisition device, bring visual effects that are different from the display effects of the first data stream, enhance the visual experience, and reflect the ease of use and rich functionality of the electronic device.

[0046] The acquisition device is an image acquisition device, and the electronic device includes Figure 8Taking the framework of the electronic device shown as including the application layer, the camera service layer and the camera HAL (hardware abstraction) layer as an example, the camera service in the camera service layer monitors or detects whether the target application in the application layer has a data request for the specified acquisition device. If so, the camera service layer hands the data request to the camera HAL layer for processing, and the camera HAL layer responds to the data request to obtain the target data stream. The camera HAL layer provides the target data to the target application of the application layer via the camera service layer, so that the target application regards the target data stream as the first data stream for processing. Among them, the processing of the target data stream by the target application can refer to the description of the processing of the first data stream by the target application, which is not repeated here.

[0047] Considering that the electronic device in this application has two or more cameras, and different cameras can collect the external environment from different angles, in order to realize the use of different cameras in the target application. Figure 2 As shown, in some embodiments, S102 can be further as follows: S1020: based on the data request, determine the target acquisition device; S1022: obtain the data stream formed by the target acquisition device as the target data stream; wherein the target acquisition device is different from the designated acquisition device. In this embodiment, the target application can call different acquisition devices, thereby changing the target application's original output of the first data stream to the output of the second data stream. The diversified use of the functions of the acquisition device is realized in the target application.

[0048] Exemplarily, the designated acquisition device of the target application is the rear camera, and the rear camera is switched to a wide-angle camera. The target application displays the video image captured by the rear camera. The cameraservice in the camera service layer monitors or detects that the target application in the application layer has a data request to switch the rear camera to a wide-angle camera. The camera service hands the switching request of the camera to the camera HAL layer for processing. The camera HAL layer closes the permission of the target application to use the rear camera, and opens the permission to use the wide-angle camera. The external environment that was originally captured by the rear camera is now captured by the wide-angle camera. It can be understood that the wide-angle camera has a wider capture angle than the rear camera. Using a wide-angle camera to capture the same external environment will inevitably increase the capture surface of the external environment. The video stream captured by the wide-angle camera is fed back to the target application in the application layer via the camera service layer, and the target application displays the captured wide-angle video stream. In this way, users can watch the wide-angle image of the same external environment, improving the user experience.

[0049] If the target application is a video call software, in the process of using the video call software to realize a video call between the user of the party and the user of the other party, the above exemplary solution can be used to change the small wide-angle video stream of the external environment where the electronic device of the party is located, which is originally output, into a large wide-angle video stream of the external environment. Thus, in the target application of the electronic device, the user can view the same external environment from different angles.

[0050] The above exemplary scheme takes switching to one camera as an example, and it is also possible to switch to two cameras at the same time, such as switching from the rear camera to the front camera and the wide-angle camera at the same time. Among them, the opening permission of the front camera is opened by the camera HAL layer. The front camera will collect the corresponding video stream in the same external environment from its own collection angle. The camera HAL layer will splice the videos collected by the front camera and the wide-angle camera from different collection angles and at the same collection time, thereby obtaining a second data stream. The second data stream is returned to the target application of the application layer via the camera service layer. Among them, the purpose of splicing is to synchronously display the images collected by different cameras at the same collection time in the target application. The target application can realize the synchronous display of different images at the same collection time in a sub-area manner of the display screen. For example, through the target application, the picture collected by the front camera at the collection time 1 is displayed in the upper half of the display screen, and the picture collected by the wide-angle camera at the collection time 1 is displayed in the lower half of the display screen. Therefore, the display of the target application can realize the viewing of the same external environment from different angles.

[0051] In addition, the default cameras of the target application can also be two or more, and the target camera (image acquisition device) determined based on the data request can be one or more cameras other than the default camera. For example, the default cameras are the front camera and the rear camera, and at least one of the wide-angle camera, telephoto camera, depth of field camera and monocular camera can be used as the camera after switching. This realizes the free switching between cameras in the target application.

[0052] To achieve the target application, different cameras are used together, such as Figure 3 As shown, in some embodiments, S102 may be further as follows: S1021: based on the data request, determine multiple target acquisition devices; S1023: obtain the data stream formed by each of the multiple target acquisition devices as the target data stream; wherein the multiple target acquisition devices at least include the designated acquisition device. In this embodiment, the target application can call other cameras at the same time while maintaining the call to the default acquisition device, thereby changing the output of the first data stream originally collected only by the designated acquisition device to the output of the second data stream obtained by the data collected by the designated acquisition device and other different acquisition devices. The use of the functions of the acquisition device in the target application is diversified.

[0053] Exemplarily, the designated acquisition device of the target application is the front camera, and the rear camera is switched to the front camera and the depth of field camera. The target application displays the face image captured by the front camera. The camera service in the camera service layer monitors or detects that the target application in the application layer has a data request to switch the front camera to the front camera and the depth of field camera, that is, to add a call to the depth of field camera on the basis of the default front camera. The camera service passes the camera switching request to the camera HAL layer for processing. The camera HAL layer enables the target application to use the depth of field camera, maintains the use of the front camera, and controls the front camera and the depth of field camera to collect the external environment of the electronic device at their respective collection angles. The camera HAL layer synthesizes the images collected by the front camera and the depth of field camera from different collection angles and at the same collection time to obtain a second data stream. The second data stream is returned to the target application in the application layer via the camera service layer.

[0054] It can be understood that compared with the face image captured by the front camera, the face can be more prominently displayed in the synthesized image, and the face background is weakened. If the camera HAL layer uses certain processing methods such as blurring after synthesis, then from a visual perspective, the face background is blurred and the face is highlighted in the synthesized image. Compared with the display effect of the original default camera's video stream, the target application's additional calls to other cameras can greatly highlight the face part that the user expects to be displayed and weaken the background part that is not expected to be displayed.

[0055] Also exemplarily, the designated acquisition device of the target application is a rear camera, and the rear camera is switched to a front camera and a rear camera. The target application displays the video screen captured by the rear camera. The camera service in the camera service layer monitors or detects that the target application in the application layer has a data request to switch the rear camera to the front camera and the rear camera, that is, to add a call to the front camera on the basis of the default rear camera. The camera service hands the camera switching request to the camera HAL layer for processing. The camera HAL layer enables the target application to use the front camera, maintains the use of the rear camera, and controls the front camera and the rear camera to collect the external environment of the electronic device at their respective collection angles. The camera HAL layer will splice the videos collected by the front camera and the rear camera from different collection angles and at the same collection time, thereby obtaining a second data stream. The second data stream is returned to the target application in the application layer via the camera service layer. Through the target application, the pictures collected by the front camera and the rear camera at the same collection time are displayed separately in two different areas of the display screen. Therefore, in the target application, by calling different cameras at the same time, it is possible to simultaneously watch images of the same external environment from different angles.

[0056] If the target application is video call software, in the process of using the video call software to realize a video call between the user of the own side and the user of the other side, using the aforementioned exemplary scheme, the video call software originally could only display the picture captured by the rear camera, but now the video call software can display the pictures captured by both the front camera and the rear camera at the same time.

[0057] It can be understood that when using video call software to realize a video call between the user of the party and the user of the other party, the display screen also needs to reserve a certain display area for the picture collected by the camera of the other party's electronic device. In this way, the display screen can be divided into N display areas, where N is a positive integer greater than or equal to 2. Among them, N-1 display areas are used to display the pictures collected by each of the N-1 cameras called by the target application; the Nth display area is used to display the picture collected by the other party's head portrait or the rear camera of the other party.

[0058] It is understandable that in addition to the inherent function of capturing images, the camera also has additional functions such as beauty, filters, and exposure parameter adjustment. The following solution can be used to implement the use of these functions in the target application. Figure 4 As shown, in some embodiments, the S102 includes: S4010: determining a target functional module based on a data request; S4012: processing each frame of data in a data stream formed by acquisition by the designated acquisition device based on the target functional module to form a target data stream.

[0059] The functional modules in the embodiments of the present application include but are not limited to: beauty function module, filter function module, cutout function module, prop function module, exposure parameter adjustment function module, anti-shake function module, white balance function module, etc. The target function module is one or more of the aforementioned function modules. Based on the target function module, the first data stream is processed to obtain a second data stream different from the first data stream, thereby realizing the use of more functions of the acquisition device in the target application and diversifying the use of functions.

[0060] In the embodiment of the present application, the function modules are located in the HAL layer, and the algorithms provided by the function modules realize their respective functions. For example, the beauty function is realized by the beauty algorithm provided by the beauty function module; the filter function is realized by the filter algorithm provided by the filter function module.

[0061] For example, taking the implementation of the camera's beauty function as an example, when the target application displays a video stream collected by a default camera such as a front camera without a beauty effect or when the target application calls the default camera, the camera service monitors or detects that the target application in the application layer has a user's need to enable the beauty function module, and believes that there is a data request generated by the target application for the default camera. The camera service passes the data request to the camera HAL layer for processing. The camera HAL layer calls the beauty function module, uses the beauty algorithm provided by the beauty function module, and performs beauty processing on each frame of the video stream collected by the default camera without beauty, to obtain a video stream with a beauty effect. The video stream with a beauty effect is returned to the target application in the application layer via the camera service layer. In the target application, the display of the original video stream without the beauty effect is changed to the display of the video stream with the beauty effect.

[0062] The above exemplary solution is equivalent to a solution for processing the video stream captured by the default camera to display the effect. This processing solution can occur when the unprocessed video stream is displayed in the target application, or when the target application starts calling the default camera. This processing solution can realize the use of more functions of the camera in the target application, bringing different visual experience to users.

[0063] If the target application is a video call software, when the user of the party uses the video call software to make a video call with the other party, the above exemplary solution can be used to allow the other party to see the video stream of the user of the party with different display effects, such as the face image of the user of the party without and with beauty. In this way, the camera's beauty, filter, cutout, exposure parameter adjustment and other functions can be realized in the target application.

[0064] like Figure 5 As shown, in some embodiments, the step of obtaining the target data stream based on the data request in S102 includes: S501: determining a target display area of ​​a display screen based on the data request; S503: obtaining screen data of the target display area to form a target data stream; the screen data is used to present display content on the target display area. Screen sharing is implemented in the target application.

[0065] Exemplarily, the display screen is a display device of an electronic device, and the target display area is the entire area or a partial area on the display device. Taking the target display area as the entire area on the display device as an example, the camera service monitors or detects that there is a user's demand to share the local screen with the other user in the target application in the application layer, and believes that there is a data request generated by the target application to the default camera. The camera service passes the data request to the camera HAL layer for processing. The camera HAL layer takes a real-time screenshot of the screen data on the local display screen, and uses the real-time screenshot screen data as the second data stream. The second data stream is transmitted from the camera HAL layer to the application layer via the camera service layer. The target application in the application layer transmits the captured screen data to the display screen of the other electronic device to realize the sharing of the local screen by using the target application.

[0066] It should be noted that if the target application is a video call software, in the process of using the video call software to realize the video call between the user of the party and the user of the other party, a part of the display area on the display screen of the electronic device of the party, such as the first display area, is used to display the facial image of the user of the party collected by the default camera of the party (taking the default camera as the front camera as an example), and another part of the area on the display screen, such as the second display area, is used to display the facial image of the user of the other party. In this video call scenario, if the user of the party wants to use the video call scenario to teach the other party how to use the mobile phone or remotely tutor the child's homework. This can be achieved through screen sharing. Specifically, if the camera service monitors or detects that there is a user's need to share the local screen with the other user in the target application in the application layer, it is considered that there is a data request generated by the target application for the default camera. The camera service submits the data request to the camera HAL layer for processing. In order to realize the teaching of the method of using the other party's mobile phone, the user of the party will perform the operation of the method of using the mobile phone in the mobile phone, and the operation process of the method of using will be displayed on the display screen, specifically in the first display area. The camera HAL layer takes a screenshot of the screen data displayed in the first display area on the display screen to obtain the screenshot data.

[0067] It can be understood that in the process of using video call software to realize a video call between the local user and the other user, the default camera captures the facial image of the local user, and the facial image captured by the default camera is used as a preview stream in the HAL layer, and is provided to the target application of the application layer via the camera service layer to enable the target application to display the facial image, and is transmitted to the other user through the transmission channel to enable the local user's facial image to be displayed on the other user's display screen.

[0068] In the embodiment of the present application, in order to realize screen sharing by using video call software, the HAL layer replaces the face image captured by the default camera with the screenshot data. If the screenshot data is regarded as a screen recording stream, the HAL layer replaces the preview stream with the screen recording stream and provides it to the target application of the application layer via the camera service layer. This allows the target application to display the screen recording stream and transmit it to the other user through the transmission channel to realize screen sharing, such as Figure 6 The display effect diagram shown.

[0069] In some embodiments, the HAL layer can process the screen recording stream according to the preview stream, and replace the preview stream with the processed screen recording stream. The size or size of each image in the screen recording stream can be cropped and the direction can be adjusted according to the size, dimension, and direction of the image in the preview stream. The above processing and replacement of the screen recording stream can be achieved through a plug-in in the HAL layer.

[0070] In actual applications, the screen sharing solution can be used to tutor children's homework using the target application during a video call, and to share the entertainment scenes (such as playing games or watching movies) of one user with the other user. Therefore, there is no need to install and enable screen sharing software, and the inherent video call function of video call software can be used to achieve screen sharing of one user.

[0071] Screen sharing is achieved by using video call software, and there is no need to install screen sharing software, which is easy to promote in engineering. In addition, for users, the operation is simple and it is easy to switch freely between video chat and screen sharing.

[0072] In the aforementioned Figure 2 to Figure 5 In the scheme shown, the following scheme may be used to implement determination of at least one of the target acquisition device, the target functional module and the target display area of ​​the display screen.

[0073] Based on the data request, an interactive interface is displayed in a window form; based on the interactive operation on the interactive interface, a target instruction is obtained; the target instruction is used to determine a target object, and the target object includes at least one of a target acquisition device, a target functional module and a target display area.

[0074] In the embodiment of the present application, taking the designated acquisition device as an image acquisition device as an example, an interactive interface is pre-set, and the interactive interface displays at least two of the three types of information, namely, description information of each camera of the electronic device, description information of each functional module, and description information of the display area. The description information can be embodied in text form or in icon form.

[0075] When the target application calls the designated image acquisition device, it is considered that there is a data request from the target application to the designated image acquisition device. In response to the data request, the interactive interface is called, and the interactive interface is displayed in the form of a window on the display interface of the target application. The user can perform a selection operation on the object to be selected in the interactive interface (which can be used as a type of interactive operation). When the operation is detected, a target instruction for selecting which object is generated, thereby realizing the determination of the selected object. For example, the default camera is the front camera, and based on the user's interactive operation on the interactive interface, the wide-angle camera is selected as the target acquisition device. Or, based on the user's interactive operation on the interactive interface, the beauty function module is selected as the target function module.

[0076] In the above scheme, if the interactive interface displayed in the form of a window is regarded as a pop-up window interface, then in the embodiment of the present application, the desired camera and functional module can be selected based on the interactive operation of the pop-up window interface. The pop-up window interface provides an intuitive selection scheme for the selection of the target object.

[0077] It should be noted that in the aforementioned Figure 2-Figure 4 Of the three options shown, Figure 2 and / or Figure 3 The solutions shown can be used with Figure 4 By way of example, Figure 2 As shown in the example of switching from the front camera to the rear camera, combined with Figure 4 The solution shown can also process the video stream collected by the switched rear camera by applying beautification, filters, adjusting exposure parameters, etc. In this way, for the target application that can call the camera, more additional functions of the camera can be used, thereby enhancing the usability of the target application.

[0078] Based on the aforementioned Figure 4 The solution shown in the figure, the solution for determining the target functional module based on the data request can be achieved by Figure 7 The solution shown is implemented. Figure 7 As shown, it includes: S701: based on the data request, obtaining the data stream formed by the designated acquisition device, so that the target application at least presents the data stream through the display screen; S702: obtaining the gesture input generated when the data stream is presented on the display screen; S703: determining the target functional module based on the gesture input.

[0079] That is to say, in this embodiment, the data stream can be adjusted by gesture input to obtain the target data stream during the process of the third-party application presenting the data stream in real time through the display screen. For example, user A uses the video communication application through device A to communicate with user B who uses device B via video. The video communication application of device A is in a working state to obtain the data collected by the data collection device (such as the front camera of device A), respond to the data request and provide the image frame collected by the front camera in real time (that is, the original data stream of the data request), so that the video communication application displays the image frame in real time through the display screen and sends the image frame to the device B in real time, so that device B can also see the picture of user A. In this process, the camera service monitors whether the gesture collection device (such as the touch sensing unit of the display screen of device A) obtains the input gesture. If the input gesture is obtained, the target data stream is generated based on the target functional module corresponding to the input gesture. For example, user A can make gesture input on the display screen (for example, through a pinch gesture), and the touch of the display screen of device A obtains the gesture input as a target gesture. The target function module of the target gesture is to adjust the focal length, and the focal length adjustment is determined based on the parameters of the target gesture (such as the track generated on the display screen), and the focal length of the designated acquisition device is changed based on the adjusted focal length, and the target data stream is obtained by the adjusted designated acquisition device. And the target data stream is provided to the video communication application, so that the video communication application displays the image frame of the target data stream in real time through the display screen and sends the target data stream to device B in real time. Thereby achieving the effect of adjusting the focal length of the camera through gestures during the video call by the video communication application (for example, a third-party application such as WeChat). Compared with the related art, the video communication application can only rely on user A to approach device A or user A to hold device A close to user A's face during the video call, which is more efficient, and the interaction method is more natural and efficient.

[0080] Here, unlike the aforementioned method of selecting a target object based on a pop-up window interface, this embodiment uses an input gesture to select a target object, making the selection more direct and efficient.

[0081] For example, taking the target application as a video call software, when the video call software calls the default camera such as the front camera, the user expects to realize the video call function of the video call software, and believes that there is a data request from the target application to the specified image acquisition device. The camera service in the camera service layer monitors or detects that the target application in the application layer has a data request to call the front camera, and hands the data request to the camera HAL layer for processing. The camera HAL layer enables the use permission of the front camera, and controls the front camera to collect the external environment of the electronic device at the collection angle of the front camera, such as collecting the face image of the user in the call video. The video stream collected by the front camera is transmitted from the camera HAL layer to the application layer via the camera service layer. The target application in the application layer displays the face image of the user in the first display area of ​​the display screen, and transmits the face image to the electronic device of the other user for display.

[0082] In the process of using video call software to display the face image of the user on the first display area of ​​the display screen and the face image of the other user on the second display area of ​​the display screen, if the cameraservice in the camera service layer detects that there is a gesture input such as sliding or drawing a pattern on the display screen, it is considered that there is a need for the user on the side to change the display effect of the face image, collects the input gesture action, and hands it over to the cameraHAL layer for processing. The camera HAL layer analyzes the attributes of the gesture action, such as analyzing the trajectory of the gesture action. If it is found through the analysis of the trajectory of the gesture action that the gesture action is an action of sliding from the left side of the display screen to the right side, according to the pre-set correspondence between each functional module and the gesture action between various functional modules, it is known that the user wants to increase the exposure parameter of the face image in the video call, then the camera HAL layer calls the exposure parameter adjustment function module, and uses the exposure parameter adjustment algorithm provided by the exposure parameter adjustment function module to adjust the exposure parameter of the face image collected by the front camera upward, so that the face image becomes brighter. The face image with the adjusted exposure parameter is transmitted from the camera HAL layer to the application layer via the camera service layer. The target application of the application layer displays the face image of the user on the one hand after the exposure parameters are adjusted in the first display area of ​​the display screen, and transmits the image to the electronic device of the other hand for display.

[0083] Among them, the correspondence between each functional module and the gesture actions between various functional modules is preset. For example, the gesture action corresponding to turning on the beauty function module is to slide from the top to the bottom of the display screen, and the gesture action corresponding to turning off the beauty function module is to slide from the bottom to the top of the display screen. When increasing the exposure parameter, the gesture action corresponding to the exposure parameter adjustment function module is to slide from the left side of the display screen to the right side; when reducing the exposure parameter, the gesture action corresponding to the exposure parameter adjustment function module is to slide from the right side of the display screen to the left side.

[0084] The above is an explanation of adjusting the exposure parameters of the video stream collected by the default camera by using input gestures. In addition, the video stream collected by the default camera can also be adjusted for display effects such as beauty and filters based on input gestures. This adjustment of the display effect of the video stream collected by the default camera based on input gestures in the target application not only realizes the use of more additional functions of the camera in the target application, but also realizes direct and efficient adjustment of the display effect. It provides convenience for users to adjust the display effect of the video stream.

[0085] It should be noted that the target application in the embodiments of the present application may be a local application in the electronic device that can call the image acquisition device. If the target application is a local camera application, the local camera application may implement camera switching, beauty, filter and other display effect processing functions of the video stream acquired by the camera based on the camera switching function module, beauty, filter and other function modules set for the local camera application in the electronic device.

[0086] The target application in the embodiment of the present application can also be a third-party application of the electronic device. Taking the target application as a third-party application that can call the image acquisition device as an example, the third-party application can call the camera switching function module, beauty, filter and other function modules set for the local camera application in the electronic device to implement the corresponding functions. It is also possible to set up a set of function module groups specifically for third-party applications in the electronic device, and these function module groups include multiple function modules related to the camera that can realize the aforementioned additional functions. For example, it includes a function module for realizing the processing of the video stream captured by the camera, such as beauty, filter and other functions, a function module for realizing camera switching, and a function module for realizing screen sharing.

[0087] In the solution of setting up a set of functional modules for third-party applications, a corresponding interface (API) is set for each functional module in the functional module at the HAL layer. When responding to a data request, the HAL layer implements the call of the functional module to be called through the interface corresponding to the functional module to be called.

[0088] In actual applications, the function modules set for the local camera application in the electronic device include not only the aforementioned modules, but also other reasonable modules such as time-lapse photography, fun disguise, watermark attachment and other modules. That is, the function modules set for the local camera application in the electronic device are usually more, so the function modules related to the image acquisition device included in the third-party application (target application) can be set to be less than the function modules related to the image acquisition device included in the specific application (local application that can call the image acquisition device, such as the local camera application). That is, in the embodiment of the present application, the priority target application is the third-party application. Based on the call to the camera in the third-party application, not only the existing functions of the camera can be realized, but also the additional functions of the camera can be realized, realizing the successful embedding of the camera function in the third-party application. In this way, the use of the camera function by the third-party application is no longer single, and the use of the camera function is more diversified, which can make users feel a good visual experience.

[0089] In the embodiment of the present application, in the solution of calling the camera by a third-party application, in addition to the camera switching function, the processing function of display effects such as beauty and filters, and the screen sharing function, it also has the function of identifying the other party's data and / or extracting key information during the video call. Exemplarily, in the process of the video call software calling the default camera to make a video call with the other user, the camera service used to monitor the target application intercepts the data transmitted by the other user (such as the file image of the other user), and performs optical character recognition (OCR, Optical Character Recognition) on the intercepted data to obtain the text data corresponding to the file image. Semantic recognition technology can also be used to identify the main semantics of the file data, so as to obtain the key information or main meaning expressed by the file image. This automatic recognition of the other party's data and / or automatic extraction of key information greatly facilitates users and improves user experience.

[0090] In the above scheme, the acquisition device is an image acquisition device as an example for explanation. In addition, the acquisition device in the embodiment of the present application can also be an audio acquisition device (microphone). It can be understood that the microphones can be divided into the following types based on their uses: conference microphones, voice microphones, instrument microphones, KTV / performance microphones, etc. In the embodiment of the present application, based on the data request of the target application for the default microphone (such as a voice microphone), the call from the default microphone to other types of microphones can be realized, and the call from the default microphone to the default microphone and other types of microphones can also be realized. In this way, the audio output effects of different microphones are realized in the target application.

[0091] In addition, when the acquisition device is an audio acquisition device, the audio function modules that can be set in the electronic device include an audio speed change function module, an audio noise reduction function module, and an audio effect adjustment module. Among them, the audio speed change algorithm provided by the audio speed change function module can be used to slow down or speed up the audio collected by the microphone, thereby realizing slow output or fast output of the audio speech speed in the target application. It is also possible to use the noise reduction algorithm provided by the audio noise reduction function module to perform noise reduction processing on the audio collected by the microphone, thereby realizing clear output of the audio and removing noise in the target application. It is also possible to use the effect adjustment algorithm provided by the audio effect adjustment module to process the audio collected by the microphone with audio effects, such as changing the normal sound of the audio into a heavy sound or a low cannon sound. Thereby, the output of the desired output effect of the audio can be realized in the target application. The use of the microphone function by the target application is more diversified and no longer single.

[0092] The present application also provides a data processing device embodiment, such as Fig. 9 As shown, the device comprises:

[0093] The first obtaining unit 901 is used to obtain a data request from a target application to a designated collection device;

[0094] A second obtaining unit 902 is used to obtain a target data stream based on the data request;

[0095] The third obtaining unit 903 is configured to use the target data stream as a response to the data request so that the target application processes the target data stream, wherein the target data stream is different from the data stream formed by the designated acquisition device.

[0096] In some embodiments, the second obtaining unit 902 is used to determine a target acquisition device; the target acquisition device is different from the designated acquisition device;

[0097] A data stream formed by the target acquisition device is obtained as the target data stream.

[0098] In some embodiments, the second obtaining unit 902 is used to determine multiple target acquisition devices; the multiple target acquisition devices at least include the designated acquisition device; and obtain the data stream formed by the acquisition of each of the multiple target acquisition devices as the target data stream.

[0099] In some embodiments, the second obtaining unit 902 is used to determine a target functional module; based on the target functional module, process each frame of data in the data stream formed by the designated acquisition device to form a target data stream.

[0100] In some embodiments, the second obtaining unit 902 is used to determine a target display area of ​​the display screen; obtain screen data of the target display area to form a target data stream; and the screen data is used to present display content on the target display area.

[0101] In some embodiments, the second obtaining unit 902 is used to display an interactive interface in a window form based on the data request; obtain a target instruction based on an interactive operation on the interactive interface; the target instruction is used to determine a target object, and the target object includes at least one of a target acquisition device, a target functional module, and a target screen.

[0102] In some embodiments, the second acquisition unit 902 is used to obtain the data stream formed by the designated acquisition device based on the data request, so that the target application can at least present the data stream through a display screen; obtain the gesture input generated when the display screen presents the data stream; and determine the target functional module based on the gesture input.

[0103] In some embodiments, the target application is any application capable of calling an image acquisition device, and the target application includes fewer functional modules related to the image acquisition device than the specific application.

[0104] It should be noted that the data processing device of the embodiment of the present application solves the problem in a principle similar to that of the aforementioned data processing method. Therefore, the implementation process and implementation principle of the data processing device can refer to the description of the implementation process and implementation principle of the aforementioned method, and the repeated parts will not be repeated.

[0105] According to an embodiment of the present application, the present application also provides an electronic device and a readable storage medium. The electronic device includes at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the aforementioned data processing method.

[0106] The readable storage medium is a non-transitory computer readable storage medium storing computer instructions, and the computer instructions are used to make the computer execute the aforementioned data processing method. For a description of the computer readable storage medium, please refer to Fig.10 See the relevant instructions in .

[0107] Fig.10A schematic block diagram of an example electronic device 800 that can be used to implement an embodiment of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0108] like Fig.10 As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0109] A number of components in the device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0110] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as data processing methods. For example, in some embodiments, the data processing method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the data processing method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the data processing method in any other appropriate manner (e.g., by means of firmware).

[0111] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0112] The program code for implementing the method of the present application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, implements the functions / operations specified in the flow chart and / or block diagram. The program code can be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0113] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0114] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0115] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0116] A computer system may include clients and servers. Clients and servers are generally remote from each other and typically interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other. The server may be a cloud server or a server of a distributed system.

[0117] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A data processing method, comprising: Obtaining a data request from a target application to a designated collection device, wherein the target application is a third-party application, and the designated collection device is a default collection device of the target application; Based on the data request, obtaining a target data stream; Using the target data stream as a response result of the data request so that the target application processes the target data stream, wherein the target data stream is different from the data stream formed by the designated acquisition device; The obtaining of the target data stream comprises: Determine a target acquisition device; the target acquisition device is different from the designated acquisition device, and the camera hardware abstraction layer cameraHAL enables the target application to use the target acquisition device; The camera hardware abstraction layer cameraHAL obtains the data stream formed by the target acquisition device as the target data stream.

2. The method according to claim 1, wherein obtaining the target data stream further comprises: determining a plurality of target acquisition devices; The multiple target acquisition devices at least include the designated acquisition device; A data stream formed by each of the plurality of target acquisition devices is obtained as the target data stream.

3. The method according to claim 1, wherein obtaining the target data stream further comprises: Determine the target functional module; Based on the target function module, each frame of data in the data stream formed by the target acquisition device is processed to form a target data stream.

4. The method according to claim 1, wherein obtaining the target data stream comprises: Determine a target display area of ​​the display screen; Obtaining screen data of the target display area to form a target data stream; The screen data is used to present display content on the target display area.

5. The method according to any one of claims 1 to 4, further comprising: Based on the data request, displaying an interactive interface in the form of a window; Obtaining a target instruction based on an interactive operation on the interactive interface; The target instruction is used to determine a target object, and the target object includes at least one of a target acquisition device, a target function module, and a target screen.

6. The method according to claim 3, wherein the target function module is determined, comprising: Based on the data request, obtaining a data stream formed by the designated acquisition device, so that the target application at least presents the data stream through a display screen; Obtaining gesture input generated when the display screen presents the data stream; A target functional module is determined based on the gesture input.

7. According to the method described in claim 1, the target application is any application that has the ability to call the image acquisition device, the target application includes fewer functional modules related to the image acquisition device than the specific application includes. The specific application includes a local application that can call the image acquisition device, and the local application does not include a third-party application.

8. A data processing device, comprising: A first obtaining unit is used to obtain a data request of a target application to a designated collection device, wherein the target application is a third-party application, and the designated collection device is a default collection device of the target application; A second obtaining unit, configured to obtain a target data stream based on the data request; a third obtaining unit, configured to use the target data stream as a response result of the data request, so that the target application processes the target data stream, wherein the target data stream is different from the data stream formed by the acquisition of the designated acquisition device; The second obtaining unit is further used to determine a target acquisition device; the target acquisition device is different from the designated acquisition device, and the camera hardware abstraction layer cameraHAL enables the target application program to use the target acquisition device; The camera hardware abstraction layer cameraHAL obtains the data stream formed by the target acquisition device as the target data stream.

9. The apparatus according to claim 8, wherein the second obtaining unit is configured to implement at least one of the following based on the data request: Based on the data request, a plurality of second target acquisition devices are determined; the plurality of second target acquisition devices at least include the designated acquisition device; and a data stream formed by each of the plurality of second target acquisition devices is obtained as the target data stream; Based on the data request, determining a target functional module; Based on the target function module, processing each frame of data in the data stream formed by the target acquisition device to form a target data stream; Determine a target display area of ​​the display screen; obtain screen data of the target display area to form a target data stream; The screen data is used to present display content on the target display area.