Imaging data acquisition method and apparatus, and device and readable storage medium

Abstract

The present application relates to the technical field of augmented reality. Disclosed are an imaging data acquisition method and apparatus, and a device and a readable storage medium. The imaging data acquisition method comprises: receiving a camera access request, and on the basis of the camera access request, determining a target camera to be accessed; if the target camera is not turned on when the camera access request is received, turning on the target camera, acquiring a preview image stream captured by the turned-on target camera, acquiring from the preview image stream imaging data of the target camera, and sharing the preview image stream; and if the target camera is turned on when the camera access request is received, determining whether the number of processes currently accessing the target camera is less than or equal to a preset threshold, and if so, acquiring from the shared preview image stream the imaging data of the target camera. The present application enables shared camera use in augmented reality devices.

Description

Camera data acquisition method, device and equipment and readable storage medium

[0001] The present application claims priority to the Chinese patent application No. 202410840940.5, filed on June 26, 2024, and entitled "Camera data acquisition method, device and equipment and readable storage medium", the whole content of which is incorporated herein by reference. TECHNICAL FIELD

[0002] The present application relates to the field of augmented reality technology, in particular to a camera data acquisition method, device, equipment and readable storage medium. BACKGROUND

[0003] AR(Augmented Reality, augmented reality) devices can superimpose real environment and virtual video in real time into the same picture or space for the wearer to view. Common AR devices include AR glasses or AR headsets, etc.

[0004] However, the current AR devices strictly require that the same camera can only be used by one process, that is, only one process is allowed to obtain camera data captured by the camera. For example, if a camera in an AR device is used for gesture control, it cannot be used for code scanning recognition, object tracking, etc. Other processes will disconnect the previous process through certain strategies to ensure that the same camera is only used by one process. SUMMARY

[0005] The main purpose of the present application is to provide a camera data acquisition method, device, equipment and readable storage medium, which aims to solve the technical problem that the same camera of the current AR device can only be used by one process.

[0006] To achieve the above purpose, the present application provides a camera data acquisition method, which is applied to an augmented reality device, and comprises the following steps:

[0007] Receiving a camera access request, and determining a target camera to be accessed according to the camera access request;

[0008] If the target camera is not opened when the camera access request is received, the target camera is opened, a preview image stream captured by the opened target camera is obtained, camera data of the target camera is obtained from the preview image stream, and the preview image stream is shared;

[0009] If the target camera is opened when the camera access request is received, it is determined whether the number of processes currently accessing the target camera is less than or equal to a preset threshold, and if so, camera data of the target camera is obtained from the shared preview image stream.

[0010] In an embodiment, after the step of determining the target camera to be accessed according to the camera access request, the method further comprises:

[0011] If the target camera is opened when the camera access request is received, and the number of processes currently accessing the target camera is greater than the preset threshold, a process to access the target camera is determined according to the camera access request.

[0012] The priority of the process to access the target camera is obtained, and the priorities of all processes currently accessing the target camera are obtained.

[0013] The lowest priority among the priorities is determined, and the process corresponding to the lowest priority is closed.

[0014] In an embodiment, after the step of closing the process corresponding to the lowest priority, the method further comprises:

[0015] If the closed process is not the process to open the target camera, the step of determining whether the number of processes currently accessing the target camera is less than or equal to a preset threshold is returned to.

[0016] If the closed process is the process to open the target camera, the highest priority among the priorities is determined, the target camera is reopened based on the process corresponding to the highest priority, and the step of obtaining the preview image stream captured by the opened target camera is returned to.

[0017] In an embodiment, the step of closing the process corresponding to the lowest priority comprises:

[0018] If there are multiple processes currently accessing the target camera in the process corresponding to the lowest priority, each process corresponding to the lowest priority that is currently accessing the target camera is taken as a candidate closed process.

[0019] At least one candidate closed process is selected from the candidate closed processes as a target process, and the target process is closed.

[0020] In an embodiment, the step of selecting at least one candidate closed process from the candidate closed processes as a target process comprises:

[0021] obtaining an access request initiation time corresponding to each of the candidate closing processes;

[0022] determining an earliest access request initiation time from the access request initiation times, and selecting a candidate closing process corresponding to the earliest access request initiation time as a target process.

[0023] In an embodiment, the step of obtaining the preview image stream captured by the opened target camera comprises:

[0024] initiating a preview request for the target camera, and obtaining the preview image stream captured by the opened target camera based on the preview request.

[0025] In an embodiment, the step of obtaining the camera data of the target camera from the shared preview image stream comprises:

[0026] obtaining allocation information according to the camera access request, and obtaining the camera data of the target camera from the shared preview image stream based on the allocation information.

[0027] In addition, to achieve the above object, the present application also provides a camera data obtaining device, which comprises:

[0028] a receiving module configured to receive a camera access request, and determine a target camera to be accessed according to the camera access request;

[0029] a sharing module configured to, if the target camera is not opened, open the target camera, obtain a preview image stream captured by the opened target camera, obtain camera data of the target camera from the preview image stream, share the preview image stream, and return to the step of receiving the camera access request;

[0030] a obtaining module configured to, if the target camera is opened when the camera access request is received, determine whether a number of processes currently accessing the target camera is less than or equal to a preset threshold, and if yes, obtain the camera data of the target camera from the shared preview image stream.

[0031] In addition, to achieve the above object, the present application also provides an augmented reality device, which comprises a microphone array and a processor, wherein the microphone array is electrically connected with the processor; and the processor is configured to perform the steps of the camera data obtaining method.

[0032] In addition, to achieve the above object, the application further provides a readable storage medium, which is a computer readable storage medium, and a program for implementing the camera data acquisition method is stored on the computer readable storage medium, and the program is executed by a processor to implement the steps of the camera data acquisition method.

[0033] The application further provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the steps of the camera data acquisition method.

[0034] In the application, the augmented reality device receives a camera access request, determines a target camera to be accessed according to the camera access request, opens the target camera if the target camera is not opened when the camera access request is received, acquires a preview image stream photographed by the opened target camera, acquires camera data of the target camera from the preview image stream, and shares the preview image stream, and determines whether the number of processes currently accessing the target camera is less than or equal to a preset threshold if the target camera is opened when the camera access request is received, and acquires camera data of the target camera from the shared preview image stream if yes. In this way, in the embodiment of the application, if the target camera is not opened after the camera access request is received, the target camera is opened, and the preview image stream photographed by the target camera is shared, so that camera data of the target camera can be acquired from the shared preview image stream when other processes access the target camera again, and thus, the camera data of the target camera can be acquired by multiple processes based on the shared preview image stream, and the same camera of the AR device can be used by multiple processes at the same time. BRIEF DESCRIPTION OF DRAWINGS

[0035] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and serve to explain the principles of the application together with the specification.

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, for those skilled in the art, other drawings can also be obtained based on these drawings without creative labor.

[0037] FIG. 1 is a flowchart of a first embodiment of the camera data acquisition method of the application;

[0038] FIG. 2 is a schematic diagram of a camera opening process involved in an embodiment of the camera data acquisition method of the application;

[0039] Fig. 3 is a schematic diagram of a process flow involved in an embodiment of the camera data acquisition method of the present application;

[0040] Fig. 4 is a schematic diagram of the device structure of the camera data acquisition device of the present application;

[0041] Fig. 5 is a schematic diagram of the device structure of the hardware operating environment involved in the camera data acquisition device of an embodiment of the present application.

[0042] The purposes, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0043] In order to make the above purposes, features and advantages of the present application more apparent and understandable, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the embodiments of the present application and the accompanying drawings. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0044] The main solution of the present application is: receiving a camera access request, determining a target camera for pre-access according to the camera access request; if the target camera is not opened when the camera access request is received, opening the target camera, acquiring a preview image stream captured by the opened target camera, acquiring camera data of the target camera from the preview image stream, and sharing the preview image stream; if the target camera is opened when the camera access request is received, determining whether the number of processes currently accessing the target camera is less than or equal to a preset threshold, and if so, acquiring camera data of the target camera from the shared preview image stream.

[0045] After the camera access request is received, if the target camera is not opened, the target camera is opened, and the preview image stream captured by the target camera is shared, so that when a subsequent access request for the target camera is received again, camera data of the target camera is acquired from the shared preview image stream, thereby realizing that multiple processes can acquire camera data of the target camera based on the shared preview image stream, and further realizing that the same camera of an AR device can be used by multiple processes at the same time.

[0046] It should be noted that the execution subject of the embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an augmented reality device capable of realizing the above functions, such as AR glasses, AR helmet, etc. For example, the AR glasses are used as the execution subject to illustrate the embodiments of the present application.

[0047] Based on this, the present application proposes a camera data acquisition method of the first embodiment. Please refer to FIG. 1. The camera data acquisition method comprises steps S10-S30:

[0048] Step S10, receiving a camera access request, and determining a target camera for pre-access according to the camera access request;

[0049] It should be noted that the AR glasses can be AR glasses based on an Android camera system.

[0050] When a certain camera is needed, a camera access request for the camera can be initiated. Specifically, the camera identifier of the camera can be encapsulated in the camera access request, so as to determine the target camera for pre-access based on the camera identifier encapsulated in the camera access request.

[0051] It should be noted that after the process initiates the camera access request for the target camera, if the process accepts the access request for the target camera, the process can acquire the camera data of the target camera, that is, the process starts to use the target camera. During the use of the target camera, the process will not send a camera access request again. Only when the process does not need to use the target camera, a camera stop access request is initiated to terminate the process to acquire the camera data of the target camera, that is, the use of the target camera by the process is terminated.

[0052] It can be understood that at the operating system level, an application (or application program) specifically executes a task through a process, for example, the A application accesses the target camera in the AR glasses. Specifically, at the operating system level, the actual A application executes the target camera access task through a process. Based on this, the process described in the embodiments of the present application can be understood as an application.

[0053] Step S20, if the target camera is not opened when the camera access request is received, opening the target camera, acquiring a preview image stream photographed by the opened target camera, acquiring the camera data of the target camera from the preview image stream, and sharing the preview image stream;

[0054] The preview image stream specifically comprises image frames photographed by the target camera based on a preset photographing frequency.

[0055] It can be understood that in the architecture of the Android camera system, in turn, includes the application layer, the framework layer, the hardware abstraction layer (HAL) and the hardware layer. Among them, the framework layer provides functions such as querying camera device information, opening the camera device, and listening to the camera state change. Based on this, the target camera can be opened by starting the OpenCamera process of the framework layer. During this period, the system requests to configure the stream object to request configuration stream information such as image format, color space, resolution, and HAL pipeline from the HAL layer. After the configuration stream information such as image format, color space, resolution, and HAL pipeline is configured, the configuration stream is obtained. After the configuration stream object is requested, that is, after the configuration stream information is configured, the preview image stream captured by the target camera can be obtained through the configuration stream object. Specifically, the preview image stream is obtained through the pipeline in the configuration stream. Among them, the pipeline is a pipeline in the HAL layer for managing image data flow from the image sensor (such as the camera) to the application, which ensures that image data is transmitted in an efficient and consistent manner, while handling possible data format conversion. The pipeline can include multiple processing stages, such as denoising, white balance adjustment, color correction, cropping, scaling, etc. These processing stages help improve image quality and meet the needs of specific applications.

[0056] The camera data of the target camera is obtained from the preview image stream. Specifically, one frame, multiple frames, or all frames of images can be obtained from the preview image stream as the camera data of the target camera. It is easy to understand that different processes have different needs for the image frames that need to be obtained, such as a gesture recognition process may need to obtain all image frames, while a photographing process may only need to obtain one frame of image. Based on this, the corresponding image frames can be obtained from the preview image stream as the obtained camera data based on the characteristics of the process itself.

[0057] For sharing of the preview image stream, specifically, the stream access interface of the preview image stream can be shared to enable other processes to access the preview image stream through the shared stream access interface.

[0058] In step S30, if the target camera has been opened when the camera access request is received, it is determined whether the number of processes currently accessing the target camera is less than or equal to a preset threshold. If so, the camera data of the target camera is obtained from the shared preview image stream.

[0059] It is considered that the number of configuration stream flows allowed to be configured by the HAL layer is limited, such as only 3 configuration stream flows are allowed to be configured, and each process needs to acquire an image stream through a configuration stream. Based on this, preferably, the preset threshold value is specifically the number of configuration stream flows allowed to be configured by the AR glasses, so that the configuration stream can be successfully configured, and then it is guaranteed that the process can successfully acquire the image stream.

[0060] The process currently accessing the target camera is the process currently using the target camera.

[0061] If the number of processes currently accessing the target camera is greater than the preset threshold value, the number of configuration stream flows configured in the AR glasses has reached the upper limit and cannot be configured any more. At this time, the process that initiates the camera access request this time may fail to acquire the image stream due to the limitation of the configuration stream. Based on this, the camera access request of the process can be rejected, or at least one process currently accessing the target camera can be closed to release the configuration stream of the process, so that the process that initiates the camera access request this time can successfully complete the configuration of the configuration stream, and then successfully acquire the image stream. Further, considering that the process that opens the target camera is the provider of the preview image stream, based on this, preferably, the process that opens the target camera is not closed to ensure the continuous provision of the preview image stream.

[0062] It should be noted that in the architecture of the Android camera system, after the HAL layer returns the image stream flow data, the flow data returned by the HAL layer can be transmitted to the framework layer through a Surface object or an ImageReader object, and then transmitted to the application layer for application use. The Surface object is an interface layer between the framework layer and the hardware abstraction layer (HAL), which is managed by the window management system of Android, and is used for graphics rendering and media playback. The ImageReader object is an interface for receiving image data, which is part of the Camera2 API, allowing applications to directly access image data for further processing, such as real-time preview, shooting, video recording, etc. Based on this, the image stream can be specifically read through the Surface object or the ImageReader object.

[0063] In the embodiment, the AR glasses receiving a camera access request, determining a target camera to be accessed according to the camera access request, if the target camera is not opened when the camera access request is received, opening the target camera, obtaining a preview image stream photographed by the opened target camera, obtaining camera data of the target camera from the preview image stream, and sharing the preview image stream, if the target camera is opened when the camera access request is received, determining whether a process number currently accessing the target camera is less than or equal to a preset threshold, if yes, obtaining the camera data of the target camera from the shared preview image stream. In this way, if the target camera is not opened after the camera access request is received, the target camera is opened, and the preview image stream photographed by the target camera is shared, so that when a subsequent access request of other processes to the target camera is received, the camera data of the target camera is obtained from the shared preview image stream, thereby realizing that multiple processes can obtain the camera data of the target camera based on the shared preview image stream, and realizing that the same camera of the AR glasses can be used by multiple processes at the same time, that is, realizing that the same camera of the AR glasses is shared and used.

[0064] Based on the first embodiment, in the second embodiment, the same or similar contents as the above embodiment one can be referred to the above description, and will not be described hereinafter. On this basis, the step of obtaining the preview image stream photographed by the opened target camera includes:

[0065] Step A10, initiating a preview request for the target camera, and obtaining the preview image stream photographed by the opened target camera based on the preview request.

[0066] In the embodiment, even if the process of opening the target camera does not apply to start the preview request, the preview request for the target camera is initiated, so that the real-time preview image of the target camera can be obtained, and a shared data basis is provided for sharing of the target camera. That is, the preview image stream is the real-time preview image of the target camera, and is all image data photographed by the target camera, so that the process of sharing and using the target camera can obtain the required image data from the preview image stream.

[0067] In a possible implementation, the step of obtaining the shared preview image stream includes:

[0068] Step B10, obtaining stream information according to the camera access request, and obtaining the camera data of the target camera from the shared preview image stream based on the stream information.

[0069] The flow configuration information includes, but is not limited to, an image format, a color space, a resolution, and a pipeline of the HAL, wherein the image parameter information such as the image format, the color space, and the resolution can be encapsulated in the camera access request, and the image parameter information can be obtained by unsealing the camera access request, and the pipeline of the HAL is generated by the system and can be understood as a pipeline for transmitting the preview image stream.

[0070] The shared preview image stream based on the flow configuration information can be obtained through the pipeline in the flow configuration information, and the actually obtained image stream can be understood as stream data output after the preview image stream passes through the pipeline.

[0071] Specifically, the flow configuration information can be merged into a flow configuration object created by a process for opening the target camera to update the configuration flow in the flow configuration object, so that a uniform configuration flow object is used by a subsequent process to obtain image stream data. Specifically, for a process, the preview image stream is obtained based on the configuration flow information corresponding to the process, so that the obtained image stream is stream data in an expected format.

[0072] After the image stream is obtained through the pipeline in the flow configuration information, one frame, multiple frames, or all frames of images in the image stream can be further obtained as camera data of the target camera. It is easy to understand that different processes have different requirements for the image frames to be obtained, and the corresponding image frames can be obtained from the image stream based on the characteristics of the process itself as the obtained camera data.

[0073] Based on the first embodiment and / or the second embodiment of the present application, in the third embodiment of the present application, the same or similar contents as those in the above-mentioned first embodiment and second embodiment can be referred to the above description, and will not be described hereinafter. On this basis, after the step of determining the target camera to be accessed by the process according to the camera access request, the method further includes:

[0074] In step C10, if the target camera is opened when the camera access request is received, and the number of processes currently accessing the target camera is greater than a preset threshold, the process to be accessed to the target camera is determined according to the camera access request.

[0075] Specifically, the process identifier can be encapsulated in the camera access request to determine the process represented by the process identifier as the process to be accessed to the target camera.

[0076] In step C20, the priority of the process to be accessed to the target camera is obtained, and the priorities of all processes currently accessing the target camera are obtained.

[0077] The priority information of the processes can be preset in advance, which includes a setting rule of the priority and / or an adjusting rule of the priority, etc. (e.g., the priority of a process for a target camera is reduced by one when the process accesses the target camera). The priority of the process that pre-accesses the target camera and the priority of all the processes that currently access the target camera are obtained based on the preset priority information.

[0078] For the convenience of subsequent description and illustration, the priority of the process that pre-accesses the target camera is regarded as a first priority, and the priority of the process that currently accesses the target camera is regarded as a second priority.

[0079] In step C30, the lowest priority in the priorities is determined, and the process corresponding to the lowest priority is closed.

[0080] The lowest priority in the priorities is the lowest priority in the first priority and the second priority.

[0081] The process corresponding to the lowest priority can be one or more processes with the lowest priority. Further, if there is a process that currently accesses the target camera in the process corresponding to the lowest priority, one or more processes with the lowest priority and currently accessing the target camera are preferably closed to terminate the access of the process to the target camera, i.e., to end the use of the process to the target camera. In addition, the closed process can be randomly selected or selected based on a preset rule (e.g., a process that initiates a camera access request first, a process that initiates a camera access request last, etc.).

[0082] If there is no process that currently accesses the target camera in the process corresponding to the lowest priority, i.e., the process that pre-accesses the target camera has the lowest priority, the process is closed, i.e., the camera access request of the process is rejected.

[0083] In the embodiment, when the number of processes that currently access the target camera is greater than a preset threshold, the process with the lowest priority is selected to be closed, so that the process with a high priority can use the camera preferentially.

[0084] In a possible implementation, after the step of closing the process corresponding to the lowest priority, the method further includes:

[0085] In step D10, if the closed process is not the process that opens the target camera, the step of determining whether the number of processes that currently access the target camera is less than or equal to a preset threshold is executed again.

[0086] If the closed process is not the process of opening the target camera, the process of opening the target camera still continuously provides the preview image stream for other processes to share the camera data of the target camera.

[0087] Further, if the closed process is not the process of initiating the camera access request, i.e., the process of pre-accessing the target camera, the step of determining whether the number of processes currently accessing the target camera is less than or equal to the preset threshold is returned to execute, so that after the process currently accessing the target camera is closed, the process of pre-accessing the target camera acquires the camera data of the target camera from the shared preview image stream, thereby responding to the camera access request of the process in time.

[0088] Step D20, if the closed process is the process of opening the target camera, the highest priority in each of the priorities is determined, and the target camera is reopened based on the process corresponding to the highest priority, and the step of acquiring the preview image stream captured by the opened target camera is returned to execute.

[0089] If the closed process is the process of opening the target camera, the target camera is reopened based on the process corresponding to the highest process priority. It should be noted that if there are multiple processes corresponding to the highest process priority, one process with the highest process priority can be selected to reopen the target camera to provide the preview image stream again through the process of reopening the target camera. And the process with the highest priority is selected to reopen the target camera, because the priority of the process is high, the probability of the process being forcibly closed subsequently can be reduced.

[0090] Further, if the process of opening the target camera actively gives up using the target camera during the use of the target camera and initiates a camera stop access request, a process with the highest process priority can be selected from the processes currently accessing the target camera to reopen the target camera to provide the preview image stream again through the process of reopening the target camera.

[0091] In a possible implementation, the step of closing the process corresponding to the lowest priority comprises:

[0092] Step E10, if there are multiple processes currently accessing the target camera in the process corresponding to the lowest priority, each process currently accessing the target camera in the process corresponding to the lowest priority is taken as a candidate closed process.

[0093] Step E30, at least one candidate closed process is selected from the candidate closed processes as a target process, and the target process is closed.

[0094] Preferably, one candidate closing process is selected as the target process so that as many processes as possible share the use of the camera, thereby ensuring that as many applications in the AR glasses as possible execute normally.

[0095] In one possible implementation, the step of selecting at least one candidate closing process as the target process from the candidate closing processes comprises:

[0096] Step F10, obtaining the access request initiation time corresponding to each of the candidate closing processes;

[0097] The access initiation time is a time representing the process of initiating the camera access request. As one implementation, the time when the process receives the camera access request can be taken as the access request initiation time corresponding to the process. As another implementation, the process encapsulates the initiation time of the camera access request in the camera access request when initiating the camera access request, and then obtains the access request initiation time corresponding to the process through the camera access request.

[0098] Step F20, determining the earliest access request initiation time from the access request initiation times, and selecting the candidate closing process corresponding to the earliest access request initiation time as the target process.

[0099] In this embodiment, the process with the earliest access request initiation time is selected as the target process, that is, the process is closed in the first-in-first-out principle when the priorities are the same, thereby ensuring that the time of using the camera by each process is relatively balanced, and the phenomenon that the process is kicked out immediately after using the camera and cannot continue to use the camera can be reduced.

[0100] By way of example, in order to assist in understanding the technical concept or technical principle of the camera data acquisition combined with the above-mentioned embodiment one and embodiment two, a specific embodiment is listed. In this specific embodiment, referring to FIGS. 2 to 3, it is assumed that two processes need to share the use of the camera, that is, process A (application A): the system gesture background uses the camera data and does not need preview, and process B (application B): the third party uses the camera data (there can be a preview request, or there can be no preview request). Process A initiates the camera access request first. In FIGS. 2 to 3, the solid line represents the flow of process A, and the dashed line represents the flow of process B. Based on this, the camera data acquisition flow is as follows:

[0101] For process A:

[0102] (1) Process A initiates a camera access request, at this time the camera is not opened, and the application opens the camera, runs the open process of the framework layer, and if the camera is not opened and the camera is allowed to start, the camera is opened, and a client object (also called a camera object) is applied to the HAL layer. When the framework layer receives the client object returned by the HAL layer, it is considered that process A is a qualified client that can obtain image stream data from the HAL layer. At the same time, when a camera is in an unopened state and is applied to be opened by a process, there are two cases: one is to start a preview request, and the other is not to start a preview request. If the preview request is started, it is executed normally, and if the preview request is not started, a preview request for the camera is initiated and sent to the HAL layer, so that the HAL layer returns the obtained image stream (i.e., camera data) of the camera. The framework layer creates a virtual Surface object for receiving the preview image stream (i.e., preview data).

[0103] (2) The image stream returned by the HAL is obtained through the Surface object or the ImageReader object.

[0104] For process B:

[0105] (1) Process B initiates a camera access request and modifies the competition mechanism. At this time, it is found through the running of the open process of the framework layer that the camera has been opened, and the client object of process A is given to process B, and it is not necessary to apply to the HAL layer again. That is, the access of process B is switched to the client access of process A.

[0106] (2) After process B obtains the camera object, it continues to create a configuration stream session, and sends the configuration stream information to the system.

[0107] (3) The system judges that there is a configuration stream object (created by process A) in the system, and then merges the configuration stream information into the previous configuration stream object. After the configuration stream succeeds, if there is a preview request, the preview request is intercepted, and the virtual Surface object of process A is transferred to the preview Surface object of process B.

[0108] (4) The image stream (i.e., camera data) returned by the HAL is obtained through the Surface object or the ImageReader object.

[0109] If process C needs to use the camera at the same time, the competition mechanism is started, process A is retained, process B is kicked out, and process C continues to operate, and the above operation of process B is repeated.

[0110] In the specific embodiments, the system refers to the Android camera system.

[0111] It should be noted that the above specific embodiments are only used to understand the present application, and do not constitute a limitation on the camera data acquisition process of the present application. Based on this technical concept, more forms of simple transformation are within the protection scope of the present application.

[0112] In addition, the embodiment of the present application further provides a camera data acquisition device. Referring to FIG. 4, the camera data acquisition device comprises:

[0113] The receiving module 10 is configured to receive a camera access request, and determine a target camera for pre-access according to the camera access request;

[0114] The sharing module 20 is configured to open the target camera if the target camera is not opened at present, acquire a preview image stream photographed by the opened target camera, acquire camera data of the target camera from the preview image stream, share the preview image stream, and return to execute the step of receiving the camera access request;

[0115] The acquiring module 30 is configured to determine whether a process number of the target camera being accessed at present is less than or equal to a preset threshold if the target camera is opened when the camera access request is received, and acquire the camera data of the target camera from the shared preview image stream if yes.

[0116] In an embodiment, the camera data acquisition device further comprises a closing module, which is configured to:

[0117] If the target camera is opened when the camera access request is received, and the process number of the target camera being accessed at present is greater than the preset threshold, a process of pre-accessing the target camera is determined according to the camera access request;

[0118] The priority of the process of pre-accessing the target camera is acquired, and the priorities of all processes of the target camera being accessed at present are acquired;

[0119] The lowest priority in the priorities is determined, and the process corresponding to the lowest priority is closed.

[0120] In an embodiment, the camera data acquisition device is further configured to:

[0121] If the closed process is not the process of opening the target camera, the acquiring module determines whether the process number of the target camera being accessed at present is less than or equal to the preset threshold;

[0122] If the closed process is a process of opening the target camera, a highest priority among the priorities is determined, the target camera is reopened based on a process corresponding to the highest priority, and the sharing module acquires a preview image stream captured by the reopened target camera.

[0123] In an embodiment, the closing module is further configured to:

[0124] If there are multiple processes currently accessing the target camera among the processes corresponding to the lowest priority, a process currently accessing the target camera among the processes corresponding to the lowest priority is selected as a candidate closing process.

[0125] At least one candidate closing process is selected from the candidate closing processes as a target process, and the target process is closed.

[0126] In an embodiment, the closing module is further configured to:

[0127] An access request initiation time corresponding to each candidate closing process is acquired.

[0128] An earliest access request initiation time among the access request initiation times is determined, and a candidate closing process corresponding to the earliest access request initiation time is selected as a target process.

[0129] In an embodiment, the sharing module 20 is further configured to:

[0130] A preview request for the target camera is initiated, and a preview image stream captured by the target camera opened based on the preview request is acquired.

[0131] In an embodiment, the acquiring module 10 is further configured to:

[0132] Stream allocation information is acquired according to the camera access request, and camera data of the target camera is acquired from the shared preview image stream based on the stream allocation information.

[0133] In addition, an embodiment of the present application further provides an augmented reality device, which comprises a memory, a processor, and a camera data acquisition program stored in the memory and executable on the processor. The camera data acquisition program, when executed by the processor, implements the steps of the camera data acquisition method.

[0134] Further, as shown in FIG. 5, the augmented reality device can also include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to a program stored in a read only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. In the RAM 1004, various programs and data required for the operation of the augmented reality device are also stored. The processing device 1001, the ROM 1002, and the RAM 1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; the storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the augmented reality device to communicate wirelessly or by wire with other devices to exchange data. Although the augmented reality device having various systems is shown in the figure, it should be understood that all the shown systems are not required to be implemented or possessed. More or less systems can be alternatively implemented or possessed.

[0135] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network by a communication device, or installed from the storage device 1003, or installed from the ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the methods of the embodiments of the present disclosure are performed.

[0136] The augmented reality device provided by the embodiments of the present disclosure adopts the camera data acquisition method in the above-mentioned embodiments, and can solve the technical problem of camera data acquisition. Compared with the prior art, the augmented reality device provided by the present disclosure has the same beneficial effects as the camera data acquisition method provided by the above-mentioned embodiments, and other technical features in the augmented reality device are the same as the features disclosed in the previous embodiment method, which will not be repeated here.

[0137] It should be understood that various aspects of the disclosure can be implemented in hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any appropriate manner in any one or more embodiments or examples.

[0138] The above description is merely illustrative of the application and not restrictive.

[0139] In addition, to achieve the above object, the embodiment of the present application further provides a readable storage medium having computer readable program instructions (i.e. computer programs) stored thereon, the computer readable program instructions being used to execute the camera data acquisition method in the above embodiment.

[0140] The computer readable storage medium provided by the embodiment of the present application may, for example, be a U disk, but is not limited to an electric, magnetic, optical, electromagnetic, infrared, or semiconductor system, system or device, or any combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electric connection with one or more conductive wires, a portable computer disk, a hard disk, a random access memory (RAM), a read only memory (ROM), an erasable programmable read only memory (EPROM or flash memory), an optical fiber, a portable compact disk read only memory (CD-ROM), an optical storage device, a magnetic storage device, or any appropriate combination of the above. In the embodiment, the computer readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer readable storage medium can be transmitted by any appropriate medium, including but not limited to an electric wire, an optical cable, an RF (Radio Frequency), etc., or any appropriate combination of the above.

[0141] The above computer readable storage medium can be contained in an augmented reality device; or can exist separately and not be assembled into an augmented reality device.

[0142] The computer readable storage medium described above carries one or more programs, when the one or more programs are executed by the augmented reality device, cause the augmented reality device to: receive a camera access request, determine a target camera to be accessed according to the camera access request, if the target camera is not opened when the camera access request is received, open the target camera, acquire a preview image stream captured by the opened target camera, acquire camera data of the target camera from the preview image stream, and share the preview image stream, and if the target camera is opened when the camera access request is received, determine whether a number of processes currently accessing the target camera is less than or equal to a preset threshold, and if yes, acquire the camera data of the target camera from the shared preview image stream.

[0143] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0144] The computer program instructions can also be loaded onto a computer or other programmable information processing apparatus to cause a series of operations to be performed on the computer or other programmable information processing apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable information processing apparatus implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0145] The modules described in the embodiments of the present application can be implemented in the form of software or in the form of hardware. In some cases, the names of the modules do not constitute a limitation on the modules themselves.

[0146] The readable storage medium provided by the present application is a computer readable storage medium, which stores computer readable program instructions (i.e., a computer program) for executing the above-mentioned camera data acquisition method, and can solve the technical problem of camera data acquisition. Compared with the prior art, the computer readable storage medium provided by the present application has the same beneficial effects as the camera data acquisition method provided by the above-mentioned embodiments, which will not be repeated here.

[0147] In addition, the embodiments of the present application also provide a computer program product, which includes a camera data acquisition program, and the camera data acquisition program is executed by a processor to implement the steps of the camera data acquisition method as described above.

[0148] The computer program product of the present application has substantially the same implementation as the above-mentioned camera data acquisition method, which will not be repeated here.

[0149] It should be noted that in this paper, the term "including", "containing" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or system including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or includes elements inherent to such process, method, article or system. Without more limitations, the element defined by the statement "including a" does not exclude the presence of another identical element in the process, method, article or system including the element.

[0150] The serial numbers of the above-mentioned embodiments of the present application are only for description, not representing the advantages and disadvantages of the embodiments.

[0151] Through the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment method can be realized by software and necessary general hardware platform, of course, it can also be realized by hardware, but in many cases, the former is a better implementation. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software sensor, which is stored in a storage medium (such as ROM / RAM, magnetic disc, optical disc) and includes a plurality of instructions for making a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) execute the method described in each embodiment of the present application.

[0152] The above merely preferred embodiments of the present application and are not intended to limit the patent scope of the present application, any equivalent structure or equivalent process transformation using the content of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for acquiring camera data, characterized in that, The camera data acquisition method, applied to augmented reality devices, includes the following steps: Receive a camera access request and determine the target camera to be accessed based on the camera access request; If the target camera is not turned on when the camera access request is received, the target camera is turned on, the preview image stream captured by the turned-on target camera is obtained, the camera data of the target camera is obtained from the preview image stream, and the preview image stream is shared. If the target camera is already open when the camera access request is received, it is determined whether the number of processes currently accessing the target camera is less than or equal to a preset threshold. If so, the camera data of the target camera is obtained from the shared preview image stream.

2. The method as described in claim 1, characterized in that, After the step of determining the target camera to be accessed based on the camera access request, the method further includes: If the target camera is already open when the camera access request is received, and the number of processes currently accessing the target camera is greater than a preset threshold, then the process to access the target camera is determined based on the camera access request. Obtain the priority of the process that is about to access the target camera, and obtain the priority of all processes that are currently accessing the target camera; Determine the lowest priority among the priorities and shut down the process corresponding to the lowest priority.

3. The method as described in claim 2, characterized in that, After the step of shutting down the process corresponding to the lowest priority, the method further includes: If the closed process is not the process that opened the target camera, then return to the step of determining whether the number of processes currently accessing the target camera is less than or equal to a preset threshold. If the process that is closed is the process that opens the target camera, then the highest priority among the priorities is determined, the target camera is reopened based on the process corresponding to the highest priority, and the process returns to the step of obtaining the preview image stream captured by the opened target camera.

4. The method as described in claim 2, characterized in that, The step of shutting down the process corresponding to the lowest priority includes: If there are multiple processes currently accessing the target camera among the processes corresponding to the lowest priority, then the processes currently accessing the target camera among the processes corresponding to the lowest priority will be selected as candidate processes to be closed. Select at least one candidate process to be shut down as the target process from the candidate shutdown processes, and shut down the target process.

5. The method as described in claim 4, characterized in that, The step of selecting at least one candidate shutdown process as the target process from the candidate shutdown processes includes: Obtain the access request initiation time corresponding to each of the candidate shutdown processes; Determine the earliest access request initiation time among the initiation times of each access request, and select the candidate shutdown process corresponding to the earliest access request initiation time as the target process.

6. The method according to any one of claims 1 to 5, characterized in that, The step of acquiring the preview image stream captured by the opened target camera includes: Initiate a preview request for the target camera, and obtain a preview image stream captured by the opened target camera based on the preview request.

7. The method according to any one of claims 1 to 5, characterized in that, The step of obtaining the camera data of the target camera from the shared preview image stream includes: Based on the camera access request, obtain the distribution information, and based on the distribution information, obtain the camera data of the target camera from the shared preview image stream.

8. A camera data acquisition device, characterized in that, The camera data acquisition device includes: The receiving module is used to receive camera access requests and determine the target camera to be accessed based on the camera access requests. The sharing module is used to open the target camera if the target camera is not currently open, obtain the preview image stream captured by the open target camera, obtain the camera data of the target camera from the preview image stream, share the preview image stream, and return to the step of receiving the camera access request. The acquisition module is used to determine whether the number of processes currently accessing the target camera is less than or equal to a preset threshold if the target camera is already open when the camera access request is received. If so, the module acquires the camera data of the target camera from the shared preview image stream.

9. An augmented reality device, characterized in that, The augmented reality device includes: a memory, a processor, and a camera data acquisition program stored in the memory and executable on the processor, wherein the camera data acquisition program, when executed by the processor, implements the steps of the camera data acquisition method as described in any one of claims 1 to 7.

10. A readable storage medium, characterized in that, The readable storage medium is a computer-readable storage medium, and the computer-readable storage medium stores a program that implements the camera data acquisition method. The program that implements the camera data acquisition method is executed by a processor to implement the steps of the camera data acquisition method as described in any one of claims 1 to 7.

Images