Data generation method, image pickup apparatus, head-mounted display apparatus, and readable medium

Abstract

Embodiments of the present disclosure disclose a data generation method, a camera device, a head-mounted display device and a readable medium. The method is applied to a camera device, and the camera device comprises a camera, an inertial measurement device and a master control unit. A specific embodiment of the method comprises: in response to detecting that the camera starts to expose, determining a synchronization signal generation time according to an updated exposure time and an exposure start time; in response to determining that the master control unit receives the synchronization signal, determining a time when the synchronization signal is received as an image time; performing correlation processing on an image corresponding to the synchronization signal collected by the camera and the image time to obtain a correlation image; in response to determining that the master control unit receives inertial measurement data collected by the inertial measurement device, determining a time when the master control unit receives the inertial measurement data as an inertial measurement time; and performing correlation processing on the inertial measurement data and the inertial measurement time to obtain correlation inertial measurement data. The embodiment improves the accuracy of data synchronization.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to the technical field of computer, and particularly to a data generation method, a camera device, a head-mounted display device and a readable medium. BACKGROUND

[0002] In the seamless integration of physical environment and virtual content, virtual reality technology needs to collect image data and pose data, and perform data synchronization processing on the image data and the pose data. At present, when performing data synchronization processing on the image data and the pose data, the commonly used way is: determining the timestamp of the image data through one chip, determining the timestamp of the pose data through another chip, to perform data synchronization through the timestamps, or generating a synchronization signal when the camera generates an image and sending it to the pose data acquisition device, and the pose data acquisition device generates pose data after receiving the synchronization signal, to realize the synchronization of the image data and the pose data.

[0003] However, the inventors have found that when the above-mentioned way is used to synchronize the image data and the pose data, the following technical problems often exist: the way of determining the timestamp by using different chips has poor data synchronization accuracy because the timestamps come from different clock sources; the way of sending a synchronization signal to the pose data acquisition device cannot output data when the pose data acquisition device does not receive the synchronization signal, resulting in discontinuous pose data, and there is still a fixed time offset between the image data and the pose data after data synchronization, resulting in poor data synchronization accuracy.

[0004] The above information disclosed in this Background section is only for the purpose of enhancing the understanding of the background of the present inventive concepts, and therefore, it can contain information that does not form the prior art that is already known to those of ordinary skill in the art in the country. SUMMARY

[0005] The summary of the present disclosure is intended to introduce the concepts in a simplified form, which will be described in detail in the specific embodiments section. The summary of the present disclosure is not intended to identify key or essential features of the claimed technical solutions, nor is it intended to limit the scope of the claimed technical solutions.

[0006] Some embodiments of the present disclosure propose a data generation method, a camera device, a head-mounted display device and a computer readable medium to solve one or more of the technical problems mentioned in the background section.

[0007] In a first aspect, some embodiments of the present disclosure provide a data generation method applied to a camera device, the camera device comprising a camera, an inertial measurement device, and a master control unit. The method comprises: in response to detecting that the camera starts exposure, determining a synchronization signal generation time of the camera according to an updated exposure time and an exposure start time, so that the camera sends a synchronization signal to the master control unit at the synchronization signal generation time; wherein the camera and the inertial measurement device are connected to the master control unit; in response to determining that the master control unit receives the synchronization signal, determining a time when the master control unit receives the synchronization signal as an image time; performing association processing on an image corresponding to the synchronization signal collected by the camera and the image time to obtain an associated image; in response to determining that the master control unit receives inertial measurement data collected by the inertial measurement device, determining a time when the master control unit receives the inertial measurement data as an inertial measurement time; and performing association processing on the inertial measurement data and the inertial measurement time to obtain associated inertial measurement data.

[0008] Optionally, the method further comprises: for each of the obtained associated inertial measurement data, determining, as synchronous inertial measurement data, associated inertial measurement data in which the corresponding inertial measurement time is the same as the image time corresponding to the associated image; and performing synchronization processing on the associated image and the synchronous inertial measurement data to obtain synchronization data.

[0009] Optionally, the method further comprises: determining positioning information of the camera device in a three-dimensional space according to the synchronization data.

[0010] Optionally, the determination of the synchronization signal generation time of the camera according to the updated exposure time and the exposure start time comprises: determining the synchronization signal generation time of the camera according to an exposure type of the camera, the updated exposure time, and the exposure start time.

[0011] Optionally, the determination of the synchronization signal generation time of the camera according to the exposure type of the camera, the updated exposure time, and the exposure start time comprises: generating an intermediate exposure time according to the exposure type and the exposure time; and determining the sum of the exposure start time and the intermediate exposure time as the synchronization signal generation time of the camera.

[0012] Optionally, the generation of the intermediate exposure time according to the exposure type and the exposure time comprises: in response to determining that the exposure type is row-by-row exposure, determining the sum of a preset transmission time and the exposure time as a total exposure time; and generating the intermediate exposure time according to the determined total exposure time.

[0013] Optionally, before the step of generating the intermediate exposure time according to the determined total exposure time, the method further comprises: in response to determining that the exposure type is global exposure, determining the exposure time as the total exposure time.

[0014] In a second aspect, some embodiments of the present disclosure provide a camera device, comprising a camera, an inertial measurement device, and a master control unit, wherein the camera is connected to the master control unit, the camera is configured to generate a synchronization signal, and send the generated synchronization signal to the master control unit at a synchronization signal generation time, and send an image collected by the camera to the master control unit; the inertial measurement device is connected to the master control unit, the inertial measurement device is configured to collect inertial measurement data, and send the inertial measurement data to the master control unit; and the master control unit is configured to implement the method described in any of the implementations of the first aspect.

[0015] In a third aspect, some embodiments of the present disclosure provide a head-mounted display device, comprising a head-mounted display device body and a camera device as described in any of the implementations of the second aspect.

[0016] In a fourth aspect, some embodiments of the present disclosure provide a computer readable medium having stored thereon a computer program, wherein the program, when executed by a processor, implements the method described in any of the implementations of the first aspect.

[0017] The above embodiments of this disclosure have the following beneficial effects: the data generation method of some embodiments of this disclosure improves the accuracy of data synchronization. Specifically, the reason for poor data synchronization accuracy is that: using different chips to determine the timestamps results in poor data synchronization accuracy because the timestamps come from different clock sources; sending synchronization signals to the attitude data acquisition device means that the attitude data acquisition device cannot output data when it does not receive the synchronization signal, resulting in discontinuous acquired attitude data, and a fixed time offset still exists between the image data and attitude data after data synchronization, leading to poor data synchronization accuracy. Based on this, the data generation method of some embodiments of this disclosure is applied to a camera device, which includes a camera device, an inertial measurement device, and a main control unit. First, in response to detecting that the camera device has started exposure, the synchronization signal generation time of the camera device is determined according to the updated exposure duration and exposure start time, so that the camera device sends the synchronization signal to the main control unit at the synchronization signal generation time, wherein the camera device and the inertial measurement device are both connected to the main control unit. Therefore, the camera device can send a synchronization signal to the main control unit at the moment the synchronization signal is generated, thus controlling the timing of the synchronization signal generation by the camera device. Then, in response to determining that the main control unit has received the synchronization signal, the moment when the main control unit receives the synchronization signal is determined as the image time. Thus, the timestamp of the image can be determined according to the clock of the main control unit. Next, the image corresponding to the synchronization signal acquired by the camera device and the image time are correlated to obtain a correlated image. Thus, the image and its timestamp can be correlated. Secondly, in response to determining that the main control unit has received inertial measurement data acquired by the inertial measurement device, the moment when the main control unit receives the inertial measurement data is determined as the inertial measurement time. Thus, the timestamp of the inertial measurement data can be determined according to the clock of the main control unit. Finally, the inertial measurement data and the inertial measurement time are correlated to obtain correlated inertial measurement data. Thus, the inertial measurement data and its timestamp can be correlated. Because the timestamps of both the image and the inertial measurement data are determined by the main control unit, timestamps from different clock sources are avoided, improving the accuracy of data synchronization. Furthermore, because the synchronization signal is sent directly to the main control unit, the inertial measurement device can always be in working condition and can output data at any time, avoiding the problems of discontinuous inertial measurement data collected by the inertial measurement device and fixed time offset between the image and the inertial measurement data, thereby improving the accuracy of data synchronization. Attached Figure Description

[0018] The above and other features, advantages, and aspects of embodiments of the present disclosure will become more apparent by describing in detail some embodiments thereof with reference to the annexed drawings in which: like reference numerals refer to like elements throughout. The annexed drawings are schematic and are not intended to accurately depict the proportions or relative positions of elements.

[0019] Figure 1 is a flowchart of some embodiments of a data generation method according to the present disclosure;

[0020] Figure 2 is an exposure schematic diagram under line exposure of some embodiments of a data generation method according to the present disclosure;

[0021] Figure 3 is an exposure schematic diagram under global exposure of some embodiments of a data generation method according to the present disclosure;

[0022] Figure 4 is a flowchart of some other embodiments of a data generation method according to the present disclosure;

[0023] Figure 5 is a structural schematic diagram of a camera device suitable for use to implement some embodiments of the present disclosure;

[0024] Figure 6 is a structural schematic diagram of a head-mounted display device suitable for use to implement some embodiments of the present disclosure. DETAILED DESCRIPTION

[0025] Embodiments of the present disclosure will be described below in greater detail with reference to the drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms, and should not be construed as being limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure can be more thoroughly and completely understood. It should be understood that the drawings and embodiments of the present disclosure are merely for exemplary purposes, and are not intended to limit the scope of protection of the present disclosure.

[0026] It should also be noted that, for ease of description, only parts related to the present application are shown in the drawings. The embodiments in the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0027] It should be noted that the terms "first", "second", and the like mentioned in the present disclosure are merely used to distinguish different devices, modules, or units, and are not intended to limit the order or interdependence of the functions performed by these devices, modules, or units.

[0028] It should be noted that the terms "one", "multiple" mentioned in the present disclosure are illustrative and not restrictive, and those skilled in the art should understand that, unless otherwise explicitly indicated in the context, it should be understood as "one or more".

[0029] The names of messages or information exchanged between the plurality of devices in the embodiments of the present disclosure are only for illustrative purposes, and are not intended to limit the scope of the messages or information.

[0030] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0031] Figure 1 A flow 100 of some embodiments of a data generation method according to the present disclosure is shown. The data generation method, applied to a camera device, comprising an image capturing device, an inertial measurement device, and a master control unit, comprises the following steps:

[0032] At step 101, in response to detecting that the image capturing device starts exposure, a synchronization signal generation time of the image capturing device is determined according to an updated exposure time length and an exposure start time, so that the image capturing device sends a synchronization signal to the master control unit at the synchronization signal generation time.

[0033] In some embodiments, an execution subject (e.g., a camera device) of the data generation method can, in response to detecting that the image capturing device starts exposure, determine a synchronization signal generation time of the image capturing device according to an updated exposure time length and an exposure start time, so that the image capturing device sends a synchronization signal to the master control unit at the synchronization signal generation time. The image capturing device can be a device for capturing images. For example, the image capturing device can be a camera. The inertial measurement device can be a device for measuring at least three-axis attitude angles, angular rates, and accelerations of an object. The master control unit can be a master control chip. The image capturing device and the inertial measurement device are both connected to the master control unit. The updated exposure time length can be an adjusted exposure time length when the image capturing device adjusts the exposure time length according to external brightness. The exposure start time can be a time when the image capturing device starts exposure. The synchronization signal generation time can be a time when the image capturing device generates a synchronization signal. In practice, the execution subject can determine the synchronization signal generation time of the image capturing device according to the exposure type of the image capturing device, the updated exposure time length, and the exposure start time.

[0034] In some optional implementations of some embodiments, first, the execution subject can generate an exposure intermediate time length according to the exposure type and the exposure time length. Then, the sum of the exposure start time and the exposure intermediate time length can be determined as the synchronization signal generation time of the image capturing device.

[0035] In some optional implementations of some embodiments, first, the execution subject can determine the sum of the preset transmission duration and the exposure duration as the total exposure duration in response to determining that the exposure type is row-by-row exposure. When the exposure type is row-by-row exposure, the transmission duration can be the duration from the end of the first row exposure to the end of the last row exposure, which is preset. In row-by-row exposure, the exposure duration of all rows is the same. Then, the exposure intermediate duration can be generated according to the determined total exposure duration. In practice, the execution subject can determine half of the total exposure duration as the exposure intermediate duration. As an example, the exposure diagram under row-by-row exposure can refer to FIG. 2. Figure 2 . Figure 2 In the exposure diagram, the synchronization signal generation moment is at the middle of the exposure start moment and the last row exposure end moment.

[0036] Optionally, the execution subject can also determine the exposure duration as the total exposure duration in response to determining that the exposure type is global exposure. In global exposure, all rows are exposed at the same time, so the exposure duration can be determined as the total exposure duration. As an example, the exposure diagram under global exposure can refer to FIG. 3. Figure 3 . Figure 3 In the exposure diagram, the synchronization signal generation moment is at the middle of the exposure start moment and the exposure end moment.

[0037] Optionally, the updated exposure duration can be replaced by a preset exposure duration, at which time, the out-of-line brightness stabilization can be applied without adjusting the exposure duration according to the out-of-line brightness.

[0038] Step 102, in response to determining that the master control unit receives the synchronization signal, determining the moment when the master control unit receives the synchronization signal as the image moment.

[0039] In some embodiments, the execution subject can determine the moment when the master control unit receives the synchronization signal as the image moment in response to determining that the master control unit receives the synchronization signal. In practice, the execution subject can determine the moment when the master control unit receives the synchronization signal as the image moment.

[0040] Optionally, the master control unit can generate a system interrupt when it receives the synchronization signal. In practice, the execution subject can determine the moment when the master control unit generates the system interrupt as the image moment.

[0041] Step 103, associating the image corresponding to the synchronization signal captured by the camera device and the image moment to obtain an associated image.

[0042] In some embodiments, the execution subject can associate the image corresponding to the synchronization signal collected by the camera with the image time point to obtain an associated image. In practice, the execution subject can determine the image time point as the time stamp of the image to associate the image with the image time point.

[0043] At step 104, in response to determining that the master unit receives the inertial measurement data collected by the inertial measurement device, the execution subject determines the time point at which the master unit receives the inertial measurement data as the inertial measurement time point.

[0044] In some embodiments, the execution subject can determine the time point at which the master unit receives the inertial measurement data as the inertial measurement time point in response to determining that the master unit receives the inertial measurement data collected by the inertial measurement device. In practice, the execution subject can determine the time point at which the master unit receives the inertial measurement data as the inertial measurement time point.

[0045] Optionally, the master unit receiving the inertial measurement data can generate a system interrupt. In practice, the execution subject can determine the time point at which the master unit receives the inertial measurement data as the inertial measurement time point.

[0046] At step 105, the execution subject associates the inertial measurement data with the inertial measurement time point to obtain associated inertial measurement data.

[0047] In some embodiments, the execution subject can associate the inertial measurement data with the inertial measurement time point to obtain associated inertial measurement data. In practice, the execution subject can determine the inertial measurement time point as the time stamp of the inertial measurement data to associate the inertial measurement data with the inertial measurement time point.

[0048] The above various embodiments of the present disclosure have the following beneficial effects: through the data generation method of some embodiments of the present disclosure, the accuracy of data synchronization is improved. Specifically, the reason for poor accuracy of data synchronization is that different chips are used to determine the timestamp, and since the timestamp comes from different clock sources, the accuracy of data synchronization is poor; the way of sending a synchronization signal to the attitude data acquisition device, the attitude data acquisition device cannot output data when it does not receive the synchronization signal, resulting in discontinuous collected attitude data, and there is still a fixed time offset between the image data and the attitude data after data synchronization, resulting in poor accuracy of data synchronization. Based on this, the data generation method of some embodiments of the present disclosure is applied to a camera device, which includes a camera device, an inertial measurement device, and a master control unit. First, in response to detecting that the camera device starts to expose, the synchronization signal generation time of the camera device is determined according to the updated exposure time and the exposure start time, so that the camera device sends a synchronization signal to the master control unit at the synchronization signal generation time, wherein the camera device and the inertial measurement device are connected to the master control unit. Thus, the camera device can send a synchronization signal to the master control unit at the synchronization signal generation time, achieving the time of controlling the camera device to generate a synchronization signal. Then, in response to determining that the master control unit receives the synchronization signal, the time when the master control unit receives the synchronization signal is determined as the image time. Thus, the timestamp of the image can be determined according to the clock of the master control unit. Thereafter, the image corresponding to the synchronization signal collected by the camera device and the image time are associated to obtain an associated image. Thus, the image and the timestamp of the image can be associated. Secondly, in response to determining that the master control unit receives the inertial measurement data collected by the inertial measurement device, the time when the master control unit receives the inertial measurement data is determined as the inertial measurement time. Thus, the timestamp of the inertial measurement data can be determined according to the clock of the master control unit. Finally, the inertial measurement data and the inertial measurement time are associated to obtain associated inertial measurement data. Thus, the inertial measurement data and the timestamp of the inertial measurement data can be associated. Because the timestamps of the image and the inertial measurement data are both determined by the master control unit, it can be avoided that the timestamps come from different clock sources, and the accuracy of data synchronization is improved. Because the synchronization signal is sent directly to the master control unit, the inertial measurement device can always be in a working state and can output data at any time, avoiding the problems of discontinuous inertial measurement data collected by the inertial measurement device and fixed time offset between the image and the inertial measurement data, thereby improving the accuracy of data synchronization.

[0049] Further reference Figure 4FIG. 4 shows a flow 400 of another embodiment of the data generation method. The flow 400 of the data generation method is applied to a camera device including an image capturing apparatus, an inertial measurement apparatus, and a master control unit, and includes the following steps:

[0050] Step 401, in response to detecting that the image capturing apparatus starts exposure, determining a synchronization signal generation time of the image capturing apparatus according to the updated exposure time length and the exposure start time, so that the image capturing apparatus sends a synchronization signal to the master control unit at the synchronization signal generation time.

[0051] Step 402, in response to determining that the master control unit receives the synchronization signal, determining a time at which the master control unit receives the synchronization signal as an image time.

[0052] Step 403, associating the image corresponding to the synchronization signal captured by the image capturing apparatus and the image time to obtain an associated image.

[0053] Step 404, in response to determining that the master control unit receives the inertial measurement data collected by the inertial measurement apparatus, determining a time at which the master control unit receives the inertial measurement data as an inertial measurement time.

[0054] Step 405, associating the inertial measurement data and the inertial measurement time to obtain associated inertial measurement data.

[0055] In some embodiments, the specific implementation of steps 401-405 and the resulting technical effects can refer to Figure 1 The steps 101-105 in the corresponding embodiments will not be described here.

[0056] Step 406, for each of the obtained associated inertial measurement data, determining the associated inertial measurement data in which the corresponding inertial measurement time is the same as the image time corresponding to the associated image as synchronization inertial measurement data.

[0057] In some embodiments, for each of the obtained associated inertial measurement data, the execution subject (for example, the camera device) of the data generation method can determine the associated inertial measurement data in which the corresponding inertial measurement time is the same as the image time corresponding to the associated image as synchronization inertial measurement data. It can be understood that the frame rate of the inertial measurement apparatus collecting the inertial measurement data is much higher than the frame rate of the image capturing apparatus collecting the image, so that one frame of image corresponds to multiple inertial measurement data, and the associated inertial measurement data corresponding to the associated image needs to be selected from the multiple associated inertial measurement data.

[0058] Step 407, synchronizing the associated image and the synchronization inertial measurement data to obtain synchronization data.

[0059] In some embodiments, the execution subject can synchronize the associated image and the synchronized inertial measurement data to obtain synchronized data. In practice, the execution subject can combine the associated image and the synchronized inertial measurement data into synchronized data. In this way, the synchronization of the associated image and the synchronized inertial measurement data can be realized.

[0060] Optionally, the execution subject can further determine the positioning information of the camera device in a three-dimensional space according to the synchronized data. The three-dimensional space can be a three-dimensional point cloud space. The positioning information can be a positioning position determined according to the constructed real-time environment map. In practice, the execution subject can determine the positioning information of the camera device in the three-dimensional space according to the synchronized data through a SLAM (Simultaneous Localization And Mapping) algorithm. In this way, the accuracy of the real-time positioning data can be improved.

[0061] From Figure 4 it can be seen that, compared with the description of some embodiments corresponding to Figure 1 , the data generation method in some embodiments corresponding to Figure 4 embodies the steps extended by the data synchronization of the associated image and the associated inertial measurement data. In this way, the scheme described in these embodiments can improve the accuracy of the synchronized data.

[0062] Reference will be made to Figure 5 , which shows a structural schematic diagram of a camera device 500 suitable for implementing some embodiments of the present disclosure. Figure 5 The camera device shown is only an example and should not impose any limitation on the functions and use range of the embodiments of the present disclosure.

[0063] As shown in Figure 5 , the camera device 500 can include a camera 501, an inertial measurement device 502, and a master control unit 503. The camera 501 is connected to the master control unit 503. The master control unit 503 can be a master control chip. For example, the control pin of the camera 501 can be directly connected to the input pin of the master control unit 503. The camera 501 can be used to generate a synchronization signal and send the generated synchronization signal to the master control unit 503 at the synchronization signal generation moment, and send the captured image to the master control unit 503. The inertial measurement device 502 is connected to the master control unit 503, and the inertial measurement device 502 is used to capture inertial measurement data and send the inertial measurement data to the master control unit 503. The master control unit is configured to realize the steps in those embodiments corresponding to Figure 1 or 4, which will not be described here again.

[0064] The camera device 500 can further include a processing apparatus (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) or a program loaded into a random access memory (RAM) from a storage apparatus. In the RAM, various programs and data required for the operation of the camera device 500 are also stored. The processing apparatus, the ROM, and the RAM are connected to each other through a bus. An input / output (I / O) interface is also connected to the bus.

[0065] Generally, the following apparatuses can be connected to the I / O interface: input apparatuses including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output apparatuses including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; and a communication apparatus. The communication apparatus can allow the camera device 500 to perform wireless or wired communication with other apparatuses to exchange data. Although Figure 5 The camera device 500 is shown with various apparatuses, but it should be understood that all of the shown apparatuses are not required to be implemented or possessed. More or fewer apparatuses can alternatively be implemented or possessed. Figure 5 Each block shown in the figure can represent one apparatus or, as needed, multiple apparatuses.

[0066] With the camera device of some embodiments of the present disclosure, the image captured by the camera device can be fully synchronized with the inertial measurement data, improving the accuracy of data synchronization.

[0067] Reference is made below to Figure 6 which shows a structural schematic diagram of a head-mounted display device 600 suitable for use to implement some embodiments of the present disclosure. Figure 6 The head-mounted display device shown is merely an example and should not bring any limitation to the function and scope of use of embodiments of the present disclosure.

[0068] As shown in Figure 6 The head-mounted display device 600 includes a head-mounted display device body 601 and a camera device 602. The head-mounted display device body 601 can be a display device for a user to wear and watch a virtual scene. For example, the head-mounted display device body described above can be, but is not limited to, one of the following: AR glasses, MR glasses. The camera device 602 can be, but is not limited to, one of the following: a camera, a camera module, a camera system, etc. Figure 5 The camera device described for the corresponding embodiments is not repeated here.

[0069] The head-mounted display device 600 can further include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) or a program loaded into a random access memory (RAM) from a storage device. In the RAM, various programs and data required for the operation of the head-mounted display device 600 are also stored. The processing device, the ROM, and the RAM are connected to each other through a bus. An input / output (I / O) interface is also connected to the bus.

[0070] Generally, the following devices can be connected to the I / O interface: input devices including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; and communication devices. The communication devices can allow the head-mounted display device 600 to communicate with other devices wirelessly or wired to exchange data. Although Figure 6 The head-mounted display device 600 is shown with various devices, but it should be understood that all of the shown devices are not required to be implemented or present. More or fewer devices can alternatively be implemented or present. Figure 6 Each block shown in the flowcharts can represent a device or multiple devices as needed.

[0071] With the head-mounted display device of some embodiments of the present disclosure, the images captured by the head-mounted display device can be fully synchronized with the inertial measurement data, improving the accuracy of data synchronization. Thus, the fit of the physical environment and the virtual environment can be improved.

[0072] In particular, the processes described above with reference to the flowcharts can be implemented as a computer software program according to some embodiments of the present disclosure. For example, some embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In some such embodiments, the computer program can be downloaded and installed from a network through the communication devices, or installed from the storage device, or installed from the ROM. When the computer program is executed by the processing device, the above-mentioned functions defined in the methods of some embodiments of the present disclosure are performed.

[0073] Note that the computer readable medium in some embodiments of the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination thereof. The computer readable storage medium can be, for example but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any suitable combination of the foregoing. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In some embodiments of the present disclosure, the computer readable storage medium can be any tangible medium that contains or stores a program used by an instruction execution system, apparatus or device, or that can be used by or in connection with an instruction execution system, apparatus or device. In some embodiments of the present disclosure, the computer readable signal medium can include a computer readable program code propagated in or on a carrier medium, in which the computer readable program code is embodied. Such propagated computer readable program code can take many forms, including but not limited to, an electromagnetic signal, an optical signal or any suitable combination of the foregoing. The computer readable signal medium can also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate or transport a program for use by or in connection with an instruction execution system, apparatus or device. Program code embodied on a computer readable medium can be transmitted using any suitable medium, including but not limited to, wire, cable, wireless, RF, infrared or any suitable combination of the foregoing.

[0074] In some embodiments, the client, server, or both can communicate using any current known or future developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include local area networks ("LAN"), wide area networks ("WAN"), the Internet, and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any current known or future developed networks.

[0075] The computer readable medium can be included in the camera device, or can exist separately from the camera device. The computer readable medium carries one or more programs that, when executed by the camera device, cause the camera device to: in response to detecting that the camera device starts exposure, determine a synchronization signal generation time of the camera device according to the updated exposure time and the exposure start time, so that the camera device sends a synchronization signal to the master control unit at the synchronization signal generation time, wherein the camera device and the inertial measurement device are both connected to the master control unit; in response to determining that the master control unit receives the synchronization signal, determine a time at which the master control unit receives the synchronization signal as an image time; perform association processing on an image corresponding to the synchronization signal collected by the camera device and the image time, to obtain an associated image; in response to determining that the master control unit receives the inertial measurement data collected by the inertial measurement device, determine a time at which the master control unit receives the inertial measurement data as an inertial measurement time; and perform association processing on the inertial measurement data and the inertial measurement time, to obtain associated inertial measurement data.

[0076] Computer program code for carrying out operations of some embodiments of the disclosure can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0077] The computer program product of the first aspect can further include one or more of the following features. The computer program product can include a computer readable medium. The computer readable medium can include a computer readable storage medium or a computer readable signal medium. The computer readable storage medium can include tangible storage medium. The computer readable signal medium can include a propagated data signal with computer readable program code embodied therein. The computer readable program code can be downloaded into a working memory of a computer from the computer readable signal medium or from the computer readable storage medium. The computer readable program code can cause the computer to perform the steps of the first aspect. The computer readable program code can be executed by one or more processors associated with the computer.

[0078] The functions described above can be performed, at least in part, by one or more hardware logic components. For example, and without limitation, non-limiting examples of hardware logic components that can be used include field programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SOCs), complex programmable logic devices (CPLDs), etc.

[0079] The above description is merely exemplary of the disclosure and the application of the principles thereof. It is not intended to limit the scope of the disclosure to the precise forms disclosed. The disclosure is to cover modifications, equivalents, and alternatives falling within the scope of the disclosure as defined by the claims. For example, the features to be mutually substituted for one another are not limited to features disclosed in the embodiments of the disclosure but can include features with similar functions to those disclosed in the embodiments of the disclosure.

Claims

1. A data generation method applied to a camera device, the camera device comprising a camera, an inertial measurement device and a master control unit, the method comprising: in response to detecting that the camera starts exposure, determining a synchronization signal generation time of the camera according to an updated exposure time and an exposure start time, so that the camera sends a synchronization signal to the master control unit at the synchronization signal generation time, wherein the camera and the inertial measurement device are connected to the master control unit; in response to determining that the master control unit receives the synchronization signal, determining a time when the master control unit receives the synchronization signal as an image time; associating an image corresponding to the synchronization signal collected by the camera with the image time to obtain an associated image; in response to determining that the master control unit receives inertial measurement data collected by the inertial measurement device, determining a time when the master control unit receives the inertial measurement data as an inertial measurement time; associating the inertial measurement data with the inertial measurement time to obtain associated inertial measurement data; for each of the associated inertial measurement data, determining associated inertial measurement data in which a corresponding inertial measurement time is the same as an image time corresponding to the associated image as synchronization inertial measurement data, wherein a frame rate of the inertial measurement device for collecting inertial measurement data is greater than a frame rate of the camera for collecting images; synchronizing the associated image with the synchronization inertial measurement data to obtain synchronization data; and determining positioning information of the camera device in a three-dimensional space according to the synchronization data, wherein the three-dimensional space is a three-dimensional point cloud space, and the positioning information is a positioning position determined by a SLAM algorithm according to a constructed real-time environment map. The method further comprises: in response to determining that the exposure type is global exposure, determining the exposure time as a total exposure time. The method further comprises: in response to determining that the exposure type is row exposure, determining a preset transmission time and the exposure time as a total exposure time; and generating an intermediate exposure time according to the determined total exposure time. The method further comprises: in response to determining that the exposure type is global exposure, determining the exposure time as a total exposure time. ​ ​ ​ ​ ​ 2. The method of claim 1, wherein, ​ ​ 3. The method of claim 2, wherein, ​ ​ ​ 4. The method of claim 3, wherein, ​ ​ ​ 5. The method of claim 4, wherein, ​ ​ 6. A camera device comprising a camera, an inertial measurement device and a master unit, wherein the camera is a camera head, and wherein the camera is connected to the master unit, the camera is configured to generate a synchronization signal and to send the generated synchronization signal to the master unit at a synchronization signal generation time, and to send an image captured to the master unit; the inertial measurement device is connected to the master unit, the inertial measurement device is configured to capture inertial measurement data and to send the inertial measurement data to the master unit; the master unit is configured to implement the method according to any one of claims 1 to 5.

7. A head-mounted display device comprising a head-mounted display device body and a camera device according to claim 6.

8. A computer readable medium having stored thereon a computer program, wherein, the computer program, when executed by a processor, implements the method according to any one of claims 1 to 5.

Images