Image shooting control method and electronic device

Abstract

The present application discloses an image shooting control method and electronic device, belonging to the field of image technology. The method includes: a control component controls a signal generator to generate at least one set of start-up exposure signals, and synchronously sends at least one set of start-up exposure signals to at least two target image acquisition modules connected to the signal generator, and an image processing component obtains at least one set of start-up images captured by each target image acquisition module based on at least one start-up exposure signal. Then, when at least one set of start-up images captured by each target image acquisition module is not empty, the control component controls the signal generator to generate a target exposure signal, and synchronously sends the target exposure signal to each target image acquisition module, and the image processing component obtains the target image captured by each target image acquisition module based on the target exposure signal; the exposure frequency of the target exposure signal is different from the exposure frequency of the start-up exposure signal.

Description

Technical Field

[0001] The present application belongs to the field of image technology, and specifically relates to an image shooting control method and electronic equipment. Background Art

[0002] Currently, many algorithms rely on keypoint data from images. To provide this data, different image acquisition modules are often used to capture the same scene, obtaining target images captured by each module. These target images can reflect different perspectives and can be subsequently merged.

[0003] In order to ensure that the target images captured by different image acquisition modules can be aligned as much as possible when merging, how to keep the images captured by different image acquisition modules synchronized in time has become a technical problem that needs to be solved urgently. Summary of the Invention

[0004] The purpose of the embodiments of the present application is to provide an image capture control method that can solve the problem of how to keep images captured by different image acquisition modules synchronized in time.

[0005] In a first aspect, an embodiment of the present application provides an image capture control method, which is applied to an image synchronization device, wherein the image synchronization device includes a control component and an image processing component; the method includes:

[0006] The control component controls the signal generator to generate at least one set of start-up exposure signals, and synchronously sends the at least one set of start-up exposure signals to at least two target image acquisition modules connected to the signal generator, and the image processing component obtains at least one set of start-up images captured by each of the target image acquisition modules based on the at least one start-up exposure signal;

[0007] When at least one group of startup images captured by each of the target image acquisition modules is not empty, the control component controls the signal generator to generate a target exposure signal, and synchronously sends the target exposure signal to each of the target image acquisition modules, and the image processing component obtains the target image captured by each of the target image acquisition modules based on the target exposure signal; the exposure frequency of the target exposure signal is different from the exposure frequency of the startup exposure signal.

[0008] In a second aspect, an embodiment of the present application provides an image capture control device, which is applied to an image synchronization device. The image synchronization device includes a control component and an image processing component. The device includes:

[0009] a first control module located in the control component, configured to control the signal generator to generate at least one set of start-up exposure signals and synchronously send the at least one set of start-up exposure signals to at least two target image acquisition modules connected to the signal generator; and a first acquisition module located in the image processing component, configured to acquire at least one set of start-up images captured by each of the target image acquisition modules based on the at least one start-up exposure signal;

[0010] The second control module located in the control component is used to control the signal generator to generate a target exposure signal and synchronously send the target exposure signal to each target image acquisition module when at least one group of startup images captured by each target image acquisition module is not empty. The second acquisition module located in the image processing component is used to acquire the target image captured by each target image acquisition module based on the target exposure signal; the exposure frequency of the target exposure signal is different from the exposure frequency of the startup exposure signal.

[0011] In a third aspect, an embodiment of the present application provides an electronic device comprising a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the programs or instructions are executed by the processor, the steps of the image capture control method described in the first aspect are implemented.

[0012] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the image capture control method as described in the first aspect are implemented.

[0013] In a fifth aspect, an embodiment of the present application provides a chip, which includes a processor and a communication interface, the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the image shooting control method as described in the first aspect.

[0014] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the image shooting control method as described in the first aspect.

[0015] In an embodiment of the present application, the control component controls the signal generator to generate at least one set of start-up exposure signals, and synchronously sends at least one set of start-up exposure signals to at least two target image acquisition modules connected to the signal generator, and the image processing component obtains at least one set of start-up images captured by each target image acquisition module based on at least one start-up exposure signal. When at least one set of start-up images captured by each target image acquisition module is not empty, the control component controls the signal generator to generate a target exposure signal, and synchronously sends the target exposure signal to each target image acquisition module, and the image processing component obtains the target images captured by each target image acquisition module based on the target exposure signal; the exposure frequency of the target exposure signal is different from the exposure frequency of the start-up exposure signal. In this way, by adopting a hardware synchronization method, a signal generator is used to synchronously send exposure signals to multiple target image acquisition modules, controlling the target image acquisition modules to be exposed simultaneously, thereby ensuring, to a certain extent, that the images captured by different target image acquisition modules remain synchronized in time. At the same time, before multiple target image acquisition modules start to formally acquire target images, a start exposure signal is first sent to the target image acquisition module, the target image acquisition module is controlled to capture the start image first, and then the target image acquisition module is controlled to capture the required target image, leaving time for the target image acquisition module to open synchronously. Accordingly, when at least one group of start images captured by each target image acquisition module is not empty, that is, when each target image acquisition module is opened, the control signal generator generates a target exposure signal, which can ensure that multiple target image acquisition modules have completed opening when capturing target images, and further ensure that multiple target image acquisition modules start capturing target images at the same time, avoiding the problem of differences in the number of target images captured due to inconsistent opening times of different target image acquisition modules, and further ensuring the time synchronization between target images. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a flowchart of the steps of the image capture control method provided in an embodiment of the present application;

[0017] Figure 2 is a partial schematic diagram of an image sequence provided by an embodiment of the present invention;

[0018] Figure 3 This is a flowchart of a startup image detection provided by an embodiment of the present application;

[0019] Figure 4 is a schematic diagram of an image capture system provided in an embodiment of the present application;

[0020] Figure 5 This is a flowchart of image sequence merging provided by an embodiment of the present application;

[0021] Figure 6is a block diagram of an image capture control device provided in an embodiment of the present application;

[0022] Figure 7 This is one of the structural diagrams of the electronic device provided in the embodiment of the present application;

[0023] Figure 8 This is the second structural diagram of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0024] The following will be combined with the accompanying drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the embodiments described are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field are within the scope of protection of this application.

[0025] The terms "first," "second," and the like in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of this application can be implemented in an order other than that illustrated or described herein, and that the objects distinguished by "first," "second," and the like are generally of the same type, and do not limit the number of objects; for example, the first object can be one or more. In addition, the term "and / or" in the specification and claims refers to at least one of the connected objects, and the character " / " generally indicates that the objects connected are in an "or" relationship.

[0026] The embodiment of the present application provides an image shooting control method, which can be used to control at least two target image acquisition modules to capture images. The target image acquisition module can be an image acquisition module that supports hardware synchronization, and the image acquisition module can be a device for capturing images, such as a camera, an image sensor, and the like. The target image acquisition module can be connected to the signal generator through a signal line. For example, the signal line can be set as needed, for example, a signal line of a 3.5 mm (millimeter, mm) port or an audio and video (Audio Video, AV) synchronization port is selected, and the embodiment of the present application does not limit this. The signal generator sends an exposure signal to each target image acquisition module synchronously to control each target image acquisition module to capture simultaneously, thereby making the images captured by different target image acquisition modules synchronized in time. In actual application scenarios, multiple image acquisition modules can be set for image capture, and the target image acquisition module can be all image acquisition modules, or it can also be part of the image acquisition module, and the embodiment of the present application does not limit this.

[0027] In one application scenario, the target image acquisition module can be used to capture the hand. The resulting target image can reflect the specific hand gesture, and the gesture key points in the target image provide data support for the gesture recognition algorithm. A traditional approach typically uses a multi-view camera array for acquisition. A monocular camera with multiple viewpoints exposes the hand position at different angles, obtaining images of the hand from different viewpoints at the same time. Key points are then estimated using a neural network, and outliers are estimated and removed using the Random Sample Consensus (RANSAC) algorithm. Outliers are defined as outliers among the estimated key points. RANSAC is applicable to datasets containing outliers and can be used to solve object recognition and pose estimation problems in computer vision. For example, RANSAC randomly selects a sample of the estimated key points through an iterative process and uses these samples to fit a model. This model is then used to test other key points to determine whether they are consistent with the model. If they are, the key point is determined to be an "inlier" and retained. Conversely, if they are inconsistent, the keypoint is determined to be an "outlier" and can be removed. In this method, the estimated keypoints contain camera plane coordinates and relative depth information. However, the minimum component of a multi-view is a monocular image, which lacks depth information. Since the relative depth information of keypoints is estimated using a neural network, the depth estimation accuracy is low, meaning the estimated keypoints are not accurate. Furthermore, since the keypoints are estimated using a neural network, the accuracy of the resulting keypoint data cannot be guaranteed.

[0028] In an embodiment of the present application, the target image acquisition module may include a camera equipped with a motion capture device and a camera equipped on extended reality (Extended Reality, XR) glasses. Among them, the motion capture device has the characteristics of high precision and low latency, and is therefore widely used in the fields of virtual reality, motion analysis, biomechanics research, etc. The motion capture device can track multiple markers in real time, thereby providing accurate position and motion data. Specifically, the motion capture device can be used to track the movement of a human body or object in three-dimensional space. The motion capture device may include a camera array composed of multiple cameras and a special passive reflective marker (for example, a spherical marker). Among them, the camera array of the motion capture device can be synchronized at the millisecond level, and the reflective marker can reflect the infrared light emitted by the camera when moving. The camera system can determine the three-dimensional position of the marker by calculating the position of the reflected light. Accordingly, there is no need to perform key point estimation. The target image obtained based on the camera shooting in the motion capture device can carry key point information, so that the accuracy and precision of the key points can be ensured. Among them, the target image can be the image required to identify the key points, that is, the image actually required.

[0029] Furthermore, the target images captured by the motion capture device and the target images captured by the XR glasses can be aligned and merged. That is, the target image captured by the motion capture device and the target image captured by the XR glasses at the same moment are merged to associate the target images at the same moment. This merged image can then be used to train or evaluate the gesture recognition algorithm. For example, after merging, at least two target images corresponding to the same moment can be used. The target image captured by the XR glasses can be used as input to the gesture recognition algorithm to obtain key point information output by the gesture recognition algorithm. The key point information output by the gesture recognition algorithm is then compared with the key point information in the associated target image captured by the motion capture device to evaluate the accuracy of the gesture recognition algorithm. By coordinating the motion capture device and XR glasses to capture the position of reflected light, the motion capture device obtains a target image containing the spatial position information of the gesture key points in the real scene. Subsequently, coordinate transformation can be performed to obtain the positions of the gesture key points in the XR glasses coordinate system, thereby obtaining high-precision ground truth data.

[0030] Of course, the embodiments of the present application are not limited to the scenario of gesture image acquisition. For example, the embodiments of the present application can also be applied to human posture image acquisition, facial image acquisition, etc. The target image acquisition module is also not limited to motion capture devices and XR glasses. For example, the target image acquisition module can include a camera equipped with XR glasses and a camera equipped with any set of data acquisition equipment, or can also be applied to cameras of several modalities such as grayscale cameras and red, green, and blue (RGB) cameras, RGB cameras and depth cameras, and depth cameras and structured light cameras.

[0031] Furthermore, in a soft synchronization approach, the images captured by multiple image acquisition modules are sorted into time-synchronized images based on the system timestamps of the images arriving at the storage computer. If a soft synchronization approach is used, time differences will inevitably occur between multiple target image acquisition modules due to various factors, such as different exposure times of the image acquisition modules and transmission delays. For example, software synchronization between multiple image acquisition modules often results in a time error of at least 2-5 milliseconds (ms). The image capture control method provided by the embodiments of the present application will be described in detail below with reference to specific embodiments and their application scenarios, in conjunction with the accompanying drawings.

[0032] Figure 1 is a flowchart of the steps of the image shooting control method provided by the embodiment of the present application. The method can be applied to an image synchronization device, which includes a control component and an image processing component, such as Figure 1 As shown, the method includes:

[0033] In step 101, the control component controls the signal generator to generate at least one set of start exposure signals, and synchronously sends the at least one set of start exposure signals to at least two target image acquisition modules connected to the signal generator, and the image processing component obtains at least one set of start images taken by each of the target image acquisition modules based on the at least one start exposure signal.

[0034] In the embodiment of the present application, the signal generator can be a synchronization signal generator. The specific model of the signal generator can be selected according to actual needs, and the embodiment of the present application does not limit this. The synchronization signal generator can be used as the main device for sending exposure signals, and each target image acquisition module is connected to each other through a connecting line. For example, the synchronization signal generator can be connected to each camera of the motion capture device and the XR glasses device through a synchronization signal line, and send exposure signals to the connected devices at the same time to control the simultaneous exposure of the target image acquisition modules. Based on the signal generator, a hardware synchronization method is adopted. Compared with the soft synchronization method, hard synchronization is achieved through direct physical connection and hardware signals. The propagation and processing of hardware signals are usually more stable and the transmission delay is smaller. Therefore, the time synchronization degree of images taken by different target image acquisition modules can be improved, and the time error can be reduced. For example, the use of hardware synchronization between multiple image acquisition modules can often control the time error to at least 1ms.

[0035] The target image acquisition module can capture an image in response to an exposure signal. The exposure signal can be a signal used to trigger the target image acquisition module to capture an image. The start exposure signal can be exposure information, specifically used to trigger the target image acquisition module to capture a start image. The exposure signal can be generated and sent simultaneously. For example, after generating a portion of the exposure signal, the sending operation is executed, while continuing to generate subsequent exposure signals.

[0036] The startup image can be an image captured to synchronize the startup of different target image acquisition modules. Since the target image acquisition modules may have different opening times, even if exposure information is sent to each target image acquisition module synchronously, the target image acquisition modules may have different exposure frames, that is, the number of captured images may be different. For example, assuming that the opening times of target image acquisition module 1 and target image acquisition module 2 differ by 20ms, in the same time frame, the exposure frames of target image acquisition module 1 may be 12, and the exposure frames of target image acquisition module 2 may be 20. If the target image acquisition module is controlled to directly start capturing the target images that are actually required for subsequent merging, if there is a difference in the number of target images, it will be impossible to align the target images. If merging is performed directly, it will cause synchronization problems when synthesizing images. For example, if there is a difference in the number of target images, the image sequences captured by different target image acquisition modules cannot be strictly aligned, resulting in time misalignment and time synchronization problems.

[0037] In this step, the control component first controls the signal generator to send a start-up exposure signal, controlling the target image acquisition module to first capture the start-up image. This is equivalent to reserving time for the different target image acquisition modules to start synchronously before officially capturing the desired target image. During the capture of the start-up image, the target image acquisition module that opens slower can fully open. To a certain extent, this ensures that each target image acquisition module is fully opened when the subsequent capture of the desired target image begins. In this way, to a certain extent, even if some image acquisition modules fail to open in time, they will not fully respond to all start-up exposure signals, resulting in the number of images captured being less than the number of start-up exposure signals. The incomplete start-up image will not affect the subsequent capture of the desired target image, ensuring that the number of target images captured by each target image acquisition module is consistent as much as possible.

[0038] Step 102: When at least one set of startup images captured by each of the target image acquisition modules is not empty, the control component controls the signal generator to generate a target exposure signal, and synchronously sends the target exposure signal to each of the target image acquisition modules, and the image processing component obtains the target image captured by each of the target image acquisition modules based on the target exposure signal; the exposure frequency of the target exposure signal is different from the exposure frequency of the startup exposure signal.

[0039] In an embodiment of the present application, it can be determined that at least one group of startup images has been captured after at least one group of startup exposure signals has been sent. If at least one group of startup images captured by the target image acquisition module is not empty, it can be said that each target image acquisition module has captured the startup image, that is, each target image acquisition module has been turned on. Accordingly, the target exposure signal can continue to be generated and sent synchronously to each target image acquisition module so that each target image acquisition module starts to capture the target image at the same time. Since each target image acquisition module has been fully opened after the process of capturing the startup image, by continuing to send the target exposure signal after at least one group of startup images has been captured, it can be ensured to a certain extent that each target image acquisition module can respond to the target exposure signal in a timely manner and take pictures.

[0040] Among them, the target exposure signal can be specifically used to trigger the target image acquisition module to capture the required target image. The exposure frequency of the target exposure signal can be selected between the minimum exposure frequency and the maximum exposure frequency of all target image acquisition modules. For example, assuming that the target image acquisition module includes image acquisition module A and image acquisition module B, wherein the minimum exposure frequency-maximum exposure frequency of image acquisition module A is: 10Hz-59Hz, and the minimum exposure frequency-maximum exposure frequency of image acquisition module B is: 10Hz-40Hz, then the target exposure frequency can be selected from 10Hz-40Hz. The target exposure frequency refers to the exposure frequency of the target exposure signal.

[0041] Because the exposure frequency of the target exposure signal differs from the exposure frequency of the start-up exposure signal, the time interval between start-up images and the time interval between target images in the entire image sequence ultimately captured by the target image acquisition module differ. This facilitates distinguishing start-up images from target images, further facilitating subsequent processing of the desired target images.

[0042] In an embodiment of the present application, the control component may be located in the signal generator, that is, the control component may be a part of the signal generator. Alternatively, the control component and the signal generator may be two independent devices. The control component may control the signal generator to start up according to a pre-set configuration signal or in response to a user operation, and then perform the operation of generating and sending a start exposure signal. Exemplarily, the control component may communicate with the signal generator to send a first control instruction to the signal generator. Accordingly, the signal generator may perform the operation of generating and sending a start exposure signal in response to the first control instruction. When at least one set of start images captured by each target image acquisition module is not empty, the control component sends a second control instruction. Accordingly, the signal generator may perform the operation of generating and sending a target exposure signal in response to the second control instruction.

[0043] The image processing component and the control component can be two independent devices, that is, the image synchronization device includes multiple independent devices. Alternatively, the image processing component and the control component can be different components of the same device, that is, the image synchronization device is a single device. The image processing component can perform the acquisition operation after each target image acquisition module captures a portion of the image, that is, it can acquire the image while capturing. Alternatively, the startup image can be acquired after each target image acquisition module captures the startup image. The target image can be acquired after each target image acquisition module captures the target image. Furthermore, each target image acquisition module can actively return the captured image to the image processing component. Accordingly, the image processing component can obtain the startup image or target image captured by each image processing component by receiving the image returned by each target image acquisition module. Alternatively, the images captured by each image processing component can be stored in a preset storage device, and then a connection can be established between the preset storage device and the image processing component. Accordingly, the image processing component can obtain the startup image or target image from the preset storage device to obtain the startup image or target image captured by each image processing component.

[0044] Furthermore, in an embodiment of the present application, the image processing component can detect at least one set of startup images captured by each target image acquisition module. If it is detected that each target image acquisition module has captured at least one startup image, for example, if it is detected that the startup image captured by each target image acquisition module has been successfully acquired, it is determined that the at least one set of startup images captured by each target image acquisition module is not empty. Accordingly, the image processing component can send a prompt message to the control component indicating that the at least one set of startup images captured by each target image acquisition module is not empty, so that the control component controls the signal generator to generate and send the target exposure signal. Alternatively, the image processing component can output a prompt message to the user, and accordingly, upon receiving the prompt message, the user can control the control component to execute the operation of controlling the signal generator to generate and send the target exposure signal. Alternatively, the control component can control the signal generator to generate and send a preset number of startup exposure signals or send a startup exposure signal of a preset duration. Accordingly, after the preset number of start-up exposure signals or the preset duration of start-up exposure signals have been sent, sufficient start-up time has been reserved for each target image acquisition module. Therefore, it can be determined that the condition that at least one set of start-up images captured by each target image acquisition module is not empty is currently met. Accordingly, the control component can control the signal generator to generate and send the target exposure signal.

[0045] In an embodiment of the present application, the control component controls the signal generator to generate at least one set of start-up exposure signals, and synchronously sends at least one set of start-up exposure signals to at least two target image acquisition modules connected to the signal generator, and the image processing component obtains at least one set of start-up images captured by each target image acquisition module based on at least one start-up exposure signal. When at least one set of start-up images captured by each target image acquisition module is not empty, the control component controls the signal generator to generate a target exposure signal, and synchronously sends the target exposure signal to each target image acquisition module, and the image processing component obtains the target images captured by each target image acquisition module based on the target exposure signal; the exposure frequency of the target exposure signal is different from the exposure frequency of the start-up exposure signal. In this way, by adopting a hardware synchronization method, a signal generator is used to synchronously send exposure signals to multiple target image acquisition modules, controlling the target image acquisition modules to be exposed simultaneously, thereby ensuring, to a certain extent, that the images captured by different target image acquisition modules remain synchronized in time. At the same time, before multiple target image acquisition modules start to formally acquire target images, a start exposure signal is first sent to the target image acquisition module, the target image acquisition module is controlled to capture the start image first, and then the target image acquisition module is controlled to capture the required target image, leaving time for the target image acquisition module to open synchronously. Accordingly, when at least one group of start images captured by each target image acquisition module is not empty, that is, when each target image acquisition module is opened, the control signal generator generates a target exposure signal, which can ensure that multiple target image acquisition modules have completed opening when capturing target images, and further ensure that multiple target image acquisition modules start capturing target images at the same time, avoiding the problem of differences in the number of target images captured due to inconsistent opening times of different target image acquisition modules, and further ensuring the time synchronization between target images.

[0046] In some embodiments, the at least one set of start-up exposure signals includes M first trigger signals, each of which is used to instruct the target image acquisition module to capture one of the start-up images. M is a positive integer greater than a preset threshold, and M is positively correlated with a turn-on delay between the at least two target image acquisition modules.

[0047] In an embodiment of the present application, the first trigger signal can be a rising edge signal, that is, the first trigger signal can be a high-level signal. Accordingly, the target image acquisition module can capture a startup image in response to a high-level signal. The preset number threshold can be pre-set according to actual conditions. For example, the preset number threshold can be set based on the maximum opening time required by the target image acquisition module. For example, the larger the maximum opening time, the larger the preset number threshold can be. The preset number threshold can be greater than the number of images that can be captured during the maximum opening time. In this way, at least one group of startup exposure signals is set to include M first trigger signals greater than the preset number threshold, ensuring that each target image acquisition module is opened after responding to all startup exposure signals to capture the startup image. When actually setting M, the opening delay between the target image acquisition modules can be further combined. Specifically, the larger the opening delay, the larger the number of M can be. In this way, it can be ensured to a greater extent that each target image acquisition module can be fully opened during the process of capturing the startup image.

[0048] In some embodiments, different groups of start-exposure signals have different exposure frequencies. For any group of start-exposure signals, the start-exposure signal is a first continuous pulse signal including a first trigger signal, and the time interval between the first trigger signals matches the exposure frequency of the start-exposure signal. Accordingly, synchronously sending the at least one group of start-exposure signals to at least two target image acquisition modules connected to the signal generator includes: step 1011: for any group of start-exposure signals, simultaneously sending the first continuous pulse signal to each target image acquisition module via a signal line between the first trigger signal and the target image acquisition module.

[0049] In an embodiment of the present application, the trigger signal can be a rising edge signal, that is, a high-level signal, i.e., a pulse signal that does not carry timestamp information. The exposure frequency of each group of start-up exposure signals can be dynamically set as needed, as long as the exposure frequency of each group of start-up exposure signals is different and different from the exposure frequency of the target exposure signal. By ensuring that the exposure frequency of each group of start-up exposure signals is different, it is convenient to subsequently distinguish different groups of start-up images. For example, taking the example of at least one group of start-up exposure signals specifically including three groups of start-up exposure signals, where the exposure frequencies of these three groups of start-up exposure signals are m1, m2, and m3, respectively, and the exposure frequency of the target exposure signal is k, the target exposure frequency k used for synchronously capturing the target image can be first set, and then three groups of start-up exposure signals with frequencies m1, m2, and m3, respectively, can be sequentially issued. The specific number of first trigger signals included in each group of start-up exposure signals can be dynamically set as needed, and the number of first trigger signals included in each group of start-up exposure signals can be the same or different.

[0050] Specifically, the number of first trigger signals included in each group of start-up exposure signals may be positively correlated with the number of target image acquisition modules. The number of target image acquisition modules may be set according to actual needs, and the embodiments of the present application do not limit this. Specifically, when the number of target image acquisition modules is greater, the noise in the image synchronization acquisition system is greater, and a longer time may be required to fully open different target image acquisition modules. Therefore, the number of first trigger signals may be greater. Exemplarily, when the number of target image acquisition modules is greater than a preset module number threshold, for example, greater than 30, the number of first trigger signals may be increased, for example, set to 12 or 20. When the number of target image acquisition modules is not greater than the preset module number threshold, the number of first trigger signals may be reduced, for example, set to 8 or 7. Furthermore, the number of first trigger signals may also be related to other noise factors. For example, when the stability of the power supply voltage is less than a preset stability threshold, more first trigger signals are set.

[0051] Exemplarily, the number of first signals included in these three groups of start exposure signals may be 10, 10, and 12, respectively. In an embodiment of the present application, by generating a continuous pulse signal including multiple first trigger signals as a start exposure signal, the target image acquisition module can be controlled to continuously perform exposure shooting, thereby ensuring image shooting efficiency to a certain extent. Among them, the time interval between the first trigger signals in the start exposure signal matches the exposure frequency of the start exposure signal, which may refer to: the time interval between the first trigger signals in the start exposure signal can be calculated based on the exposure frequency of the start exposure signal. Specifically, the exposure frequency may refer to the number of image frames captured in a unit time. Accordingly, in milliseconds, for any group of start exposure signals, the time interval between the first trigger signals in the group of start exposure signals may be 1000ms / exposure frequency of the start exposure signal.

[0052] When the signal generator generates at least one set of start-exposure signals, it can generate the at least one set of start-exposure signals according to the set of at least one set of start-exposure frequencies and first signal quantities. For example, a rising edge is used as the first trigger signal, and the set of start-exposure frequencies and first signal quantities are expressed in the format of A@B, where A represents the first signal quantity and B represents the start-exposure frequency. Assuming that the at least one set of start-exposure frequencies and trigger signal quantities includes: 10@11Hz, 10@122Hz, and 12@66Hz, the signal generator can sequentially generate a continuous pulse signal consisting of 10 rising edges with a time interval of 1000 / 11 between each rising edge, and synchronously send it to each target image acquisition module via a signal line. Next, the signal generator can generate a continuous pulse signal consisting of 10 rising edges with a time interval of 1000 / 122 between each rising edge, and synchronously send it to each target image acquisition module via a signal line. Finally, the signal generator can generate a continuous pulse signal consisting of 12 rising edges with a time interval of 1000 / 66 between each rising edge, and synchronously send it to each target image acquisition module via a signal line, as a synchronization protocol to initiate system acquisition.

[0053] It should be noted that after all the start exposure signals are sent, the target exposure signal can be sent according to the target exposure frequency. The number of trigger signals included in the target exposure signal can be set as needed, and the embodiment of the present application does not limit this. For example, a rising edge pulse signal can be continuously sent to each target image acquisition module at the same time according to the target time interval until the sending duration reaches the preset acquisition duration. The target time interval corresponds to the target exposure frequency, the target time interval can be 1000 / k, and the preset acquisition duration can be set as needed, for example, the preset acquisition duration can be 10 minutes.

[0054] In an embodiment of the present application, a first continuous pulse signal is sent through a signal line between each target image acquisition module. The signal line serves as a physical connection, and the propagation delay is small. Transmission through the signal line can ensure to a greater extent that each target image acquisition module receives the start exposure signal at the same time, and the synchronization accuracy is high.

[0055] In some embodiments, the image sequence captured by each target image acquisition module includes the at least one set of start-up images and the target image. In the embodiment of the present application, the following may also be included:

[0056] In step S21, the image processing component detects the first number of images in the image sequence corresponding to each of the target image acquisition modules; the image sequence corresponding to the target image acquisition module includes the target images captured by the target image acquisition module, and the first number of images is the number of target images included in the image sequence.

[0057] Step S22 : When the number of first images in each of the image sequences is the same, the image processing component automatically merges the target images in the image sequences of the at least two target image acquisition modules.

[0058] In an embodiment of the present application, after the target exposure signal is sent, the image processing component can perform image processing based on the image sequence corresponding to each target image acquisition module. Specifically, the number of target images included in each image sequence can be detected as the first image number. If the number of first images in each image sequence is the same, then to a certain extent, it can be explained that after the stage of shooting the start image, when the first target exposure signal is sent, each target image acquisition module has been turned on and each target image acquisition module starts shooting the target image at the same time, and because the number of target images captured by each target image acquisition module is the same. Therefore, to a certain extent, it can be explained that the target images in the target image sequences captured by different target image acquisition modules are not misaligned, and in the multiple target image sequences captured by each target image acquisition module according to the target exposure frequency, the timestamps corresponding to the target images in the same order are consistent and synchronized in time. Therefore, when the number of first images in each image sequence is consistent, the target images in the image sequences corresponding to at least two target image acquisition modules can be automatically merged.

[0059] Specifically, the target images corresponding to the same timestamp in all image sequences can be identified, and the target images corresponding to the same timestamp can be associated and stored, thereby achieving automatic merging. For example, the first frame target image in the target image sequence can be identified as 0, and the image identifiers of subsequent frames can be superimposed in sequence. For example, the identifiers can be 1, 2, 3... Since different target image sequences are shot synchronously, the target images with the same identifier in different target image sequences are the target images corresponding to the same timestamp. Therefore, the target images with the same number can be associated and stored to complete the alignment according to the frame identifier. For example, when storing, the timestamp can be used as the index, and an image folder named after the timestamp can be created, and all target images corresponding to the timestamp can be stored in the image folder.

[0060] In some embodiments, the image sequence further includes at least one set of startup images captured by the target image acquisition module. In the embodiment of the present application, the following may also be included:

[0061] In step S31, the image processing component detects whether the number of second images in each image sequence is consistent with the number of first signals; the number of second images is the number of images included in the last set of start images taken in the image sequence, and the number of first signals is the number of first trigger signals included in the last set of start exposure signals sent.

[0062] The detecting, by the image processing component, the number of first images of the image sequence corresponding to each of the target image acquisition modules may include: step S221, the image processing component detecting the number of first images of each of the image sequences when the number of the second images is consistent with the number of the first signals.

[0063] In an embodiment of the present application, the last frame of all the startup images can be searched in reverse order of the sending order of the startup exposure signal. Accordingly, the number of second images can be determined by counting down from the last frame. Whether the last set of startup images shot is complete is determined by comparing whether the number of second images is consistent with the number of first signals. Due to differences in the opening time of different target image acquisition modules, the received startup exposure signal may be incomplete. For example, the signal generator sends two groups of startup exposure signals, and the target image acquisition module may only receive partial exposure information in the first group and a complete second group of startup exposure signals. Accordingly, in an embodiment of the present application, only the number of images included in the last group of startup images is compared with the number of first trigger signals included in the startup exposure signal of the corresponding group. When the two numbers are consistent, it is determined that the startup image is complete, which reduces the difficulty of comparison to a certain extent and improves the rationality of the comparison.

[0064] If the number of images included in the last set of startup images captured is the same as the number of first trigger signals included in the last set of startup exposure signals sent, that is, the number of second images in each image sequence is consistent with the number of first signals, then it can be determined that when the last set of startup images was captured, all target image acquisition modules had completed startup and all target image acquisition modules were capturing synchronously. In the case of inconsistent startup times, the number of first-set startup images captured by different target image acquisition modules may vary. For example, the number of first-set startup images captured by some target image acquisition modules may be less than the number of first trigger signals included in the first set of startup exposure signals sent. If the number of second images is consistent with the number of first signals, it means that over time, when the last set of startup images was captured, all target image acquisition modules had completed startup and were able to capture synchronously. Therefore, if the number of second images is consistent with the number of first signals, it can be more certain that each target image acquisition module began capturing target images at the same time and maintained synchronization during capture, indicating that the number of target images in the image sequence captured by each target image acquisition module is more likely to be consistent. In this case, the number of first images in each image sequence is detected to determine whether the number of target images captured by different target image acquisition modules is consistent. This can avoid unnecessary execution of the operations of detecting the number of first images and determining whether the number of first images is consistent, thereby saving processing resources to a certain extent.

[0065] It should be noted that, in the embodiment of the present invention, the order of sending the start-up exposure signal can be reversed. When the number of the second image is consistent with the number of the first signal, it is detected whether the other groups of start-up images are complete, that is, whether the number of each subsequent group of start-up images is consistent with the number of first trigger signals included in the start-up exposure signal of the corresponding group, so as to determine the opening order of different target image acquisition modules. Subsequently, the target image acquisition module can be optimized based on the opening order of different target image acquisition modules to control the simultaneous opening of different target image acquisition modules as much as possible. For example, each group of start-up images in the image sequence taken by the A image acquisition module is complete, only the third group of start-up images in the image sequence taken by the B image acquisition module are complete, and the second and third groups of start-up images in the image sequence taken by the C image acquisition module are complete, then the opening order can be determined to be: A image acquisition module-C image acquisition module-B image acquisition module.

[0066] In some embodiments, the present application may further include:

[0067] Step S41: For any of the image sequences, the image processing component determines an image whose time interval with the previous frame image is a first time interval and whose time interval with the next frame image is a second time interval as an identification image; the first time interval matches the exposure frequency of the last set of start exposure signals sent, and the second time interval matches the exposure frequency of the target exposure signal.

[0068] Step S42: The image processing component counts the total number of the identification image and the associated images of the identification image to obtain the first number of images in the image sequence; the associated images include images that are located before the identification image and have a time interval with the next frame image that is the first time interval.

[0069] In an embodiment of the present application, for any image in an image sequence, the image processing component can calculate the time interval between the image and the previous frame based on the timestamp of the previous frame and the timestamp of the current image. The time interval between the image and the next frame can be calculated based on the timestamp of the current image and the timestamp of the next frame. The identification image refers to the startup image captured by the last frame. For the startup image captured by the last frame, the previous frame of the image is the startup image, and the next frame of the image is the target image. Since the target exposure frequency is different from the startup exposure frequency, they can be distinguished based on the time interval.

[0070] Specifically, the image processing component can start from the middle position of the image sequence and traverse forward. For any frame image traversed, it can detect whether the time interval between the image and the previous frame image is the first time interval, and whether the time interval between the image and the next frame image is the second time interval. If so, it can be determined that the currently traversed image is the startup image taken in the last frame, and the image can be recorded as the identification image. Then, starting from the identification image, the forward search can be continued to select the image that is located before the identification image and has a time interval with the next frame image of the first time interval as the associated image, that is, to search for the same group of images as the identification image. The total number of identification images and the associated images of the identification image is the number of the last group of startup images taken in the image sequence.

[0071] Furthermore, the frame image after the identification image can be determined as the first target image, and the previous frame image of the most forward associated image can be used as the identification image of the next group of startup images. Accordingly, the next group of startup images includes the group of identification images and the image that is located before the identification image and has a time interval of a third time interval with the frame image after it. The third time interval can be a time interval that matches the exposure frequency of the group of startup exposure signals. For example, t1 represents the first time interval, t2 represents the second event interval, Figure 2 is a partial schematic diagram of an image sequence provided by an embodiment of the present invention, such as Figure 2 As shown, P1 in the figure can be the identification image, and N1 can be the first frame target image.

[0072] It should be noted that in the embodiment of the present application, the operation of detecting the number of first images in each image sequence can also be performed when each set of startup images is complete. That is to say, when each target image acquisition module is opened at the same time, whether to automatically merge is determined based on whether the number of first images in each image sequence is consistent. For example, Figure 3 This is a flowchart of a startup image detection provided by an embodiment of the present application, such as Figure 3As shown, the image processing component can first search for the startup images captured based on the startup exposure signal. Each set of startup images corresponding to the startup exposure signals can be traversed in reverse order to determine whether they are complete. Specifically, the corresponding time interval can be calculated based on the exposure frequency of the currently traversed startup exposure signal. Then, based on the calculated time interval and the timestamp information of each image in the image sequence, the number of images that match the time interval is searched. Next, a determination is made as to whether the number of images matches the number of first trigger signals included in the currently traversed startup exposure signal. If not, the search can be determined to have failed. Conversely, if they match, the startup images corresponding to that set of startup exposure signals can be determined to be complete, and the search can be continued for the next set of startup exposure signals, even if not all of them have been traversed. If all m sets of startup images have been traversed, it is determined that all m sets of startup images are complete. Next, the first target image can be determined, i.e., the first image captured at the target exposure frequency.

[0073] In some embodiments, the embodiments of the present application may further include the following steps:

[0074] Step 103: After the target image is captured, the control component controls the signal generator to generate at least one set of end exposure signals, and synchronously sends the at least one set of end exposure signals to each target image acquisition module, and the image processing component obtains at least one set of end images captured by each target image acquisition module based on the at least one end exposure signal; the exposure frequency of the end exposure signal is different from the exposure frequency of the target exposure signal.

[0075] In an embodiment of the present application, the target image acquisition process is determined to be complete and the target image capture is complete when the user presses the end button, when the number of target exposure signals sent reaches a preset transmission threshold, or when the target exposure signal transmission duration reaches a preset duration threshold. Accordingly, an end exposure signal can be further sent to control the target image acquisition module to continue capturing at least one set of end images. For example, the control component can send a third control instruction to the signal generator, and the signal generator can generate and send the end exposure signal in response to the third control instruction.

[0076] Furthermore, each target image acquisition module can proactively return the captured end image to the image processing component. Accordingly, the image processing component can obtain the end image captured by each image processing component by receiving the end image returned by each target image acquisition module. Alternatively, the end image captured by each image processing component can be stored in a preset storage device, and then a connection can be established between the preset storage device and the image processing component. Accordingly, the image processing component can obtain the end image from the preset storage device to obtain the end image captured by each image processing component.

[0077] Since the target image acquisition module may not have completed the exposure of the last frame at the end of the target image acquisition, the target image acquisition module is provided with a buffer stage after the target exposure signal is sent. To a certain extent, each target image acquisition module can complete the response to the last target exposure information, avoiding incomplete target image acquisition caused by sudden stops, and ensuring the integrity of the target image. In the embodiment of the present application, by controlling the target image acquisition module to capture the start image and the end image respectively before starting to acquire the target image and after completing the acquisition of the target image, the problem of data loss or incompleteness caused by inconsistent opening and closing times of the target image acquisition modules can be reduced at the same time, and the probability of frame misalignment in the target images acquired by different target image acquisition modules can be further reduced.

[0078] In some embodiments, different groups of end-exposure signals have different exposure frequencies. For any group of end-exposure signals, the end-exposure signal is a second continuous pulse signal including a second trigger signal, and the time interval between the second trigger signals matches the signal frequency of the end-exposure signal.

[0079] The step of synchronously sending the at least one group of end exposure signals to each of the target image acquisition modules may specifically include: step 1031, for any group of the end exposure signals, simultaneously sending the second continuous pulse signal to each of the target image acquisition modules through the signal line between the module and each of the target image acquisition modules.

[0080] In an embodiment of the present application, the type of the second trigger signal can be consistent with the first trigger signal described above. For example, the second trigger signal can be a rising edge signal, that is, the second trigger signal can be a high-level signal. A second trigger signal is used to instruct the target image acquisition module to capture a final image. The exposure frequency of each group of final exposure signals can be dynamically set as needed, as long as the exposure frequency of each group of final exposure signals is different and different from the target exposure frequency. In this way, by ensuring that the exposure frequency of each group of final exposure signals is different, it is convenient to subsequently distinguish between different groups of final images.

[0081] For example, assuming that at least one set of end-exposure signals specifically includes three sets of end-exposure signals, with exposure frequencies n1, n2, and n3, respectively, after confirming that the user has pressed the end button to stop acquisition, three sets of end-exposure signals with frequencies n1, n2, and n3, respectively, can be sequentially issued to end exposure. The specific number of second trigger signals included in each set of end-exposure signals can be dynamically set as needed, and the number of second trigger signals included in each set of end-exposure signals can be the same or different. The number of second trigger signals included in at least one set of end-exposure signals can be positively correlated with the number of target image acquisition modules and noise factors. Specifically, the number of second trigger signals included in each set of end-exposure signals is positively correlated with the number of target image acquisition modules. The number of target image acquisition modules can be set based on actual needs and is not limited in this embodiment of the present application. Specifically, the greater the number of target image acquisition modules, the greater the noise in the synchronous image acquisition system. Therefore, the number of second trigger signals can be increased. For example, when the number of target image acquisition modules exceeds a preset module threshold, for example, greater than 30, the number of second trigger signals can be increased, for example, to 12 or 20. If the number of target image acquisition modules is not greater than a preset module number threshold, the number of second trigger signals is reduced, for example, to 8 or 7. Furthermore, the number of second trigger signals may also be related to other noise factors. For example, if the stability of the power supply voltage is less than a preset stability threshold, more second trigger signals may be set.

[0082] In an embodiment of the present application, by controlling a signal generator to generate a continuous pulse signal including multiple second trigger signals as an end-exposure signal, the target image acquisition module can be controlled to continuously perform exposure capture, thereby ensuring image capture efficiency to a certain extent. The time interval between the second trigger signals in the end-exposure signal matches the exposure frequency of the end-exposure signal. This may mean that the time interval between the second trigger signals in the end-exposure signal is calculated based on the exposure frequency of the end-exposure signal. Specifically, in milliseconds, for any set of end-exposure signals, the time interval between the second trigger signals in the set of end-exposure signals may be 1000 ms / exposure frequency of the end-exposure signal.

[0083] When the signal generator generates at least one set of end exposure signals, it can generate at least one set of end exposure signals according to the set at least one set of end exposure frequencies and the number of second signals. For example, assuming that at least one set of end exposure frequencies and the number of trigger signals include: 10@66Hz, 10@122Hz, 12@11Hz, the signal generator can generate a continuous pulse signal including 10 rising edges and a time interval of 1000 / 66 between each rising edge in the set order, and send it synchronously to each target image acquisition module through the signal line. Then, it generates a continuous pulse signal including 10 rising edges and a time interval of 1000 / 122 between each rising edge, and sends it synchronously to each target image acquisition module through the signal line. Finally, it generates a continuous pulse signal including 12 rising edges and a time interval of 1000 / 11 between each rising edge, and sends it synchronously to each target image acquisition module through the signal line.

[0084] In an embodiment of the present application, a second continuous pulse signal is sent through a signal line between each target image acquisition module. The signal line serves as a physical connection, and the propagation delay is small. Transmission through the signal line can ensure to a greater extent that each target image acquisition module receives the end exposure signal at the same time, and the synchronization accuracy is high.

[0085] Figure 4 is a schematic diagram of an image capturing system provided in an embodiment of the present application, such as Figure 4 As shown, the synchronization signal generator can be connected to the XR camera in the XR glasses and the camera in the motion capture device respectively. Optitrack represents the motion capture device, and the conversion center (e.g., esync hub) is used to convert the exposure signal into a network synchronization protocol that conforms to the motion capture device. The esync hub enables Optitrack to be connected to an external synchronization signal generator.

[0086] In some embodiments, the image sequence also includes the at least one group of end images taken by the target image acquisition module. In an embodiment of the present application, the following is also included: step S51, in which the image processing component detects whether the number of third images in each image sequence is consistent with the number of second signals; the number of third images is the number of images included in the last group of end images taken in the image sequence, and the number of second signals is the number of second trigger signals included in the last group of end exposure signals sent.

[0087] Furthermore, the image processing component detecting the number of first images of each image sequence when the number of second images is consistent with the number of first signals may specifically include: step S221a, the image processing component detecting whether the number of first images of each image sequence is consistent when the number of second images is consistent with the number of first signals, and the number of third images is consistent with the number of second signals.

[0088] In this embodiment of the present application, the number of third images can be determined by counting backward from the last captured end image, in the reverse order of the end exposure signal transmission. By comparing the number of third images with the number of second signals, the completeness of the last captured set of end images can be determined. Because different target image acquisition modules switch from capturing images according to the target exposure signal to capturing images according to the end exposure signal, some end exposure signals may be missed. For example, due to stability issues, the target image acquisition module may still be processing the previous exposure signal, potentially losing newly received exposure signals. In this embodiment of the present application, only the number of images in the last set of end images is compared with the number of second trigger signals in the corresponding set of end exposure signals. If the two numbers match, the end images are considered complete, which reduces the difficulty of the comparison and improves the rationality of the comparison. Specifically, the process can be repeated starting from the last captured end image. For any image found, the time interval between the image and the next image can be checked to see if it is a fourth time interval. The fourth time interval can be a time interval that matches the exposure frequency of the last set of start exposure signals. If so, the image is considered to be in the same set as the last end image. Then, the total number of images in the same group and the last frame ending image is counted to obtain the third number of images.

[0089] Of course, in the embodiment of the present application, the operation of detecting the number of first images in each image sequence can also be performed after each set of end images is complete, and the embodiment of the present application is not limited to this. Furthermore, the previous frame image of the same set of images that is positioned closest to the front can be used as the identification image for the next set of end images. Accordingly, the next set of end images includes the group identification image and the image that precedes the identification image and is separated from the next frame image by a fifth time interval. The fifth time interval can be a time interval that matches the exposure frequency of the group end exposure signal.

[0090] If the number of images included in the last set of end images shot is the same as the number of second trigger signals included in the last set of end exposure signals sent, that is, the number of third images in each image sequence is consistent with the number of second signals, and the end image is also complete, then it can be said that when shooting the last set of end images shot, the time synchronization between each target image acquisition module is good. In this way, it is proved to a greater extent that the target images shot previously are most likely to be synchronized in time. Accordingly, in this case, the probability of detecting inconsistent target images can be avoided to a greater extent. Since the number of target images is often large, in the embodiment of the present application, by controlling the target image acquisition module to shoot the start image and the end image, and by detecting whether the start image and the end image are complete, through double verification, if both are complete, it is only necessary to check whether the number of target images in different image sequences is consistent, which can reduce the amount of calculation to a certain extent.

[0091] In the embodiments of the present application, a synchronization protocol based on multiple sets of exposure start and end signals facilitates dual verification and the determination of the first and last exposure frames of the target image. By further controlling the target image acquisition module to capture the final image after the target image is captured, and accordingly determining whether the final image is complete, and if the final image is also complete, then detecting the number of first images in each image sequence to determine whether the number of target images captured by different target image acquisition modules is consistent, this can significantly avoid unnecessary execution of the operations of detecting the number of first images and determining whether the number of first images is consistent, thereby significantly saving processing resources.

[0092] It should be noted that, in an embodiment of the present application, an image whose time interval with the previous frame image is the second time interval and whose time interval with the next frame image is the sixth time interval can be determined as the last frame target image. The sixth time interval can be a time interval that matches the exposure frequency of a set of end exposure signals sent first. Accordingly, for any image sequence, the number of images from the first frame target image (i.e., the exposure head frame) to the last frame target image (i.e., the exposure tail frame) can be counted to obtain the first image number of the image sequence. For example, assuming that there are 98 frames of images between the first frame target image and the last frame target image, the first image number of the image sequence can be 100.

[0093] Furthermore, in the case of an image sequence in which the number of first images is inconsistent with the number of first images of other image sequences, manual merging can be performed, and the failure of data set synchronization can be confirmed, and the current recognition result (for example, the result indicating whether the number of second images is consistent with the number of first signals, the result indicating whether the number of third images is consistent with the number of second signals, and the result indicating whether the number of first images is consistent) can be discarded. For example, the target image sequence captured by each target image acquisition module can be displayed. Then, according to the user's selection operation, the selected target images are associated and stored. The selection operation can be triggered by the user to select the target images captured at the same time.

[0094] Figure 5 This is a flowchart of an image sequence merging provided by an embodiment of the present application, such as Figure 5 As shown, if the encrypted signal at the start of synchronization is complete and the encrypted signal at the end of synchronization is complete, that is, if the number of second images in each image sequence is consistent with the number of first signals, and the number of third images is consistent with the number of second signals, the number of first images in each image sequence can be calculated. If the numbers of first images are consistent, an automatic merge is performed. Otherwise, the dataset synchronization is confirmed to have failed and a manual merge can be performed.

[0095] In some embodiments, the minimum exposure frequency of the start exposure signal and the minimum exposure frequency of the end exposure signal may be no less than N times the exposure frequency of the target exposure signal; where N is an integer no less than 2. In other words, the minimum exposure frequency used by the start exposure signal and the end exposure signal is no less than twice the target exposure frequency. For example, if the target exposure frequency is 10 Hz, then the minimum exposure frequency used by the start exposure signal and the end exposure signal is greater than 20 Hz. This ensures that the time interval between target images is significantly different from the time interval between the start image and the time interval between the end image, making it easier to distinguish and ensuring the accuracy of the first and last target frames identified.

[0096] It should be noted that, since spatial synchronization depends on time synchronization, in an embodiment of the present application, on the basis of achieving time synchronization, the associated stored target images can be spatially synchronized. For example, for any group of associated stored target images, the positional relationship of the target image captured by the XR glasses relative to the target image captured by the motion capture device can be determined to achieve spatial synchronization. In some embodiments, the image shooting control method provided in an embodiment of the present application can be executed by an image shooting control device. In the embodiment of the present application, the image shooting control device provided in an embodiment of the present application is taken as an example to illustrate the image shooting control method executed by the image shooting control device.

[0097] See also Figure 6 The embodiment of the present application provides an image capture control device, which is applied to an image synchronization device. The image synchronization device includes a control component and an image processing component. The device includes:

[0098] The first control module 201 located in the control component is used to control the signal generator to generate at least one set of start exposure signals, and synchronously send the at least one set of start exposure signals to at least two target image acquisition modules connected to the signal generator. The first acquisition module 202 located in the image processing component is used to acquire at least one set of start images taken by each of the target image acquisition modules based on the at least one start exposure signal.

[0099] The second control module 203 located in the control component is used to control the signal generator to generate a target exposure signal and synchronously send the target exposure signal to each target image acquisition module when at least one group of startup images captured by each target image acquisition module is not empty. The second acquisition module 204 located in the image processing component is used to acquire the target image captured by each target image acquisition module based on the target exposure signal; the exposure frequency of the target exposure signal is different from the exposure frequency of the startup exposure signal.

[0100] In some embodiments, the at least one group of start-up exposure signals includes M first trigger signals, each of which is used to instruct the target image acquisition module to capture one of the start-up images; wherein M is a positive integer greater than a preset number threshold, and M is positively correlated with the turn-on delay between the at least two target image acquisition modules. In some embodiments, different groups of start-up exposure signals have different exposure frequencies. For any group of start-up exposure signals, the start-up exposure signal is a first continuous pulse signal including a first trigger signal, and the time interval between the first trigger signals matches the exposure frequency of the start-up exposure signal. The first control module 201 is specifically configured to: for any group of start-up exposure signals, control the signal generator to simultaneously send the first continuous pulse signal to each of the target image acquisition modules via a signal line between the signal generator and each of the target image acquisition modules.

[0101] In some embodiments, the device also includes: a first detection module located in the image processing component, used to detect the number of first images in the image sequence corresponding to each of the target image acquisition modules; the image sequence corresponding to the target image acquisition module includes the target images captured by the target image acquisition module, and the first number of images is the number of target images included in the image sequence; a merging module located in the image processing component, used to automatically merge the target images in the image sequences corresponding to the at least two target image acquisition modules when the number of first images in each of the image sequences is the same.

[0102] In some embodiments, the device also includes: a second detection module located in the image processing component, used to detect whether the number of second images in each of the image sequences is consistent with the number of first signals; the second number of images is the number of images included in the last set of start images taken in the image sequence, and the first number of signals is the number of first trigger signals included in the last set of start exposure signals sent; the first detection module is specifically used to: when the number of second images is consistent with the number of first signals, detect the number of first images in each of the image sequences.

[0103] In some embodiments, the device further includes: a determination module located in the image processing component, for determining, for any of the image sequences, an image whose time interval with the previous frame image is a first time interval and whose time interval with the subsequent frame image is a second time interval, as an identification image; the first time interval matches the exposure frequency of the last set of start exposure signals sent, and the second time interval matches the exposure frequency of the target exposure signal; a statistics module located in the image processing component, for counting the total number of the identification images and the associated images of the identification images to obtain the first number of images in the image sequence; the associated images include an image that is located before the identification image and whose time interval with the subsequent frame image is the first time interval.

[0104] In some embodiments, the device further includes: a third control module located in the control component, used to control the signal generator to generate at least one set of end exposure signals after the target image is captured, and synchronously send the at least one set of end exposure signals to each target image acquisition module; a third acquisition module located in the image processing component, used to acquire at least one set of end images captured by each target image acquisition module based on the at least one end exposure signal; the exposure frequency of the end exposure signal is different from the exposure frequency of the target exposure signal.

[0105] In some embodiments, the exposure frequencies of different groups of end exposure signals are different. For any group of the end exposure signals, the end exposure signal is a second continuous pulse signal including a second trigger signal, and the time interval between the second trigger signals matches the signal frequency of the end exposure signal; the third control module is specifically used to: for any group of the end exposure signals, control the signal generator to send the second continuous pulse signal to each of the target image acquisition modules through the signal line between the signal generator and the target image acquisition modules at the same time.

[0106] In some embodiments, the image sequence also includes the at least one group of end images taken by the target image acquisition module, and the device also includes: a third detection module located in the image processing component, used to detect whether the number of third images in each of the image sequences is consistent with the number of second signals; the number of third images is the number of images included in the last group of end images taken in the image sequence, and the number of second signals is the number of second trigger signals included in the last group of end exposure signals sent; the first detection module is specifically further used to: when the number of second images is consistent with the number of first signals, and the number of third images is consistent with the number of second signals, detect whether the number of first images in each of the image sequences is consistent.

[0107] In some embodiments, the minimum exposure frequency of the start exposure signal and the minimum exposure frequency of the end exposure signal are not less than N times the exposure frequency of the target exposure signal; where N is an integer not less than 2. The image capture control device and the image capture control method described in the previous embodiment have the same advantages over the related art and are not further described here.

[0108] The image capture control device in the embodiment of the present application can be an electronic device, or a component in an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or a device other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (MID), an augmented reality (AR) / virtual reality (VR) device, a robot, a wearable device, an ultra-mobile personal computer (UMPC), a netbook or a personal digital assistant (PDA), etc. It can also be a server, a network attached storage (NAS), a personal computer (PC), a television (TV), an ATM or a self-service machine, etc., which is not specifically limited in the embodiment of the present application. The image capture control device in the embodiment of the present application can be a device with an operating system. The operating system can be an Android operating system, an iOS operating system, or other possible operating systems, which is not specifically limited in the embodiment of the present application. The image capture control device provided in the embodiment of the present application can realize Figure 1 The various processes implemented in the embodiment of the method achieve the same technical effect, and to avoid repetition, they are not described here. In some embodiments, Figure 7 As shown, the embodiment of the present application further provides an electronic device M40, including a processor M401 and a memory M402, wherein the memory M402 stores a program or instruction that can be run on the processor M401, and when the program or instruction is executed by the processor M401, the various steps of the above-mentioned image capture control method embodiment are implemented and can achieve the same technical effect. To avoid repetition, they are not described here. It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices described above.

[0109] Figure 8The following is a schematic diagram of the hardware structure of another electronic device implementing an embodiment of the present application. The electronic device 500 includes, but is not limited to, components such as a radio frequency unit 501, a network module 502, an audio output unit 503, an input unit 504, a sensor 505, a display unit 506, a user input unit 507, an interface unit 508, a memory 509, and a processor 510. It will be appreciated by those skilled in the art that the electronic device 500 may further include a power supply (such as a battery) to power each component. The power supply may be logically connected to the processor 510 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption through the power management system. Figure 8 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be repeated here.

[0110] The aforementioned control component and image processing component may be a processor 510. The processor 510 is configured to control a signal generator to generate at least one set of start-up exposure signals, synchronously send the at least one set of start-up exposure signals to at least two target image acquisition modules connected to the signal generator, and obtain at least one set of start-up images captured by each target image acquisition module based on the at least one start-up exposure signal; if the at least one set of start-up images captured by each target image acquisition module is not empty, control the signal generator to generate a target exposure signal, synchronously send the target exposure signal to each target image acquisition module, and obtain a target image captured by each target image acquisition module based on the target exposure signal; the exposure frequency of the target exposure signal is different from the exposure frequency of the start-up exposure signal.

[0111] Different groups of start-up exposure signals have different exposure frequencies. For any group of the start-up exposure signals, the start-up exposure signal is a first continuous pulse signal including a first trigger signal, and the time interval between the first trigger signals matches the exposure frequency of the start-up exposure signal. The processor 510 is further configured to control the signal generator to simultaneously send the first continuous pulse signal to each target image acquisition module via a signal line between the signal generator and each target image acquisition module. The processor 510 is further configured to detect the number of first images in the image sequence corresponding to each target image acquisition module; the image sequence corresponding to the target image acquisition module includes the target images captured by the target image acquisition module, and the first number of images is the number of target images included in the image sequence; if the number of first images in each image sequence is the same, the target images in the image sequences corresponding to the at least two target image acquisition modules are automatically merged.

[0112] The image sequence also includes at least one set of startup images captured by the target image acquisition module. The processor 510 is further configured to detect whether the number of second images in each of the image sequences is consistent with the number of first signals. The second number of images is the number of images included in the last set of startup images captured in the image sequence, and the first number of signals is the number of first trigger signals included in the last set of startup exposure signals sent. Detecting the number of first images in each of the image sequences includes: if the second number of images is consistent with the first number of signals, detecting the number of first images in each of the image sequences. The processor 510 is further configured to, for any of the image sequences, determine an image that is separated from a previous image by a first time interval and separated from a subsequent image by a second time interval as an identifier image; the first time interval matches the exposure frequency of the last set of startup exposure signals sent, and the second time interval matches the exposure frequency of the target exposure signal; and to count the total number of the identifier images and images associated with the identifier images to obtain the number of first images in the image sequence. The associated images include images that precede the identifier image and are separated from the subsequent image by the first time interval.

[0113] The processor 510 is further configured to, after the target image is captured, control the signal generator to generate at least one set of end-exposure signals, synchronously send the at least one set of end-exposure signals to each target image acquisition module, and acquire at least one set of end images captured by each target image acquisition module based on the at least one end-exposure signal; the exposure frequency of the end-exposure signal is different from the exposure frequency of the target exposure signal. Different sets of end-exposure signals have different exposure frequencies; for any set of end-exposure signals, the end-exposure signal is a second continuous pulse signal including a second trigger signal, and the time interval between the second trigger signals matches the signal frequency of the end-exposure signal; the processor 510 is further configured to, for any set of end-exposure signals, control the signal generator to simultaneously send the second continuous pulse signal to each target image acquisition module via a signal line between the signal generator and each target image acquisition module. Each of the image sequences also includes at least one set of end images. The processor 510 is further configured to detect whether the number of third images in each of the image sequences is consistent with the number of second signals; the third number of images is the number of images included in the last set of end images captured in the image sequence, and the second number of signals is the number of second trigger signals included in the last set of end exposure signals sent; if the second number of images is consistent with the first number of signals, and the third number of images is consistent with the second number of signals, the processor 510 is configured to detect whether the number of first images in each of the image sequences is consistent. The electronic device has the same advantages as the image capture control method described in the previous embodiment over the related art, and these advantages are not further described here.

[0114] It should be understood that in an embodiment of the present application, the input unit 504 may include a graphics processing unit (GPU) 5041 and a microphone 5042, and the graphics processor 5041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 506 may include a display panel 5061, and the display panel 5061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 507 includes a touch panel 5071 and at least one of other input devices 5072. The touch panel 5071 is also called a touch screen. The touch panel 5071 may include two parts: a touch detection device and a touch controller. Other input devices 5072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and an operating stick, which will not be repeated here.

[0115] The memory 509 can be used to store software programs and various data. The memory 509 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, applications or instructions required for at least one function (such as a sound playback function, an image playback function, etc.). In addition, the memory 509 may include a volatile memory or a non-volatile memory, or the memory 509 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM), and a direct memory bus random access memory (DRRAM). The memory 509 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0116] The processor 510 may include one or more processing units. In some embodiments, the processor 510 integrates an application processor and a modem processor. The application processor primarily handles operations related to the operating system, user interface, and application programs, while the modem processor primarily processes wireless communication signals, such as a baseband processor. It is understood that the modem processor may not be integrated into the processor 510. The present application also provides a readable storage medium storing a program or instructions. When executed by the processor, the program or instructions implement the various processes of the aforementioned image capture control method embodiment and achieve the same technical effects. To avoid repetition, these details are not described here. The processor is the processor in the electronic device described in the aforementioned embodiment. The readable storage medium includes a computer-readable storage medium, such as a computer read-only memory (ROM), random access memory (RAM), a magnetic disk, or an optical disk. Another embodiment of the present application provides a chip comprising a processor and a communication interface, the communication interface coupled to the processor. The processor is configured to execute the program or instructions to implement the various processes of the aforementioned image capture control method embodiment and achieve the same technical effects. To avoid repetition, these details are not described here. It should be understood that the chip mentioned in the embodiments of the present application may also be referred to as a system-on-chip, a system-on-chip, a chip system, or a system-on-chip chip. The embodiments of the present application provide a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the above-mentioned image capture control method embodiment, and can achieve the same technical effects. To avoid repetition, it will not be described here.

[0117] It should be noted that, as used herein, the terms "comprise," "include," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, article, or device. Without further limitation, an element defined by the phrase "comprises a ..." does not preclude the presence of other identical elements in the process, method, article, or device comprising that element. Furthermore, it should be noted that the scope of the methods and devices in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in reverse order depending on the functions involved. For example, the described methods may be performed in an order different from that described, and various steps may be added, omitted, or combined. Furthermore, features described with reference to certain examples may be combined in other examples. Through the above description of the embodiments, those skilled in the art will clearly understand that the above-described embodiment methods can be implemented using software plus the necessary general-purpose hardware platform, or, of course, hardware, but in many cases the former is a more preferred embodiment. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application. The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present application and the claims, all of which fall within the protection of the present application.

Claims

1. An image shooting control method, characterized in that: Applied to an image synchronization device, the image synchronization device includes a control component and an image processing component; the method includes: The control component controls the signal generator to generate at least one set of start-up exposure signals, and synchronously sends the at least one set of start-up exposure signals to at least two target image acquisition modules connected to the signal generator, and the image processing component obtains at least one set of start-up images captured by each of the target image acquisition modules based on the at least one start-up exposure signal; When at least one group of startup images captured by each of the target image acquisition modules is not empty, the control component controls the signal generator to generate a target exposure signal, and synchronously sends the target exposure signal to each of the target image acquisition modules, and the image processing component obtains the target image captured by each of the target image acquisition modules based on the target exposure signal; the exposure frequency of the target exposure signal is different from the exposure frequency of the startup exposure signal.

2. The method according to claim 1, characterized in that The at least one group of start-up exposure signals includes M first trigger signals, and one first trigger signal is used to instruct the target image acquisition module to capture one of the start-up images; Wherein, M is a positive integer greater than a preset number threshold, and M is positively correlated with the opening delay between the at least two target image acquisition modules.

3. The method according to claim 1, characterized in that The exposure frequencies of the start exposure signals of different groups are different. For any group of the start exposure signals, the start exposure signal is a first continuous pulse signal including a first trigger signal, and the time interval between the first trigger signals matches the exposure frequency of the start exposure signal. The synchronously sending the at least one group of start exposure signals to the at least two target image acquisition modules connected to the signal generator includes: for any group of the start exposure signals, simultaneously sending the first continuous pulse signal to each of the target image acquisition modules through the signal line between the modules and the target image acquisition modules.

4. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: The image processing component detects a first number of images in an image sequence corresponding to each target image acquisition module; the image sequence corresponding to the target image acquisition module includes the target images captured by the target image acquisition module, and the first number of images is the number of target images included in the image sequence; In a case where the number of first images in each of the image sequences is consistent, the image processing component automatically merges the target images in the image sequences corresponding to the at least two target image acquisition modules.

5. The method according to claim 4, characterized in that The image sequence further includes at least one set of startup images captured by the target image acquisition module; the method further includes: The image processing component detects whether the number of second images in each of the image sequences is consistent with the number of first signals; the second number of images is the number of images included in a last set of start-up images captured in the image sequence, and the first number of signals is the number of first trigger signals included in a last set of start-up exposure signals sent; The detecting, by the image processing component, the number of first images of the image sequence corresponding to each of the target image acquisition modules includes: the image processing component detecting, by the image processing component, the number of first images of each of the image sequences when the number of the second images is consistent with the number of the first signals.

6. The method according to claim 5, characterized in that The method further comprises: For any of the image sequences, the image processing component determines, as an identification image, an image having a first time interval with a previous image and a second time interval with a subsequent image; the first time interval matches an exposure frequency of the last set of start exposure signals sent, and the second time interval matches an exposure frequency of the target exposure signal; The image processing component counts the total number of the identification image and the associated images of the identification image to obtain the first number of images in the image sequence; the associated images include images that are located before the identification image and have a time interval with the subsequent frame image that is the first time interval.

7. The method according to claim 5, characterized in that The method further comprises: After the target image is captured, the control component controls the signal generator to generate at least one set of end exposure signals, and synchronously sends the at least one set of end exposure signals to each target image acquisition module, and the image processing component obtains at least one set of end images captured by each target image acquisition module based on the at least one end exposure signal; the exposure frequency of the end exposure signal is different from the exposure frequency of the target exposure signal.

8. The method according to claim 7, characterized in that Different groups of end-exposure signals have different exposure frequencies. For any group of end-exposure signals, the end-exposure signal is a second continuous pulse signal including a second trigger signal, and the time interval between the second trigger signals matches the signal frequency of the end-exposure signal. The synchronously sending the at least one group of end exposure signals to each of the target image acquisition modules includes: for any group of the end exposure signals, simultaneously sending the second continuous pulse signal to each of the target image acquisition modules through the signal line between the module and each of the target image acquisition modules.

9. The method according to claim 8, characterized in that The image sequence also includes the at least one set of ending images captured by the target image acquisition module, and the method further includes: The image processing component detects whether the third number of images in each of the image sequences is consistent with the second number of signals; the third number of images is the number of images included in a group of end images captured last in the image sequence, and the second number of signals is the number of second trigger signals included in a group of end exposure signals sent last; The image processing component detects the number of first images in each of the image sequences when the number of second images is consistent with the number of first signals, including: the image processing component detects whether the number of first images in each of the image sequences is consistent when the number of second images is consistent with the number of first signals, and the number of third images is consistent with the number of second signals.

10. The method according to claim 7, characterized in that The minimum exposure frequency of the start exposure signal and the minimum exposure frequency of the end exposure signal are not less than N times the exposure frequency of the target exposure signal; wherein N is an integer not less than 2.

11. An image capture control device, characterized in that: Applied to an image synchronization device, the image synchronization device includes a control component and an image processing component; the device includes: a first control module located in the control component, configured to control the signal generator to generate at least one set of start-up exposure signals and synchronously send the at least one set of start-up exposure signals to at least two target image acquisition modules connected to the signal generator; and a first acquisition module located in the image processing component, configured to acquire at least one set of start-up images captured by each of the target image acquisition modules based on the at least one start-up exposure signal; The second control module located in the control component is used to control the signal generator to generate a target exposure signal and synchronously send the target exposure signal to each target image acquisition module when at least one group of startup images captured by each target image acquisition module is not empty. The second acquisition module located in the image processing component is used to acquire the target image captured by each target image acquisition module based on the target exposure signal; the exposure frequency of the target exposure signal is different from the exposure frequency of the startup exposure signal.

12. An electronic device, characterized in that: The apparatus comprises a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the image shooting control method according to any one of claims 1 to 10 are implemented.

13. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by the processor, the steps of the image capture control method according to any one of claims 1 to 10 are implemented.

Images