External correction parameter generation device, system and method

Abstract

The invention discloses an external correction parameter generation device, system and method. A plurality of sensors of the device respectively generate observation data corresponding to a spatial point. A processor of the apparatus calculates a spatial position of the spatial point indicated by each of the plurality of observation data based on the observation data of each of the plurality of sensors. The processor compares the spatial positions indicated by the plurality of observation data to determine whether the plurality of observation data belong to an abnormal value or a normal value. In response to the observation data corresponding to a first sensor belonging to the abnormal value, the processor generates an external correction parameter corresponding to the first sensor based on the respective observation data of at least one second sensor. According to the external correction parameter generation technology provided by the invention, the operation execution accuracy of the device is improved.

Description

Technical Field

[0001] The present invention relates to an external calibration parameter generation device, system, and method. Specifically, the present invention relates to an external calibration parameter generation device, system, and method that can accurately generate external calibration parameters. Background Art

[0002] Users can use electronic devices (e.g., controllers, head-mounted displays) equipped with multiple sensors (e.g., sensor pairs or sensor arrays) to perform gesture tracking operations in a simultaneous localization and mapping (SLAM) system.

[0003] In the prior art, during the manufacturing process of an electronic device equipped with multiple sensors, calibration settings (e.g., position relationship, transmitter parameters, spatial relationship parameters, etc.) are performed on the multiple sensors to accurately perform subsequent conversion between two-dimensional images and three-dimensional space.

[0004] However, when a user uses an electronic device, the position or direction of the sensor may be abnormally displaced due to some reasons (eg, collision or impact), which may cause changes in the external parameters used by the electronic device to perform tracking operations.

[0005] Accordingly, if external parameter calibration is not performed on the electronic device, the electronic device will not be able to correctly perform data alignment operations when performing various operations (eg, device positioning operations, object tracking), thereby reducing the user's service experience.

[0006] In view of this, how to provide an external calibration parameter generation technology that can accurately generate external calibration parameters is a goal that the industry urgently needs to work on. Summary of the Invention

[0007] One purpose of the present disclosure is to provide an external correction parameter generating device. The external correction parameter generating device includes a plurality of sensors and a processor, and the processor is electrically connected to the plurality of sensors. Each of the plurality of sensors generates an observation data corresponding to a spatial point. The processor calculates a spatial position of the spatial point indicated by each of the plurality of observation data based on the observation data of each of the plurality of sensors. The processor compares the spatial positions indicated by each of the plurality of observation data to determine whether each of the plurality of observation data belongs to an abnormal value or a normal value. In response to the observation data corresponding to a first sensor belonging to the abnormal value, the processor generates an external correction parameter corresponding to the first sensor based on the observation data of each of at least one second sensor, wherein the observation data of each of the at least one second sensor belongs to the normal value.

[0008] In one embodiment of the present invention, determining whether each of the multiple observation data belongs to the abnormal value or the normal value includes the following operations: comparing the multiple observation data of the multiple sensors to group each of the multiple observation data into a normal cluster or an outlier cluster; and based on the normal cluster or the outlier cluster, determining whether each of the multiple observation data belongs to the abnormal value or the normal value.

[0009] In one embodiment of the present invention, the plurality of observations grouped into the normal cluster belong to the normal values, and the plurality of observations grouped into the outlier cluster belong to the abnormal values.

[0010] In one embodiment of the present invention, the multiple sensors include a first active depth camera, the first active depth camera includes a first transmitter and a first receiver, and the observation data generated by the first active depth camera includes a spatial relationship parameter captured by the first transmitter and the first receiver.

[0011] In one embodiment of the present invention, the multiple sensors include a second active depth camera, the second active depth camera includes a second transmitter and a second receiver, and the observation data generated by the second active depth camera includes a structured light spot pattern and a light spot code generated by the second transmitter and the second receiver.

[0012] In one embodiment of the present invention, the plurality of sensors include at least one sensor pair, and the observation data generated by the at least one sensor pair includes an image frame captured by the at least one sensor pair.

[0013] In one embodiment of the present invention, the at least one sensor pair is a stereo camera pair consisting of a plurality of image capturers, and the observation data generated by the stereo camera pair includes the image frames captured by the plurality of image capturers.

[0014] In one embodiment of the present invention, calculating the spatial position of the spatial point indicated by each of the multiple observation data includes the following operation: performing a global optimization operation corresponding to a simultaneous positioning and mapping algorithm on the multiple observation data to generate the spatial position corresponding to the spatial point indicated by each of the multiple observation data.

[0015] In one embodiment of the present invention, the processor further performs the following operation: adjusting at least one of a transmission time, a transmission power, and a transmission pattern corresponding to the first sensor, or a combination thereof based on the external calibration parameter.

[0016] In one embodiment of the present invention, the processor further performs the following operation: calibrating positioning information corresponding to the first sensor based on the external calibration parameter.

[0017] Another object of the present disclosure is to provide an external calibration parameter generation method for an electronic device. The method comprises receiving observation data corresponding to a spatial point from each of a plurality of sensors; calculating a spatial position of the spatial point indicated by each of the plurality of observation data based on the observation data from each of the plurality of sensors; comparing the spatial positions indicated by each of the plurality of observation data to determine whether each of the plurality of observation data is an outlier or a normal value; and, in response to the observation data corresponding to a first sensor being the outlier, generating an external calibration parameter corresponding to the first sensor based on the observation data from each of at least one second sensor, wherein the observation data from each of the at least one second sensor is the normal value.

[0018] In one embodiment of the present invention, determining whether each of the multiple observation data belongs to the abnormal value or the normal value includes the following steps: comparing the multiple observation data of the multiple sensors to group each of the multiple observation data into a normal cluster or an outlier cluster; and based on the normal cluster or the outlier cluster, determining whether each of the multiple observation data belongs to the abnormal value or the normal value.

[0019] In one embodiment of the present invention, the multiple sensors include a first active depth camera, the first active depth camera includes a first transmitter and a first receiver, and the observation data generated by the first active depth camera includes a spatial relationship parameter captured by the first transmitter and the first receiver.

[0020] In one embodiment of the present invention, the multiple sensors include a second active depth camera, the second active depth camera includes a second transmitter and a second receiver, and the observation data generated by the second active depth camera includes a structured light spot pattern and a light spot code generated by the second transmitter and the second receiver.

[0021] In one embodiment of the present invention, the plurality of sensors include at least one sensor pair, and the observation data generated by the at least one sensor pair includes an image frame captured by the at least one sensor pair.

[0022] In one embodiment of the present invention, the at least one sensor pair is a stereo camera pair consisting of a plurality of image capturers, and the observation data generated by the stereo camera pair includes the image frames captured by the plurality of image capturers.

[0023] In one embodiment of the present invention, calculating the spatial position of the spatial point indicated by each of the multiple observation data includes the following operation: performing a global optimization operation corresponding to a simultaneous positioning and mapping algorithm on the multiple observation data to generate the spatial position corresponding to the spatial point indicated by each of the multiple observation data.

[0024] In one embodiment of the present invention, the external calibration parameter generation method further comprises the following step: adjusting at least one of a transmission time, a transmission power, and a transmission pattern corresponding to the first sensor, or a combination thereof, based on the external calibration parameter.

[0025] In one embodiment of the present invention, the first sensor is disposed in a first electronic device, the at least one second sensor is disposed in a second electronic device, and the first electronic device is different from the second electronic device.

[0026] One object of the present disclosure is to provide an external calibration parameter generation system. The external calibration parameter generation device includes a plurality of sensors and a processing device, and the processing device is communicatively connected to the plurality of electronic devices. Each of the plurality of electronic devices includes at least one sensor, and each of the at least one sensor generates an observation data corresponding to a spatial point. The processing device receives the plurality of observation data from each of the plurality of electronic devices. The processing device calculates a spatial position of the spatial point indicated by each of the plurality of observation data based on the plurality of observation data. The processing device compares the spatial positions indicated by each of the plurality of observation data to determine whether each of the plurality of observation data belongs to an abnormal value or a normal value. In response to the observation data corresponding to a first sensor belonging to the abnormal value, the processing device generates an external calibration parameter corresponding to the first sensor based on the observation data of each of the at least one second sensor, wherein the observation data of each of the at least one second sensor belongs to the normal value.

[0027] The external correction parameter generation technology provided by the present disclosure (including at least a device, a system and a method) collects observation data corresponding to a spatial point generated by multiple sensors. Then, the observation data of multiple sensors for the same environment are compared to determine which sensor's observation data belongs to an abnormal value (for example, a position offset may occur). Then, the external correction parameters corresponding to the abnormal sensor can be generated through the observation data generated by other non-abnormal sensors in the environment, and the abnormal sensor can be corrected based on the external correction parameters until the observation data generated by the abnormal sensor is consistent with that of other sensors. In addition, the external correction parameter generation technology provided by the present disclosure can be assisted by correction through other electronic devices in the environment during operation (for example, controllers, tracking devices) without the need to set up additional correction assistance devices. Therefore, the external correction parameters can be correctly generated, the accuracy of the device's execution of operations is improved, and the user's service experience is improved.

[0028] The following describes the detailed technology and implementation methods of the present disclosure in conjunction with the accompanying drawings so that a person having ordinary knowledge in the technical field to which the present disclosure belongs can understand the technical features of the invention for which protection is sought. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A schematic diagram showing an application environment of the first embodiment;

[0030] Figure 2 A schematic diagram showing the architecture of an external correction parameter generating device according to a first embodiment;

[0031] Figure 3 A schematic diagram illustrating an operating environment for certain embodiments;

[0032] Figure 4 A schematic diagram illustrating an operating environment for certain embodiments;

[0033] Figure 5 A schematic diagram illustrating an operating environment for certain embodiments; and

[0034] Figure 6 A partial flowchart showing a method for generating external correction parameters according to the third embodiment.

[0035] Explanation of symbols:

[0036] 1: External calibration parameter generator

[0037] C: User

[0038] S1, S2, S3, S4, S5, ..., Sn: Sensors

[0039] 11: Processor

[0040] PS: Physical space

[0041] P, P': spatial point

[0042] P1, P2, P3, P4: sensor pairs

[0043] ECS: External Correction Parameter Generation System

[0044] F, X, T: Electronic devices

[0045] FP1, FP2, FP3, FP4, XP1, TP1: sensor pair

[0046] X2, X3: Sensors

[0047] 600: External calibration parameter generation method

[0048] S601, S603, S605, S607: Steps DETAILED DESCRIPTION

[0049] The following will explain an external correction parameter generating device, system and method provided by the present disclosure through implementation methods. However, the multiple implementation methods are not intended to limit the present disclosure to any environment, application or method as described in the multiple implementation methods. Therefore, the description of the implementation methods is only for the purpose of illustrating the present disclosure, and is not intended to limit the scope of the present disclosure. It should be understood that in the following implementation methods and drawings, elements that are not directly related to the present disclosure have been omitted and are not shown, and the sizes of each element and the size ratios between elements are only for illustration, and are not intended to limit the scope of the present disclosure.

[0050] First, the applicable scenario of this embodiment is described, and its schematic diagram is shown in Figure 1 .like Figure 1 As shown, in the application environment of the present disclosure, a user C may use an external calibration parameter generating device 1 (eg, a head mounted display) including a plurality of sensors S1 , S2 , . . . , Sn to operate.

[0051] It should be noted that the external calibration parameter generating device 1 disclosed herein is not limited to electronic devices operated by user C. The external calibration parameter generating device 1 can be any electronic device in the environment that needs to be calibrated (for example, a controller placed in the environment, a self-tracking device, or other user-operated device).

[0052] The first embodiment of the present invention is an external calibration parameter generating device 1, the structure of which is shown in FIG. Figure 2 In this embodiment, the external calibration parameter generating device 1 at least includes a plurality of sensors S1, S2, ..., Sn and a processor 11. The processor 11 is electrically connected to the sensors S1, S2, ..., Sn, where n is a positive integer.

[0053] It should be noted that the processor 11 can be any processing unit, a central processing unit (CPU), a microprocessor, or other computing devices known to those skilled in the art. The sensors S1, S2, ..., Sn can be any device with spatial sensing capabilities.

[0054] For example, the sensors S1, S2, ..., Sn may be a camera, a photodiode, an emitter / receiver, a structured light / laser, or other sensing devices with depth resolution capabilities.

[0055] For example, the sensor may be an image capturer with an image capture function (eg, a general camera or a depth camera lens) for generating a real-time image corresponding to a field of view (FOV).

[0056] For another example, the sensor may be composed of a transmitter and a receiver, and generate corresponding observation data through spatial relationship parameters and transmitter parameters (eg, transmission time, transmission power, and transmission pattern).

[0057] In some embodiments, the external calibration parameter generating device 1 may include a sensor pair or sensor array composed of some or all of the sensors S1, S2, ..., Sn. For example, each sensor pair may be a stereo camera composed of at least two cameras.

[0058] It should be noted that the present disclosure does not limit the number of sensors included in the external calibration parameter generating device 1. As long as the external calibration parameter generating device 1 has two or more sensors (i.e., at least two sensors), the external calibration parameters corresponding to the multiple sensors can be calibrated through the present disclosure.

[0059] In some embodiments, the external calibration parameter generating device 1 may be a head-mounted display (HMD), which generates external calibration parameters to calibrate its own multiple sensors. In some embodiments, the external calibration parameter generating device 1 may also be implemented by an electronic device having multiple sensors (e.g., a controller or tracking device).

[0060] In addition, sensors S1, S2, ..., Sn can be set at different positions of the external calibration parameter generating device 1 (for example, the left and right sides of the head-mounted display), and generate corresponding observation data at different observation angles (for example, different multiple image capture angles generate corresponding image frames).

[0061] First, let's briefly explain the operation of the present disclosure. In the present disclosure, the external calibration parameter generating device 1 can collect observation data from multiple sensors for at least one spatial point in physical space (e.g., corner point, feature point), and calculate the spatial position indicated by the observation data of each sensor (e.g., coordinates in three-dimensional space). Based on the aforementioned operation of comparing spatial positions, the calibration parameter generating device 1 can find sensors (i.e., sensors to be calibrated) that have different calculation results from other sensors, and generate external calibration parameters based on the observation data generated by the sensor corresponding to the correct spatial position result. The details of the implementation will be described in detail below.

[0062] First, in this embodiment, each of the sensors S1, S2, ..., Sn generates a piece of observation data corresponding to a spatial point. Then, the processor 11 receives the plurality of observation data from the sensors S1, S2, ..., Sn.

[0063] Next, in this embodiment, the processor 11 calculates a spatial position of the spatial point indicated by each of the plurality of observation data based on the observation data of each of the sensors S1 , S2 , . . . , Sn.

[0064] In some embodiments, the processor 11 may perform a global optimization on observation data from the same environment and the spatial map information created therefrom using sensors S1, S2, ..., Sn to reversely infer the spatial position of the spatial point indicated by each observation data. Specifically, the processor 11 performs a global optimization operation corresponding to a simultaneous localization and mapping algorithm on the plurality of observation data to generate the spatial position corresponding to the spatial point indicated by each of the plurality of observation data.

[0065] Next, in this embodiment, the processor 11 compares the spatial positions indicated by the plurality of observation data to determine whether the plurality of observation data is an outlier value or a normal value.

[0066] In some embodiments, the processor 11 compares the observations from sensors S1, S2, ..., Sn to group each observation into a normal cluster or an outlier cluster. The processor 11 then determines whether each observation belongs to an outlier or a normal value based on the normal or outlier clusters.

[0067] In some embodiments, the plurality of observations clustered into the normal cluster belong to the normal values, and the plurality of observations clustered into the outlier cluster belong to the outliers.

[0068] For example, the extrinsic calibration parameter generating device 1 includes five sensors S1, S2, S3, S4, and S5. After the processor 11 calculates the spatial position of the spatial point P indicated by each observation data, the spatial position of the spatial point P indicated by four sensors S1, S2, S3, and S5 is located at the three-dimensional coordinates (1, 1, 1), and the spatial position of the spatial point P indicated by one sensor S4 is located at the three-dimensional coordinates (1, 0, 1).

[0069] In this example, the processor 11 determines that the observation data of sensors S1, S2, S3, and S5 belong to the normal value, and sensors S1, S2, S3, and S5 are clustered into the normal cluster. The processor 11 determines that the observation data of sensor S4 belongs to the abnormal value, and sensor S4 is clustered into the outlier cluster.

[0070] Finally, in this embodiment, in response to the observation data of a sensor (for example, the first sensor referred to in certain contents of the present disclosure) belonging to the abnormal value, the processor 11 generates external correction parameters corresponding to the sensor to be calibrated (for example, the first sensor) based on the observation data of each of the other sensors (for example, at least one second sensor referred to in certain contents of the present disclosure), and the observation data of each of the sensors to be calibrated (for example, at least one second sensor) belongs to the normal value.

[0071] It should be noted that the present disclosure can perform six-degree-of-freedom extrinsic calibration based on the extrinsic calibration parameters generated by sensors at other observation angles. Therefore, the present disclosure can solve the problem of only being able to perform five-degree-of-freedom extrinsic calibration when the deformation direction is parallel to the epipolar line (i.e., horizontal variation / scale cannot be corrected).

[0072] For easier understanding, please refer to Figure 3 . Figure 3 The following diagram illustrates a user C wearing an external calibration parameter generating device 1 in a physical space PS. The external calibration parameter generating device 1 includes three sensors S1, S2, and S3, each of which is an active depth camera. In this example, sensors S1, S2, and S3 each generate observation data corresponding to a spatial point P. The processor 11 then calculates the spatial position of each of the spatial points P indicated by the observation data.

[0073] In this example, the spatial position of point P calculated by processor 11 based on the observation data generated by sensors S1 and S3 is consistent. The spatial position of point P calculated by processor 11 based on the observation data generated by sensor S2 is offset to the position of point P'. In this example, processor 11 determines that sensor S2 is offset. Therefore, processor 11 generates external calibration parameters based on the observation data generated by sensors S1 and S3 to calibrate sensor S2.

[0074] In some embodiments, the sensors S1 , S2 , . . . , Sn include at least one sensor pair, and the observation data generated by the at least one sensor pair includes an image frame captured by the at least one sensor pair.

[0075] In some embodiments, the at least one sensor pair is a stereo camera pair consisting of a plurality of image capturers, and the observation data generated by the stereo camera pair includes the image frames captured by the plurality of image capturers.

[0076] For easier understanding, please refer to Figure 4 . Figure 4The following diagram illustrates a user C wearing an external calibration parameter generating device 1 in a physical space PS. In this example, the external calibration parameter generating device 1 includes a sensor pair P1 consisting of sensors S1 and S2, a sensor pair P2 consisting of sensors S2 and S3, a sensor pair P3 consisting of sensors S3 and S4, a sensor pair P4 consisting of sensors S4 and S1, and an active depth camera sensor S5.

[0077] In this example, sensor pair P1, sensor pair P2, sensor pair P3, sensor pair P4, and sensor S5 each generate observation data corresponding to a spatial point P. Then, the processor 11 calculates the spatial position of the spatial point P indicated by each of the plurality of observation data.

[0078] In this example, the spatial position of spatial point P calculated by processor 11 based on the observation data generated by sensor pair P1, sensor pair P2, sensor pair P4, and sensor S5 is consistent. The spatial position of spatial point P calculated by processor 11 based on the observation data generated by sensor pair P3 is shown to have shifted to the position of spatial point P'. In this example, processor 11 determines that sensor pair P3 is offset. Therefore, processor 11 generates external calibration parameters based on the observation data generated by some or all of sensor pair P1, sensor pair P2, sensor pair P4, and sensor S5 to calibrate sensor pair P3.

[0079] In some embodiments, the sensors S1 , S2 , . . . , Sn include an active depth camera (eg, referred to as a first active depth camera or a second active depth camera in some aspects of this disclosure).

[0080] In some embodiments, the first active depth camera includes a first transmitter and a first receiver, and the observation data generated by the first active depth camera includes a spatial relationship parameter captured by the first transmitter and the first receiver, for example, using Time of Flight (TOF) depth sensing technology.

[0081] In some embodiments, the second active depth camera includes a second transmitter and a second receiver, and the observation data generated by the second active depth camera includes a structured light pattern and a light code generated by the second transmitter and the second receiver, for example, using structured light depth sensing technology.

[0082] It should be noted that the example combinations used in this disclosure are for illustrative purposes only and do not limit the sensor combinations. The multiple sensors in the external calibration parameter generating device 1 can be any combination or matching of the aforementioned first active depth camera, second active depth camera, or sensor pairs, depending on the actual application requirements of the device itself.

[0083] In some embodiments, the processor 11 adjusts at least one of a transmission time, a transmission power, and a transmission pattern corresponding to the first sensor, or a combination thereof, based on the external calibration parameter. For example, when the first sensor is an active depth camera, the processor 11 may adjust the transmission time and transmission power of the first sensor.

[0084] In some embodiments, the processor 11 calibrates positioning information corresponding to the first sensor based on the external calibration parameters.

[0085] As can be seen from the above description, the external correction parameter generating device 1 provided by the present disclosure collects the observation data corresponding to a spatial point generated by multiple sensors. Then, the observation data of multiple sensors for the same environment are compared to determine which sensor's observation data belongs to an abnormal value (for example, position offset may occur). Then, the external correction parameters corresponding to the abnormal sensor can be generated through the observation data generated by other non-abnormal sensors in the environment, and the abnormal sensor can be corrected based on the external correction parameters until the observation data generated by the abnormal sensor is consistent with that of other sensors. In addition, the external correction parameter generating device 1 provided by the present disclosure can be assisted by other electronic devices (for example, controllers, tracking devices) in the environment during operation without the need to set up additional correction auxiliary devices. Therefore, the external correction parameters can be correctly generated, the accuracy of the device's execution of operations is improved, and the user's service experience is improved.

[0086] In certain embodiments, the external calibration parameter generation technology disclosed herein can be extended to be performed between multiple electronic devices in an environment (ie, an external calibration parameter generation system), which will be described in detail below.

[0087] The second embodiment of the present disclosure is an external calibration parameter generation system ECS, the structure diagram of which is shown in FIG. Figure 5 In this embodiment, the external calibration parameter generation system (ECS) may include multiple electronic devices and a processing device (e.g., any electronic device with computing power in the system may serve as the processing device), with the processing device communicatively connected to the multiple electronic devices. Each of the multiple electronic devices includes at least one sensor, and each of the at least one sensor generates observation data corresponding to a spatial point.

[0088] In this embodiment, the processing device receives the plurality of observation data from each of the plurality of electronic devices, and then calculates a spatial position of the spatial point indicated by each of the plurality of observation data based on the plurality of observation data.

[0089] Next, the processing device compares the spatial positions indicated by each of the plurality of observation data to determine whether each of the plurality of observation data is an abnormal value or a normal value.

[0090] Finally, in response to the observation data corresponding to a first sensor belonging to the abnormal value, the processing device generates an external calibration parameter corresponding to the first sensor based on the observation data of each of at least one second sensor, wherein the observation data of each of the at least one second sensor belongs to the normal value.

[0091] In some embodiments, the first sensor is disposed in a first electronic device, the at least one second sensor is disposed in a second electronic device, and the first electronic device is different from the second electronic device.

[0092] In addition to the operations described above, the second embodiment can also perform all operations and steps of the external calibration parameter generating device 1 described in the first embodiment, having the same functions and achieving the same technical effects. A person skilled in the art of the present disclosure will readily understand how the second embodiment performs these operations and steps based on the first embodiment, having the same functions and achieving the same technical effects, and therefore, a detailed description thereof will not be given.

[0093] For easier understanding, please refer to Figure 5 . Figure 5 The schematic diagram of an external calibration parameter generation system ECS composed of multiple electronic devices and processing devices is shown. In this example, the external calibration parameter generation system ECS includes electronic device F, electronic device X, and electronic device T.

[0094] In this example, each electronic device can be used by a different user. Electronic device F includes sensor pair FP1, sensor pair FP2, sensor pair FP3, and sensor pair FP4. Electronic device X includes sensor pair XP1 and active depth camera sensors X3 and X2. Electronic device T includes sensor pair TP1. For ease of explanation, in this example, electronic device T serves as the processing device.

[0095] In this example, sensor pair FP1, sensor pair FP2, sensor pair FP3, and sensor pair FP4 of electronic device F each generate observation data corresponding to spatial point P. Sensor pair XP1 and sensors X3 and X2 of the active depth camera of electronic device X each generate observation data corresponding to spatial point P. Sensor pair TP1 of electronic device T generates observation data corresponding to spatial point P.

[0096] Next, the electronic device T receives the plurality of observation data and calculates the spatial position of the spatial point P indicated by each of the plurality of observation data.

[0097] In this example, the spatial position of spatial point P calculated by electronic device T based on the observation data generated by sensor pairs FP1-FP4, sensor pair XP1, and sensors X3 and X2 is consistent. The spatial position of spatial point P calculated by electronic device T based on the observation data generated by sensor pair TP1 is offset to the position of spatial point P'. In this example, electronic device T determines that sensor pair TP1 is offset. Therefore, electronic device T generates external calibration parameters based on the observation data generated by some or all of sensor pairs FP1-FP4, sensor pair XP1, and sensors X3 and X2 to calibrate sensor pair TP1.

[0098] The third embodiment of the present disclosure is a method for generating external calibration parameters, the flow chart of which is shown in FIG. Figure 6 The external calibration parameter generation method 600 is applicable to an electronic device, such as the external calibration parameter generation device 1 or the external calibration parameter generation system ECS described in the first embodiment. The external calibration parameter generation method 600 generates external calibration parameters through steps S601 to S607.

[0099] First, in step S601 , the electronic device receives observation data corresponding to a spatial point from each of a plurality of sensors.

[0100] Next, in step S603 , the electronic device calculates a spatial position of the spatial point indicated by each of the plurality of observation data based on the respective observation data of the plurality of sensors.

[0101] Next, in step S605 , the electronic device compares the spatial positions indicated by the plurality of observation data to determine whether the plurality of observation data is an abnormal value or a normal value.

[0102] Finally, in step S607, in response to the observation data corresponding to a first sensor belonging to the abnormal value, the electronic device generates an external calibration parameter corresponding to the first sensor based on the observation data of each of at least one second sensor, wherein the observation data of each of the at least one second sensor belongs to the normal value.

[0103] In certain embodiments, determining whether each of the plurality of observation data belongs to the outlier or the normal value includes the following steps: comparing the plurality of observation data of the plurality of sensors to group each of the plurality of observation data into a normal cluster or an outlier cluster; and determining whether each of the plurality of observation data belongs to the outlier or the normal value based on the normal cluster or the outlier cluster.

[0104] In some embodiments, the multiple sensors include a first active depth camera, the first active depth camera includes a first transmitter and a first receiver, and the observation data generated by the first active depth camera includes a spatial relationship parameter captured by the first transmitter and the first receiver.

[0105] In some embodiments, the multiple sensors include a second active depth camera, the second active depth camera includes a second transmitter and a second receiver, and the observation data generated by the second active depth camera includes a structured light spot pattern and a light spot code generated by the second transmitter and the second receiver.

[0106] In some embodiments, the plurality of sensors includes at least one sensor pair, and the observation data generated by the at least one sensor pair includes an image frame captured by the at least one sensor pair.

[0107] In some embodiments, the at least one sensor pair is a stereo camera pair consisting of a plurality of image capturers, and the observation data generated by the stereo camera pair includes the image frames captured by the plurality of image capturers.

[0108] In some embodiments, calculating the spatial position of the spatial point indicated by each of the multiple observation data includes the following operation: performing a global optimization operation corresponding to a simultaneous positioning and mapping algorithm on the multiple observation data to generate the spatial position corresponding to the spatial point indicated by each of the multiple observation data.

[0109] In some embodiments, the external calibration parameter generation method 600 further includes the following step: adjusting at least one of a transmission time, a transmission power, and a transmission pattern corresponding to the first sensor, or a combination thereof based on the external calibration parameter.

[0110] In some embodiments, the first sensor is disposed in a first electronic device, the at least one second sensor is disposed in a second electronic device, and the first electronic device is different from the second electronic device.

[0111] In some embodiments, the external calibration parameter generation method 600 further includes the following step: calibrating positioning information corresponding to the first sensor based on the external calibration parameters.

[0112] In addition to the aforementioned steps, the third embodiment also performs all operations and steps of the external calibration parameter generating device 1 and the external calibration parameter generating system ECS described in the first and second embodiments, having the same functions and achieving the same technical effects. A person skilled in the art of the present disclosure will readily understand how the third embodiment, based on the first and second embodiments, performs these operations and steps, having the same functions and achieving the same technical effects, and therefore will not be described in detail.

[0113] It should be noted that in this patent specification and claims, certain terms (including sensor, electronic device, active depth camera, transmitter, and receiver) are preceded by "first" or "second." These multiple "first" and "second" terms are used solely to distinguish different terms. For example, the "first" and "second" in "first sensor" and "second sensor" are used solely to indicate the sensors used for different operations.

[0114] In summary, the external correction parameter generation technology provided by the present disclosure (at least including devices, systems and methods) collects observation data corresponding to a spatial point generated by multiple sensors. Then, the observation data of multiple sensors for the same environment are compared to determine which sensor's observation data belongs to an abnormal value (for example, position offset may occur). Then, the external correction parameters corresponding to the abnormal sensor can be generated through the observation data generated by other non-abnormal sensors in the environment, and the abnormal sensor can be corrected based on the external correction parameters until the observation data generated by the abnormal sensor is consistent with that of other sensors. In addition, the external correction parameter generation technology provided by the present disclosure can be assisted by correction through other electronic devices in the environment during operation (for example, controllers, tracking devices) without the need to set up additional correction assistance devices. Therefore, the external correction parameters can be correctly generated, the accuracy of the device's execution of operations is improved, and the user's service experience is improved.

[0115] The above embodiments are merely examples of some embodiments of the present disclosure and illustrate the technical features of the present disclosure, and are not intended to limit the scope and extent of protection of the present disclosure. Any modifications or equivalent arrangements that can be easily accomplished by a person having ordinary skill in the technical field of the present disclosure are within the scope claimed by the present disclosure, and the scope of protection of the present disclosure is subject to the claims.

Claims

1. An external calibration parameter generating device, characterized in that: Include: multiple sensors; and a processor electrically connected to the plurality of sensors; The external calibration parameter generating device is used to perform the following operations: Each of the plurality of sensors generates an observation data corresponding to a spatial point; Calculating, by the processor, a spatial position of the spatial point indicated by each of the plurality of observation data based on the respective observation data of the plurality of sensors; Comparing, by the processor, the spatial positions indicated by each of the plurality of observation data to determine whether each of the plurality of observation data is an abnormal value or a normal value; and In response to the observation data corresponding to a first sensor belonging to the abnormal value, the processor generates an external calibration parameter corresponding to the first sensor based on the observation data of at least one second sensor, wherein the observation data of at least one second sensor belongs to the normal value.

2. The external correction parameter generating device according to claim 1, wherein: Determining whether each of the plurality of observation data is an abnormal value or a normal value includes the following operations: Comparing the plurality of observation data from the plurality of sensors to group each of the plurality of observation data into a normal cluster or an outlier cluster; and Based on the normal cluster or the outlier cluster, it is determined that each of the plurality of observations belongs to the abnormal value or the normal value.

3. The external correction parameter generating device according to claim 2, wherein: The plurality of observations grouped into the normal cluster belong to the normal values, and the plurality of observations grouped into the outlier cluster belong to the abnormal values.

4. The external calibration parameter generating device according to claim 1, wherein: The multiple sensors include a first active depth camera, the first active depth camera includes a first transmitter and a first receiver, and the observation data generated by the first active depth camera includes a spatial relationship parameter captured by the first transmitter and the first receiver.

5. The external correction parameter generating device according to claim 1, wherein: The multiple sensors include a second active depth camera, the second active depth camera includes a second transmitter and a second receiver, and the observation data generated by the second active depth camera includes a structured light spot pattern and a light spot code generated by the second transmitter and the second receiver.

6. The external calibration parameter generating device according to claim 1, wherein: The plurality of sensors include at least one sensor pair, and the observation data generated by the at least one sensor pair includes an image frame captured by the at least one sensor pair.

7. The external correction parameter generating device according to claim 6, wherein: The at least one sensor pair is a stereo camera pair consisting of a plurality of image capturers, and the observation data generated by the stereo camera pair includes the image frames captured by the plurality of image capturers.

8. The external correction parameter generating device according to claim 1, wherein: Calculating the spatial position of the spatial point indicated by each of the plurality of observation data comprises the following operations: A global optimization operation corresponding to a simultaneous positioning and mapping algorithm is performed on the plurality of observation data to generate the spatial positions corresponding to the spatial points indicated by each of the plurality of observation data.

9. The external calibration parameter generating device according to claim 1, wherein: The processor further performs the following operations: Based on the external calibration parameter, at least one of a transmission time, a transmission power, and a transmission pattern corresponding to the first sensor or a combination thereof is adjusted.

10. The external calibration parameter generating device according to claim 1, wherein: The processor further performs the following operations: Based on the external calibration parameters, positioning information corresponding to the first sensor is calibrated.

11. A method for generating external correction parameters, characterized in that: The method for generating external calibration parameters for an electronic device includes the following steps: receiving observation data corresponding to a spatial point from each of the plurality of sensors; Calculating a spatial position of the spatial point indicated by each of the plurality of observation data based on the respective observation data of the plurality of sensors; Comparing the spatial positions indicated by the plurality of observation data to determine whether the plurality of observation data are an abnormal value or a normal value; and In response to the observation data corresponding to a first sensor belonging to the abnormal value, an external calibration parameter corresponding to the first sensor is generated based on the observation data of each of at least one second sensor, wherein the observation data of each of the at least one second sensor belongs to the normal value.

12. An external correction parameter generation system, characterized in that: Include: A plurality of electronic devices, wherein each of the plurality of electronic devices comprises at least one sensor, and each of the at least one sensor generates an observation data corresponding to a spatial point; and a processing device communicatively connected to the plurality of electronic devices; The external calibration parameter generation system is used to perform the following operations: receiving, by the processing device, a plurality of observation data from each of the plurality of electronic devices; The processing device calculates a spatial position of the spatial point indicated by each of the plurality of observation data based on the plurality of observation data; The processing device compares the spatial positions indicated by each of the plurality of observation data to determine whether each of the plurality of observation data is an abnormal value or a normal value; and In response to the observation data corresponding to a first sensor belonging to the abnormal value, the processing device generates an external calibration parameter corresponding to the first sensor based on the observation data of at least one second sensor, wherein the observation data of at least one second sensor belongs to the normal value.

Images