Real-time MEMS attitude matching camera and laser sensor time synchronization method and device

Abstract

The invention relates to a real-time MEMS attitude matching camera and laser sensor time synchronization method and device, and the method comprises the steps: enabling a camera, a laser radar and MEMS equipment to be fixedly connected to form a multi-mode visual perception system, obtaining the MEMS accelerometer data of the laser radar and the camera, extracting the features of the MEMS accelerometer data through a feature extraction algorithm, and carrying out the real-time MEMS attitude matching. Matching is carried out according to the features so as to complete high-precision time alignment, and finally, a time alignment result is verified according to the moment when the equipment starts to move. Therefore, the problems of poor multi-modal data fusion precision and the like caused by the fact that a current consumer camera and a laser radar lack advanced characteristics such as an external time trigger interface and the like and are difficult to realize time synchronization with external equipment are solved.

Description

Technical Field

[0001] The present application relates to the technical field of multi-sensor integration, and in particular to a method and device for time synchronization of a camera and a laser sensor with real-time MEMS posture matching. Background Art

[0002] MEMS (Micro-Electro-Mechanical System) accelerometer is an advanced, highly integrated sensor technology that is specifically used to measure the acceleration force of an object in a specific direction. The core of this sensor is that it uses micro-electromechanical system technology to miniaturize traditional mechanical accelerometer components and integrate them into microelectronic chips. MEMS accelerometers contain extremely small mass blocks and springs. When acceleration acts on the sensor, these microstructures will displace, and then convert the physical displacement into electrical signals through capacitance changes or piezoelectric effects, thereby achieving quantitative measurement of acceleration and angular acceleration. The advantages of MEMS accelerometers are their small size, light weight, and extremely low power consumption, while they can provide high accuracy and reliability. These characteristics make MEMS accelerometers an indispensable component in modern electronic and mobile devices.

[0003] The application of MEMS accelerometers is also crucial in devices such as consumer-grade sports cameras and lidar, which are mainly aimed at the general consumer market and emphasize cost-effectiveness and user-friendliness.

[0004] However, because MEMS accelerometers typically lack advanced features such as external time trigger interfaces, it is difficult to achieve time synchronization with external devices in scenarios such as fusing visual information from a camera with distance and orientation information from a lidar, which may lead to inaccurate or misunderstood data. For example, in an autonomous vehicle, unsynchronized data may lead to erroneous obstacle detection or distance estimation.

[0005] In summary, the existing application methods of MEMS accelerometers lack the limitation of external time trigger interface, making it difficult to achieve effective time synchronization between camera and lidar, which greatly affects the quality of data fusion and needs to be solved urgently. Summary of the invention

[0006] The present application provides a method and device for time synchronization of a camera and a laser sensor with real-time MEMS attitude matching, in order to solve the problem that current consumer-grade cameras and laser radars lack advanced features such as an external time trigger interface, making it difficult to achieve time synchronization with external devices, resulting in poor multimodal data fusion accuracy, and the like.

[0007] The first aspect of the present application provides a method for time synchronization of a camera and a laser sensor with real-time MEMS posture matching, comprising the following steps: constructing a multimodal visual perception system based on a preset camera, a laser radar and a MEMS device, and collecting visual perception data corresponding to the multimodal visual perception system, wherein the visual perception data includes camera data, camera MEMS data, laser radar data and laser radar MEMS data; respectively obtaining MEMS acceleration time series corresponding to the camera MEMS data and the laser radar MEMS data, extracting feature information corresponding to the MEMS acceleration time series, and matching the MEMS acceleration time series according to the feature information to obtain MEMS matching data; respectively determining the start movement time corresponding to the camera and the laser radar, and calculating the movement time difference between the camera and the laser radar according to the start movement time, and comparing the movement time difference with the MEMS matching data to obtain the time alignment index corresponding to the camera and the laser radar, and judging whether the time alignment index meets the preset time synchronization condition, wherein if the time alignment index meets the time synchronization condition, it is judged that the camera and the laser radar have achieved time synchronization.

[0008] Optionally, in one embodiment of the present application, the multimodal visual perception system is constructed based on a preset camera, a laser radar and a MEMS device, and visual perception data corresponding to the multimodal visual perception system is collected, including: constructing a camera MEMS architecture based on the camera and the MEMS device, and constructing a laser radar MEMS architecture through the laser radar and the MEMS device; fixing the camera MEMS architecture and the laser radar MEMS architecture according to a preset fixed connection relationship to construct the multimodal visual perception system; starting the laser radar data collection function and the camera data collection function of the multimodal visual perception system in sequence, and leaving the laser radar and the camera target for a certain period of time, The laser radar data and the laser radar MEMS data are synchronously collected through the laser radar MEMS architecture, and the camera data and the camera MEMS data are synchronously collected using the camera MEMS architecture; whether the laser radar MEMS architecture and the camera MEMS architecture have completed the data collection operation is determined respectively; if the laser radar MEMS architecture and the camera MEMS architecture have completed the data collection operation, the laser radar and the camera are left stationary for the target time, and after the target time, the laser radar data collection function and the camera data collection function are stopped in turn, otherwise the data collection operation of the laser radar MEMS architecture and / or the camera MEMS architecture is continued.

[0009] Optionally, in one embodiment of the present application, the MEMS acceleration time series corresponding to the camera MEMS data and the lidar MEMS data are respectively obtained, and feature information corresponding to the MEMS acceleration time series is extracted, and the MEMS acceleration time series is matched according to the feature information to obtain MEMS matching data, including: saving the camera MEMS data and the lidar MEMS data as MEMS acceleration time series in a target basic data cell format, respectively, wherein the MEMS acceleration time series includes a timestamp, a yaw angle angular acceleration, a roll angle angular acceleration, and a pitch angle angular acceleration; obtaining the fixed connection A rotation matrix corresponding to the connection relationship is obtained, and the yaw angular acceleration, the roll angular acceleration and the pitch angular acceleration of the laser radar are rotated according to the rotation matrix, so that the Z axis of the coordinate system of the preset laser radar MEMS accelerometer is parallel to the Z axis of the coordinate system of the preset machine MEMS accelerometer; based on the yaw angular acceleration, the time threshold corresponding to the MEMS acceleration information is determined, and according to the time threshold, the MEMS acceleration time series is feature extracted to obtain the peak sequence corresponding to the MEMS acceleration time series; the MEMS acceleration time series is matched through the peak sequence to generate the MEMS matching data.

[0010] Optionally, in one embodiment of the present application, respectively determining the start movement times corresponding to the camera and the lidar, and calculating the movement time difference between the camera and the lidar based on the start movement times, includes: respectively obtaining the device still time before acquisition and the device still time after acquisition of the lidar and the camera, and respectively determining the start movement time and the end movement time corresponding to the lidar and the camera based on the device still time before acquisition and the device still time after acquisition; and calculating the movement time difference between the lidar and the camera based on the start movement time and the end movement time.

[0011] Optionally, in one embodiment of the present application, the mathematical expression of the time synchronization condition is:

[0012] |t lstart -t cstart -dt lj |<threshold

[0013] |t lend -t cend -dt lj |<threshold

[0014] Among them, t lstart represents the start time of the laser radar movement; t cstartrepresents the start time of the camera movement; t lend represents the end time of the laser radar movement; t cend Indicates the end motion moment of the camera; dt lj represents the MEMS matching data; threshold represents the preset time synchronization threshold.

[0015] The second aspect of the present application provides a camera and laser sensor time synchronization device with real-time MEMS posture matching, including: a construction module, which is used to construct a multimodal visual perception system based on a preset camera, laser radar and MEMS device, and collect visual perception data corresponding to the multimodal visual perception system, wherein the visual perception data includes camera data, camera MEMS data, laser radar data and laser radar MEMS data; a matching module, which is used to respectively obtain the MEMS acceleration time series corresponding to the camera MEMS data and the laser radar MEMS data, and extract the feature information corresponding to the MEMS acceleration time series, and match the MEMS acceleration time series according to the feature information to obtain MEMS matching data; a synchronization module, which is used to respectively determine the start movement time corresponding to the camera and the laser radar, and calculate the movement time difference between the camera and the laser radar according to the start movement time, and compare the movement time difference with the MEMS matching data to obtain the time alignment index corresponding to the camera and the laser radar, and determine whether the time alignment index meets the preset time synchronization condition, wherein if the time alignment index meets the time synchronization condition, it is determined that the camera and the laser radar have achieved time synchronization.

[0016] Optionally, in one embodiment of the present application, the construction module includes: an establishment unit, used to construct a camera MEMS architecture based on the camera and the MEMS device, and to construct a laser radar MEMS architecture through the laser radar and the MEMS device; a fixing unit, used to fix the camera MEMS architecture and the laser radar MEMS architecture according to a preset fixed connection relationship to construct the multimodal visual perception system; a starting unit, used to start the laser radar data acquisition function and the camera data acquisition function of the multimodal visual perception system in turn, and after the laser radar and the camera are left stationary for a certain period of time, synchronously collect the laser data through the laser radar MEMS architecture. The laser radar data and the laser radar MEMS data, and the camera data and the camera MEMS data are synchronously collected by using the camera MEMS architecture; a judgment unit is used to respectively judge whether the laser radar MEMS architecture and the camera MEMS architecture have completed the data collection operation; a static unit is used to if the laser radar MEMS architecture and the camera MEMS architecture have completed the data collection operation, then static the laser radar and the camera for the target time, and after the target time, sequentially stop the laser radar data collection function and the camera data collection function, otherwise continue to execute the data collection operation of the laser radar MEMS architecture and / or the camera MEMS architecture.

[0017] Optionally, in one embodiment of the present application, the matching module includes: a saving unit, used to save the camera MEMS data and the lidar MEMS data as a MEMS acceleration time series in the target basic data cell format, respectively, wherein the MEMS acceleration time series includes a timestamp, a yaw angle angular acceleration, a roll angle angular acceleration and a pitch angle angular acceleration; a rotation unit, used to obtain a rotation matrix corresponding to the fixed connection relationship, and rotate the yaw angle angular acceleration, the roll angle angular acceleration and the pitch angle angular acceleration of the lidar according to the rotation matrix, so that the preset laser radar MEMS accelerometer coordinate system Z axis is parallel to the preset machine MEMS accelerometer coordinate system Z axis; a determination unit, used to determine the time threshold corresponding to the MEMS acceleration information based on the yaw angle angular acceleration, and perform feature extraction on the MEMS acceleration time series according to the time threshold to obtain a peak sequence corresponding to the MEMS acceleration time series; a generation unit, used to match the MEMS acceleration time series through the peak sequence to generate the MEMS matching data.

[0018] Optionally, in one embodiment of the present application, the synchronization module includes: an acquisition unit, used to respectively acquire the device still time before acquisition and the device still time after acquisition of the laser radar and the camera, and determine the start movement time and the end movement time corresponding to the laser radar and the camera according to the device still time before acquisition and the device still time after acquisition; a calculation unit, used to calculate the movement time difference between the laser radar and the camera based on the start movement time and the end movement time.

[0019] Optionally, in one embodiment of the present application, the mathematical expression of the time synchronization condition is:

[0020] |t lstart -t cstart -dt lj |<threshold

[0021] |t lend -t cend -dt lj |<threshold

[0022] Among them, t lstart represents the start time of the laser radar movement; t cstart represents the start time of the camera movement; t lend represents the end time of the laser radar movement; t cend Indicates the end motion moment of the camera; dt lj represents the MEMS matching data; threshold represents the preset time synchronization threshold.

[0023] The third aspect of the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the camera and laser sensor time synchronization method for real-time MEMS posture matching as described in the above embodiment.

[0024] The fourth aspect of the present application provides a computer-readable storage medium, which stores a computer program. When the program is executed by a processor, it implements the above-mentioned real-time MEMS posture matching camera and laser sensor time synchronization method.

[0025] The fifth aspect of the present application provides a computer program product, including a computer program, which is executed to implement the above-mentioned real-time MEMS posture matching camera and laser sensor time synchronization method.

[0026] Therefore, the embodiments of the present application have the following beneficial effects:

[0027] The embodiments of the present application can construct a multimodal visual perception system based on preset cameras, laser radars and MEMS devices, and collect visual perception data corresponding to the multimodal visual perception system, wherein the visual perception data includes camera data, camera MEMS data, laser radar data and laser radar MEMS data; obtain the MEMS acceleration time series corresponding to the camera MEMS data and the laser radar MEMS data respectively, extract the feature information corresponding to the MEMS acceleration time series, and match the MEMS acceleration time series according to the feature information to obtain MEMS matching data; determine the start movement time corresponding to the camera and the laser radar respectively, and calculate the movement time difference between the camera and the laser radar according to the start movement time, and compare the movement time difference with the MEMS matching data to obtain the time alignment index corresponding to the camera and the laser radar, and judge whether the time alignment index meets the preset time synchronization condition, wherein if the time alignment index meets the time synchronization condition, it is determined that the camera and the laser radar have reached time synchronization. The present application can quickly complete the high-precision time alignment of consumer-grade devices without relying on the time synchronization interface by matching the MEMS accelerometer data fixed to the device, thereby significantly improving the accuracy and reliability of data fusion. This solves the problem that current consumer-grade cameras and lidars lack advanced features such as external time trigger interfaces, making it difficult to achieve time synchronization with external devices, resulting in poor multimodal data fusion accuracy.

[0028] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0030] Figure 1 A flowchart of a camera and laser sensor time synchronization method for real-time MEMS posture matching provided according to an embodiment of the present application;

[0031] Figure 2 A schematic diagram of a laser radar and a panoramic camera provided for one embodiment of the present application;

[0032] Figure 3 A schematic diagram of a device coordinate system of a laser radar and a panoramic camera provided for one embodiment of the present application;

[0033] Figure 4 This is an example diagram of a camera and laser sensor time synchronization device for real-time MEMS posture matching according to an embodiment of the present application;

[0034] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0035] Among them, 10-real-time MEMS posture matching camera and laser sensor time synchronization device, 100-building module, 200-matching module, 300-synchronization module, 501-memory, 502-processor, 503-communication interface. DETAILED DESCRIPTION

[0036] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0037] The following describes the camera and laser sensor time synchronization method and device for real-time MEMS posture matching of the embodiment of the present application with reference to the accompanying drawings. In view of the problems mentioned in the above background technology, the present application provides a camera and laser sensor time synchronization method for real-time MEMS posture matching, in which a multimodal visual perception system is constructed based on a preset camera, laser radar and MEMS device, and visual perception data corresponding to the multimodal visual perception system is collected, wherein the visual perception data includes camera data, camera MEMS data, laser radar data and laser radar MEMS data; the MEMS acceleration time series corresponding to the camera MEMS data and the laser radar MEMS data are obtained respectively, and the feature information corresponding to the MEMS acceleration time series is extracted, and the MEMS acceleration time series is matched according to the feature information to obtain MEMS matching data; the start movement time corresponding to the camera and the laser radar is determined respectively, and the movement time difference between the camera and the laser radar is calculated according to the start movement time, and the movement time difference and the MEMS matching data are compared to obtain the time alignment index corresponding to the camera and the laser radar, and it is judged whether the time alignment index meets the preset time synchronization condition, wherein if the time alignment index meets the time synchronization condition, it is determined that the camera and the laser radar have achieved time synchronization. This application can quickly complete the high-precision time alignment of consumer-grade devices without relying on the time synchronization interface by matching the data of the MEMS accelerometer fixed to the device, thereby significantly improving the accuracy and reliability of data fusion. This solves the problem that current consumer-grade cameras and lidars lack advanced features such as external time trigger interfaces, making it difficult to achieve time synchronization with external devices, resulting in poor multimodal data fusion accuracy.

[0038] Specifically, Figure 1 A flowchart of a method for time synchronization of a camera and a laser sensor with real-time MEMS posture matching provided in an embodiment of the present application.

[0039] like Figure 1 As shown, the camera and laser sensor time synchronization method for real-time MEMS posture matching includes the following steps:

[0040] In step S101, a multimodal visual perception system is constructed based on preset cameras, lidars and MEMS devices, and visual perception data corresponding to the multimodal visual perception system is collected, wherein the visual perception data includes camera data, camera MEMS data, lidar data and lidar MEMS data.

[0041] The embodiments of the present application can first connect the camera, lidar and MEMS device to form a multimodal visual perception system, and use the multimodal visual perception system to collect data to collect visual perception data. After the start and end of data collection, they are left to stand for more than 30 seconds respectively. After the collection is completed, the embodiments of the present application can select an axis with obvious angle changes during the collection process to respectively obtain the MEMS accelerometer data (i.e., camera MEMS data and lidar MEMS data) in the visual perception data of the lidar and camera under the axis.

[0042] Optionally, in one embodiment of the present application, a multimodal visual perception system is constructed based on a preset camera, a lidar and a MEMS device, and visual perception data corresponding to the multimodal visual perception system is collected, including: constructing a camera MEMS architecture based on the camera and the MEMS device, and constructing a lidar MEMS architecture through the lidar and the MEMS device; fixing the camera MEMS architecture and the lidar MEMS architecture according to a preset fixed connection relationship to construct a multimodal visual perception system; starting the lidar data collection function and the camera data collection function of the multimodal visual perception system in turn, and leaving the lidar and the camera stationary. After the target time, the lidar data and the lidar MEMS data are synchronously collected through the lidar MEMS architecture, and the camera data and the camera MEMS data are synchronously collected using the camera MEMS architecture; it is determined whether the lidar MEMS architecture and the camera MEMS architecture have completed the data collection operation respectively; if both the lidar MEMS architecture and the camera MEMS architecture have completed the data collection operation, the lidar and the camera are kept stationary for the target time, and after the target time, the lidar data collection function and the camera data collection function are stopped in turn, otherwise the data collection operation of the lidar MEMS architecture and / or the camera MEMS architecture continues to be executed.

[0043] In the embodiments of the present application, the camera can be fixedly connected to a low-cost MEMS to form a camera MEMS module (i.e., a camera MEMS architecture). Similarly, a laser radar can be fixedly connected to a low-cost MEMS to construct a laser radar MEMS module (i.e., a laser radar MEMS architecture). Thereafter, the embodiments of the present application can fixedly connect the camera MEMS module and the laser radar MEMS module according to a preset fixed connection relationship to construct a multimodal visual perception system.

[0044] As a feasible method, the embodiment of the present application can connect and fix the panoramic camera with built-in MEMS accelerometer and the laser radar together in a precise and stable manner, such as Figure 2 As shown, these sensors simultaneously collect data on their own status and surrounding space environment, and transmit the data to the built-in industrial computer for data processing and storage, so as to achieve all-round perception of indoor three-dimensional space information.

[0045] In the actual implementation process, the embodiment of the present application can use an industrial computer with a built-in 2.2GHz quad-core ARM Cortex-A73 and 2.0GHz quad-core ARM Cortex-A53 CPU and 8GB memory, and each sensor can use existing products, as shown below:

[0046] Livox Mid-360 LiDAR with built-in MEMS

[0047] Panoramic camera with built-in MEMS: Insta360 OneRS1-inch

[0048] In the specific implementation process, the embodiment of the present application can connect the laser radar to the industrial computer for data transmission and control, and the panoramic camera can be controlled independently; it should be noted that the embodiment of the present application can take the coordinate system of the panoramic camera as the carrier coordinate system of the multimodal visual perception system, such as Figure 3 As shown, the front direction of the multimodal visual perception system is the x-axis direction, the plane where the multimodal visual perception system is located is the xoy plane, and the xyz right-handed coordinate system is constructed. The laser radar is installed under the panoramic camera and rotated 15° clockwise around the y-axis.

[0049] Afterwards, the embodiments of the present application can start the lidar data acquisition function and the camera data acquisition function of the multimodal visual perception system in sequence, and establish a unified time system respectively; secondly, after all the data acquisition functions are started, the devices (i.e., the camera and the lidar) are left stationary for a target time (such as 30 seconds), and then data is collected according to the plan; secondly, the embodiments of the present application can synchronously collect camera, lidar and their corresponding MEMS data through the camera MEMS architecture and the lidar MEMS architecture respectively.

[0050] Determine whether the LiDAR MEMS architecture and the camera MEMS architecture have completed the data collection operation. After the collection is completed, leave the device still for 30 seconds. When the collection is finished, stop the camera data collection first, and then stop the LiDAR data collection.

[0051] If one or both of the laser radar and the camera have not completed the data collection, the data collection operation of the laser radar and the camera will continue.

[0052] Finally, the embodiments of the present application can read the MEMS data (ie, camera MEMS data and lidar MEMS data) recorded by the lidar and panoramic camera through an industrial computer and related programs.

[0053] Therefore, the embodiments of the present application connect cameras, lidars and MEMS devices and perform multimodal visual perception system data collection, thereby providing reliable data guidance and basis for the subsequent implementation of high-precision time synchronization.

[0054] In step S102, the MEMS acceleration time series corresponding to the camera MEMS data and the lidar MEMS data are respectively obtained, and the feature information corresponding to the MEMS acceleration time series is extracted, and the MEMS acceleration time series is matched according to the feature information to obtain MEMS matching data.

[0055] Furthermore, the embodiments of the present application also need to obtain the MEMS data of the lidar and the camera respectively, select the axis with obvious angle change, extract the peak value of its MEMS acceleration information as a feature, and match the two sets of acceleration time series to obtain MEMS matching data.

[0056] Optionally, in one embodiment of the present application, MEMS acceleration time series corresponding to the camera MEMS data and the lidar MEMS data are respectively obtained, and feature information corresponding to the MEMS acceleration time series is extracted, and the MEMS acceleration time series is matched according to the feature information to obtain MEMS matching data, including: saving the camera MEMS data and the lidar MEMS data as MEMS acceleration time series in the target basic data cell format, respectively, wherein the MEMS acceleration time series includes a timestamp, yaw angle angular acceleration, roll angle angular acceleration and pitch angle angular acceleration; obtaining A rotation matrix corresponding to the fixed connection relationship is obtained, and the yaw angular acceleration, roll angular acceleration and pitch angular acceleration of the laser radar are rotated according to the rotation matrix, so that the Z axis of the coordinate system of the preset laser radar MEMS accelerometer is parallel to the Z axis of the coordinate system of the preset machine MEMS accelerometer; based on the yaw angular acceleration, the time threshold corresponding to the MEMS acceleration information is determined, and the MEMS acceleration time series is feature extracted according to the time threshold to obtain the peak sequence corresponding to the MEMS acceleration time series; the MEMS acceleration time series is matched through the peak sequence to generate MEMS matching data.

[0057] In the embodiment of the present application, the specific steps of preprocessing and accurately matching the camera MEMS data and the lidar MEMS data are as follows:

[0058] Step 1: extract the data stored in the MEMS accelerometers in the laser radar and the panoramic camera (i.e., the camera MEMS data and the laser radar MEMS data), and save them according to the timestamp, yaw angle (Yaw) angular acceleration, roll angle (Roll) angular acceleration, and pitch angle (Pitch) angular acceleration as basic data units, denoted as I l =(t l , yaw l , roll l , pitch l );

[0059] Step 2: Rotation matrix R corresponding to the fixed connection relationship between the lidar and the panoramic camera lc , according to formula (1), the angular velocity vector of the laser radar is rotated so that the three axes of the coordinate system of the laser radar MEMS accelerometer are parallel to the three corresponding axes of the coordinate system of the panoramic camera MEMS accelerometer;

[0060]

[0061] Step 3: Select the angular velocity of the yaw angle, set the time threshold t, perform feature extraction on the MEMS acceleration time series, and extract the peak sequence I of each set of data. l=[I l1 , I l2 , I l3 ,…,I ln ], each of which I li (i∈{1,2,3,…,n}), its yaw li are all local maxima within time t''

[0062] Step 4: Extract the peak sequences of the LiDAR MEMS and the panoramic camera MEMS, respectively, and record them as I ll and I lc , and calculate dt according to formula 2 l Sequence, dt l The length of the sequence should be the same as I ll Consistent.

[0063] dt l =[t ll0 -t lc0 , t ll1 -t lc0 , t ll2 -t lc0 ,…,t lln -t lc0 ] (2)

[0064] Step 5: I lc Each item in t lci Add the same dt lj , and in I ll Find the corresponding item in lc Each term in the above is added with dt lj After that, you can lc If a corresponding item is found, then dt lj It is the time difference between the LiDAR system and the panoramic camera system to generate MEMS matching data.

[0065] Therefore, the embodiments of the present application effectively ensure the execution and implementation of subsequent time alignment verification operations using static data by accurately matching the dual MEMS accelerometers that match the camera MEMS data and the lidar MEMS data.

[0066] In step S103, the start movement times corresponding to the camera and the laser radar are determined respectively, and the movement time difference between the camera and the laser radar is calculated based on the start movement time, and the movement time difference and the MEMS matching data are compared to obtain the time alignment indicators corresponding to the camera and the laser radar, and determine whether the time alignment indicators meet the preset time synchronization conditions. If the time alignment indicators meet the time synchronization conditions, it is determined that the camera and the laser radar have achieved time synchronization.

[0067] Furthermore, the embodiments of the present application also need to extract the static point data of the laser radar and the camera, and detect the moment when they start to move based on the postures of the laser radar and the camera, and compare the time difference between the start of movement of the camera and the laser radar with the MEMS matching data to verify whether the time alignment result (i.e., the time alignment index) meets the preset time synchronization condition, and when the time alignment index meets the time synchronization condition, it is determined that the camera and the laser radar have achieved time synchronization.

[0068] Therefore, the embodiments of the present application synchronously integrate the panoramic camera and lidar integrated with MEMS accelerometers, match the data recorded by the MEMS accelerometers of the panoramic camera and lidar during data acquisition, and thus achieve time synchronization of the panoramic camera and lidar without an external time trigger interface.

[0069] Optionally, in one embodiment of the present application, the start movement times corresponding to the camera and the lidar are determined respectively, and the movement time difference between the camera and the lidar is calculated based on the start movement time, including: obtaining the device still time before acquisition and the device still time after acquisition of the lidar and the camera respectively, and determining the start movement time and the end movement time corresponding to the lidar and the camera respectively based on the device still time before acquisition and the device still time after acquisition; based on the start movement time and the end movement time, calculating the movement time difference between the lidar and the camera.

[0070] It should be noted that the embodiment of the present application can extract the start and end time points t of the laser radar and the panoramic camera according to the device static time before the start and end of the acquisition. lstart ,t lend ,t cstart and t cend The motion time difference between the lidar and the camera is calculated by the start and end time of the motion, so as to determine whether effective time synchronization is achieved between the camera and the lidar according to the motion time difference.

[0071] Therefore, the embodiments of the present application combine the device of the fixedly connected MEMS with the time series feature extraction algorithm to realize the time synchronization of the panoramic camera and the lidar in the absence of an external time trigger interface, so as to achieve the purpose of effective time synchronization between the camera and the lidar in the absence of an external time trigger interface, thereby ensuring the accuracy and reliability of data fusion.

[0072] Optionally, in one embodiment of the present application, the mathematical expression of the time synchronization condition is:

[0073] |t lstart -t cstart -dt lj |<threshold

[0074] |t lend -t cend -dt lj |<threshold

[0075] Among them, t lstart Indicates the start time of the laser radar movement; t cstart Indicates the moment when the camera starts moving; t lend Indicates the end time of the laser radar movement; t cend Indicates the end time of the camera movement; dt lj Indicates MEMS matching data; threshold indicates the preset time synchronization threshold.

[0076] In an embodiment of the present application, the mathematical expression of the above time synchronization condition is:

[0077]

[0078] Among them, t lstart Indicates the start time of the laser radar movement; t cstart Indicates the moment when the camera starts moving; t lend Indicates the end time of the laser radar movement; t cend Indicates the end time of the camera movement; dt lj Indicates MEMS matching data; threshold indicates the preset time synchronization threshold.

[0079] Therefore, the embodiment of the present application compares the motion time difference and the MEMS matching data to obtain the time alignment index corresponding to the camera and the lidar, and determines whether the time alignment index satisfies the time synchronization condition shown in formula (3). If the time alignment index satisfies the time synchronization condition, it is determined that the time alignment result achieved by the camera and the lidar is reliable.

[0080] To sum up, the time alignment result calculated by the embodiment of the present application is not only accurate and precise, experimentally true and effective, but also can serve as a solid foundation for subsequent sensor data fusion. The color three-dimensional point cloud model that fuses the panoramic camera image data and the lidar point cloud data based on the time alignment result of the embodiment of the present application has a good effect, thereby verifying the effectiveness of the time synchronization of the present application.

[0081] According to the camera and laser sensor time synchronization method for real-time MEMS posture matching proposed in an embodiment of the present application, a multimodal visual perception system is constructed based on preset cameras, laser radars and MEMS devices, and visual perception data corresponding to the multimodal visual perception system is collected, wherein the visual perception data includes camera data, camera MEMS data, laser radar data and laser radar MEMS data; MEMS acceleration time series corresponding to the camera MEMS data and the laser radar MEMS data are respectively obtained, and feature information corresponding to the MEMS acceleration time series is extracted, and the MEMS acceleration time series is matched according to the feature information to obtain MEMS matching data; the start movement time corresponding to the camera and the laser radar is respectively determined, and the movement time difference between the camera and the laser radar is calculated according to the start movement time, and the movement time difference and the MEMS matching data are compared to obtain the time alignment index corresponding to the camera and the laser radar, and it is determined whether the time alignment index meets the preset time synchronization condition, wherein if the time alignment index meets the time synchronization condition, it is determined that the camera and the laser radar have achieved time synchronization. By matching the data of MEMS accelerometers fixed to the device, this application can quickly complete high-precision time alignment of consumer-grade devices without relying on a time synchronization interface, thereby significantly improving the accuracy and reliability of data fusion.

[0082] Secondly, the camera and laser sensor time synchronization device for real-time MEMS posture matching proposed in the embodiment of the present application is described with reference to the accompanying drawings.

[0083] Figure 4 It is a block diagram of a camera and laser sensor time synchronization device for real-time MEMS posture matching according to an embodiment of the present application.

[0084] like Figure 4 As shown, the real-time MEMS posture matching camera and laser sensor time synchronization device 10 includes: a construction module 100, a matching module 200 and a synchronization module 300.

[0085] Among them, construction module 100 is used to build a multimodal visual perception system based on preset cameras, lidars and MEMS devices, and collect visual perception data corresponding to the multimodal visual perception system, wherein the visual perception data includes camera data, camera MEMS data, lidar data and lidar MEMS data.

[0086] The matching module 200 is used to respectively obtain the MEMS acceleration time series corresponding to the camera MEMS data and the lidar MEMS data, extract the feature information corresponding to the MEMS acceleration time series, and match the MEMS acceleration time series according to the feature information to obtain MEMS matching data.

[0087] The synchronization module 300 is used to determine the start movement time corresponding to the camera and the laser radar respectively, and calculate the movement time difference between the camera and the laser radar according to the start movement time, and compare the movement time difference and the MEMS matching data to obtain the time alignment index corresponding to the camera and the laser radar, and determine whether the time alignment index meets the preset time synchronization condition, wherein if the time alignment index meets the time synchronization condition, it is determined that the camera and the laser radar have achieved time synchronization.

[0088] Optionally, in one embodiment of the present application, the construction module 100 includes: a building unit, a fixing unit, a starting unit, a judging unit and a static unit.

[0089] Among them, a unit is established to build a camera MEMS architecture based on a camera and a MEMS device, and to build a lidar MEMS architecture through a lidar and a MEMS device.

[0090] The fixing unit is used to fix the camera MEMS architecture and the lidar MEMS architecture according to a preset fixed connection relationship to build a multimodal visual perception system.

[0091] The starting unit is used to start the lidar data acquisition function and the camera data acquisition function of the multimodal visual perception system in sequence, and after the lidar and camera targets are stationary for a period of time, the lidar data and the lidar MEMS data are synchronously collected through the lidar MEMS architecture, and the camera data and the camera MEMS data are synchronously collected using the camera MEMS architecture.

[0092] The judgment unit is used to judge whether the laser radar MEMS architecture and the camera MEMS architecture have completed the data collection operation.

[0093] The static unit is used to statically store the laser radar and camera for a target time if both the laser radar MEMS architecture and the camera MEMS architecture have completed the data acquisition operation, and to stop the laser radar data acquisition function and the camera data acquisition function in turn after the target time, otherwise continue to execute the data acquisition operation of the laser radar MEMS architecture and / or the camera MEMS architecture.

[0094] Optionally, in one embodiment of the present application, the matching module 200 includes: a storage unit, a rotation unit, a determination unit and a generation unit.

[0095] Among them, the saving unit is used to save the camera MEMS data and the lidar MEMS data as MEMS acceleration time series in the target basic data cell format, wherein the MEMS acceleration time series includes a timestamp, yaw angle acceleration, roll angle acceleration and pitch angle acceleration.

[0096] The rotation unit is used to obtain the rotation matrix corresponding to the fixed connection relationship, and rotate the yaw angular acceleration, roll angular acceleration and pitch angular acceleration of the laser radar according to the rotation matrix, so that the Z axis of the coordinate system of the preset laser radar MEMS accelerometer is parallel to the Z axis of the coordinate system of the preset machine MEMS accelerometer.

[0097] The determination unit is used to determine the time threshold corresponding to the MEMS acceleration information based on the yaw angle angular acceleration, and perform feature extraction on the MEMS acceleration time series according to the time threshold to obtain a peak sequence corresponding to the MEMS acceleration time series.

[0098] The generating unit is used for matching the MEMS acceleration time series through the peak sequence to generate MEMS matching data.

[0099] Optionally, in one embodiment of the present application, the synchronization module 300 includes: an acquisition unit and a calculation unit.

[0100] Among them, the acquisition unit is used to respectively obtain the device still time before and after acquisition of the laser radar and camera, and determine the start and end movement times corresponding to the laser radar and camera according to the device still time before and after acquisition.

[0101] The calculation unit is used to calculate the movement time difference between the laser radar and the camera based on the start time and the end time of the movement.

[0102] Optionally, in one embodiment of the present application, the mathematical expression of the time synchronization condition is:

[0103] |t lstart -t cstart -dt lj |<threshold

[0104] |t lend -t cend -dt lj |<threshold

[0105] Among them, t lstart Indicates the start time of the laser radar movement; t cstart Indicates the moment when the camera starts moving; t lend Indicates the end time of the laser radar movement; t cend Indicates the end time of the camera movement; dt lj Indicates MEMS matching data; threshold indicates the preset time synchronization threshold.

[0106] It should be noted that the aforementioned explanation of the embodiment of the method for time synchronization of a camera and a laser sensor with real-time MEMS attitude matching is also applicable to the device for time synchronization of a camera and a laser sensor with real-time MEMS attitude matching of this embodiment, and will not be repeated here.

[0107] A camera and laser sensor time synchronization device for real-time MEMS posture matching proposed in an embodiment of the present application includes a construction module for constructing a multimodal visual perception system based on preset cameras, laser radars and MEMS devices, and collecting visual perception data corresponding to the multimodal visual perception system, wherein the visual perception data includes camera data, camera MEMS data, laser radar data and laser radar MEMS data; a matching module for respectively obtaining MEMS acceleration time series corresponding to the camera MEMS data and the laser radar MEMS data, and extracting feature information corresponding to the MEMS acceleration time series, and matching the MEMS acceleration time series according to the feature information to obtain MEMS matching data; a synchronization module for respectively determining the start motion moments corresponding to the camera and the laser radar, and calculating the motion time difference between the camera and the laser radar according to the start motion moment, and comparing the motion time difference with the MEMS matching data to obtain the time alignment index corresponding to the camera and the laser radar, and judging whether the time alignment index meets the preset time synchronization condition, wherein if the time alignment index meets the time synchronization condition, it is judged that the camera and the laser radar have achieved time synchronization. By matching the data of MEMS accelerometers fixed to the device, this application can quickly complete high-precision time alignment of consumer-grade devices without relying on a time synchronization interface, thereby significantly improving the accuracy and reliability of data fusion.

[0108] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device may include:

[0109] A memory 501 , a processor 502 , and a computer program stored in the memory 501 and executable on the processor 502 .

[0110] When the processor 502 executes the program, the camera and laser sensor time synchronization method for real-time MEMS posture matching provided in the above embodiment is implemented.

[0111] Furthermore, the electronic device further comprises:

[0112] The communication interface 503 is used for communication between the memory 501 and the processor 502 .

[0113] The memory 501 is used to store computer programs that can be executed on the processor 502 .

[0114] The memory 501 may include a high-speed RAM memory, and may also include a non-volatile memory (non- vo Latil e memory), such as at least one disk storage.

[0115] If the memory 501, the processor 502 and the communication interface 503 are implemented independently, the communication interface 503, the memory 501 and the processor 502 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0116] Optionally, in a specific implementation, if the memory 501, the processor 502 and the communication interface 503 are integrated on a chip, the memory 501, the processor 502 and the communication interface 503 can communicate with each other through an internal interface.

[0117] The processor 502 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0118] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned method for time synchronization of a camera and a laser sensor for real-time MEMS posture matching.

[0119] An embodiment of the present application also provides a computer program product, including a computer program, which, when executed, is used to implement the above-mentioned camera and laser sensor time synchronization method for real-time MEMS posture matching.

[0120] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0121] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0122] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0123] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or N wirings (electronic devices), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways as necessary and then storing it in a computer memory.

[0124] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above embodiment, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0125] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0126] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0127] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A method for real-time MEMS attitude matching camera and laser sensor time synchronization, characterized in that: The following steps are involved: Based on the preset camera, lidar and MEMS device, a multimodal visual perception system is constructed, and visual perception data corresponding to the multimodal visual perception system is collected, wherein the visual perception data includes camera data, camera MEMS data, lidar data and lidar MEMS data; Respectively acquiring MEMS acceleration time series corresponding to the camera MEMS data and the lidar MEMS data, extracting feature information corresponding to the MEMS acceleration time series, and matching the MEMS acceleration time series according to the feature information to obtain MEMS matching data; Determine the start movement times corresponding to the camera and the laser radar respectively, calculate the movement time difference between the camera and the laser radar according to the start movement time, and compare the movement time difference with the MEMS matching data to obtain the time alignment index corresponding to the camera and the laser radar, and judge whether the time alignment index meets the preset time synchronization condition, wherein if the time alignment index meets the time synchronization condition, it is determined that the camera and the laser radar are time synchronized.

2. The method according to claim 1, characterized in that The multimodal visual perception system is constructed based on the preset camera, laser radar and MEMS device, and the visual perception data corresponding to the multimodal visual perception system is collected, including: Based on the camera and the MEMS device, a camera MEMS architecture is constructed, and through the laser radar and the MEMS device, a laser radar MEMS architecture is constructed; The camera MEMS architecture and the laser radar MEMS architecture are fixedly connected according to a preset fixed connection relationship to construct the multimodal visual perception system; The laser radar data acquisition function and the camera data acquisition function of the multimodal visual perception system are sequentially started, and after the laser radar and the camera are left stationary for a certain period of time, the laser radar data and the laser radar MEMS data are synchronously acquired through the laser radar MEMS architecture, and the camera data and the camera MEMS data are synchronously acquired using the camera MEMS architecture; Determine whether the laser radar MEMS architecture and the camera MEMS architecture have completed data acquisition operations; If both the lidar MEMS architecture and the camera MEMS architecture complete the data acquisition operation, the lidar and the camera are left stationary for the target time, and after the target time, the lidar data acquisition function and the camera data acquisition function are stopped in turn, otherwise the data acquisition operation of the lidar MEMS architecture and / or the camera MEMS architecture is continued.

3. The method according to claim 2, characterized in that The step of respectively acquiring MEMS acceleration time series corresponding to the camera MEMS data and the laser radar MEMS data, extracting feature information corresponding to the MEMS acceleration time series, and matching the MEMS acceleration time series according to the feature information to obtain MEMS matching data includes: The camera MEMS data and the laser radar MEMS data are respectively saved as MEMS acceleration time series in a target basic data cell format, wherein the MEMS acceleration time series includes a timestamp, a yaw angle acceleration, a roll angle acceleration, and a pitch angle acceleration; Obtain a rotation matrix corresponding to the fixed connection relationship, and rotate the yaw angular acceleration, the roll angular acceleration, and the pitch angular acceleration of the laser radar according to the rotation matrix, so that the Z axis of the coordinate system of the preset laser radar MEMS accelerometer is parallel to the Z axis of the coordinate system of the preset machine MEMS accelerometer; Based on the yaw angle acceleration, determine a time threshold corresponding to the MEMS acceleration information, and perform feature extraction on the MEMS acceleration time series according to the time threshold to obtain a peak sequence corresponding to the MEMS acceleration time series; The MEMS acceleration time series is matched by using the peak sequence to generate the MEMS matching data.

4. The method according to claim 3, characterized in that The method of respectively determining the start movement time of the camera and the laser radar, and calculating the movement time difference between the camera and the laser radar according to the start movement time, includes: Obtain the device still time before and after acquisition of the laser radar and the camera respectively, and determine the start and end movement times of the laser radar and the camera respectively according to the device still time before and after acquisition; Based on the start time of the movement and the end time of the movement, the movement time difference between the laser radar and the camera is calculated.

5. The method according to claim 1, characterized in that The mathematical expression of the time synchronization condition is: |t lstart -t cstart -dt lj |<threshold |t lend -t cend -dt lj |<threshold Among them, t lstart represents the start time of the laser radar movement; t cstart represents the start time of the camera movement; t lend represents the end time of the laser radar movement; t cend Indicates the end motion moment of the camera; dt lj represents the MEMS matching data; threshold represents the preset time synchronization threshold.

6. A camera and laser sensor time synchronization device with real-time MEMS posture matching, characterized in that: include: A construction module, used to construct a multimodal visual perception system based on preset cameras, laser radars and MEMS devices, and collect visual perception data corresponding to the multimodal visual perception system, wherein the visual perception data includes camera data, camera MEMS data, laser radar data and laser radar MEMS data; A matching module, used to respectively obtain the MEMS acceleration time series corresponding to the camera MEMS data and the lidar MEMS data, extract feature information corresponding to the MEMS acceleration time series, and match the MEMS acceleration time series according to the feature information to obtain MEMS matching data; A synchronization module is used to respectively determine the start movement times of the camera and the laser radar, calculate the movement time difference between the camera and the laser radar based on the start movement times, and compare the movement time difference with the MEMS matching data to obtain the time alignment indicators corresponding to the camera and the laser radar, and determine whether the time alignment indicators meet the preset time synchronization conditions, wherein if the time alignment indicators meet the time synchronization conditions, it is determined that the camera and the laser radar have achieved time synchronization.

7. The device according to claim 6, characterized in that The building blocks include: A building unit, configured to build a camera MEMS architecture based on the camera and the MEMS device, and to build a lidar MEMS architecture through the lidar and the MEMS device; A fixing unit, used for fixing the camera MEMS architecture and the laser radar MEMS architecture according to a preset fixed connection relationship to construct the multimodal visual perception system; A starting unit, used to start the laser radar data acquisition function and the camera data acquisition function of the multimodal visual perception system in sequence, and after the laser radar and the camera are left stationary for a certain period of time, synchronously collect the laser radar data and the laser radar MEMS data through the laser radar MEMS architecture, and synchronously collect the camera data and the camera MEMS data using the camera MEMS architecture; A judgment unit, used to judge whether the laser radar MEMS architecture and the camera MEMS architecture have completed the data acquisition operation; A static unit is used to statically store the laser radar and the camera for a target time if both the laser radar MEMS architecture and the camera MEMS architecture have completed the data acquisition operation, and to stop the laser radar data acquisition function and the camera data acquisition function in turn after the target time, otherwise continue to execute the data acquisition operation of the laser radar MEMS architecture and / or the camera MEMS architecture.

8. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the camera and laser sensor time synchronization method for real-time MEMS posture matching as described in any one of claims 1 to 5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement a camera and laser sensor time synchronization method for real-time MEMS posture matching as described in any one of claims 1-5.

10. A computer program product, comprising a computer program, characterized in that The computer program is executed to implement the camera and laser sensor time synchronization method for real-time MEMS posture matching as described in any one of claims 1-5.

Images