Amplitude and dual-phase control for MEMS scanning devices

Abstract

A MEMS scanning device ("device") includes at least: (1) (one or more) laser projectors controlled by a laser driver to project a laser beam; (2) (one or more) MEMS scanning mirrors controlled by a MEMS driver to scan the laser beam to generate a raster scan; (3) a display configured to receive the raster scan; (4) a thermometer configured to detect a current temperature; (5) a display viewing camera configured to capture an image of a predetermined area of ​​the display; and (6) a computer-readable medium storing temperature models, each of which is custom built using machine learning. The device uses a display viewing camera to capture (one or more) images of (one or more) predetermined patterns, which are then used to extract (one or more) features. The extracted (one or more) features are compared to (one or more) ideal features to identify differences. When the identified difference is greater than a threshold, the (one or more) temperature models are updated accordingly.

Description

Background Art

[0001] Microelectromechanical systems (MEMS) are miniaturized mechanical and / or electromechanical components typically made using micromachining techniques. The physical dimensions of MEMS devices can range from well below a micrometer to several millimeters.

[0002] One type of application of MEMS involves optical switches and micromirrors to redirect or modulate light beams. In some embodiments, the micromirrors can be accurately controlled by MEMS elements to swing back and forth at a given frequency. In some embodiments, one or more laser beams (e.g., red, green, and blue beams) of different intensities can be combined into a single beam, and the single beam is relayed to one or more MEMS scanning mirrors. Then, (one or more) MEMS scanning mirrors swing back and forth, scanning the beam in a raster pattern to project an image on a display. As long as the scanning rate is fast enough, the raster scanned laser beam can produce the impression of a stable image using a single scanning point. These (one or more) images may then produce the impression of motion. Such (one or more) projected images or motion pictures can be created by synchronously modulating the position of one or more lasers and scanning laser beams.

[0003] However, each MEMS device may have a unique set of temperature characteristics, causing each MEMS device to behave slightly differently at different temperatures. For example, a MEMS scanning device may include one or more MEMS scanning mirrors, each of which may have its own temperature characteristics that can cause the projected image to be distorted or out of specification at high or low temperatures. Furthermore, during the lifetime of the MEMS scanning device, the specific temperature characteristics may gradually change, causing the performance of the MEMS device to deteriorate over time.

[0004] The subject matter claimed herein is not limited to embodiments that solve any disadvantages or that operate only in environments such as described above. Rather, this background is provided merely to illustrate one exemplary technology area in which some embodiments described herein may be practiced. Summary of the invention

[0005] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.

[0006] Each MEMS device may have various unique temperature characteristics, so that each MEMS device may behave slightly differently at different temperatures. For example, a MEMS scanning device may include one or more MEMS scanning mirrors, each of which has its own temperature characteristics, which can cause the projected image to deform or exceed specifications at high or low temperatures. The principles described in this article solve the above problems by using machine learning to build (one or more) temperature models customized for each MEMS scanning device, and enable the MEMS scanning device to use (one or more) customized temperature models to modify its behavior based on the current temperature. In addition, during the life of the MEMS scanning device, specific temperature characteristics may gradually change. The principles described in this article also allow the MEMS scanning device to update (one or more) temperature models during its life using feedback obtained from a display observation camera.

[0007] Embodiments described herein relate to a MEMS scanning device. The MEMS scanning device includes: one or more laser projectors, one or more MEMS scanning mirrors, a display, a thermometer, and a display viewing camera. The one or more laser projectors are controlled by a laser driver to project a laser beam. The one or more MEMS scanning mirrors are controlled by a MEMS driver to scan (i.e., turn) and reflect the laser beam while scanning to project a raster scan on the display. The thermometer is configured to detect the temperature of the MEMS scanning device. The display viewing camera is configured to capture an image of a predetermined area of ​​the display.

[0008] The MEMS scanning device also includes one or more computer-readable media that store one or more temperature models. Each of the one or more temperature models is custom built using machine learning to identify a relationship between the temperature of a particular MEMS scanning device and at least one of the following: (1) the amplitude of at least one of the MEMS scanning mirrors, or (2) the phase shift of at least one of the MEMS scanning mirrors. In some cases, the one or more temperature models are built during manufacturing and stored in the MEMS scanning device during the manufacturing process. Alternatively or additionally, the one or more temperature models are built by the MEMS device itself.

[0009] As briefly mentioned above, during the life of the MEMS scanning device, specific temperature characteristics may gradually change. The MEMS scanning device is configured to update (one or more) existing temperature models using feedback received from a display viewing camera. First, the MEMS scanning device uses a thermometer to detect the current temperature. Similarly, the MEMS scanning device projects one or more predetermined patterns at a predetermined area of ​​the display based on the current temperature and (one or more) temperature models. The display viewing camera captures one or more images of (one or more) projected patterns. Then, the MEMS scanning device extracts one or more features from the captured images. The one or more features may include (but are not limited to) the location of one or more points or one or more lines. Then, the (one or more) features extracted from the captured one or more images are compared with one or more ideal features to determine whether there is a sufficient difference between the extracted (one or more) features and the (one or more) ideal features. In response to identifying a sufficient difference between the features extracted from the captured (one or more) images and the (one or more) ideal features, at least one of the (one or more) temperature models is then updated.

[0010] In some embodiments, updating the one or more temperature models includes: in response to identifying a sufficient difference between the feature(s) extracted from the captured one or more images and the one or more ideal features, adjusting at least one of (1) one or more control parameters of the MEMS actuator or (2) one or more control parameters of the laser actuator to mitigate the difference. At least one of the one or more temperature models is then updated based on the current temperature and the adjustment of at least one control parameter of the MEMS actuator and / or the laser actuator.

[0011] In some cases, the updating of the one or more models may be manually triggered by a user. Alternatively or additionally, the updating may be automatically performed at predetermined time intervals. In some embodiments, the predetermined time interval may be adjusted based on the amount of difference or adjustment in the previous update. For example, when the adjustment to at least one control parameter of the MEMS driver and / or the laser driver is greater than a threshold, the predetermined time interval may be decreased. Alternatively or additionally, when the adjustment to at least one control parameter of the MEMS driver and / or the laser driver is less than a threshold, the predetermined time interval may be increased.

[0012] In some embodiments, the one or more temperature models may include a model representing the relationship between temperature and the amplitude of each MEMS scanning mirror in the MEMS scanning mirror. For example, the MEMS scanning device may include two single-dimensional scanning mirrors (i.e., a first scanning mirror and a second scanning mirror). The first scanning mirror scans in a first dimension with a first amplitude, and the second scanning mirror scans in a second dimension with a second amplitude. The first dimension and the second dimension intersect (e.g., are orthogonal to each other). The first scanning mirror and the second scanning mirror are configured to relay a laser beam to project a raster scan onto a display.

[0013] The one or more models include at least: (1) a model representing a relationship between temperature and a first amplitude corresponding to a first scanning mirror, or (2) a model representing a relationship between temperature and a second amplitude corresponding to a second scanning mirror. In some embodiments, at least one of the one or more features is used to update the model representing the relationship between temperature and the first amplitude; and at least one of the one or more features is used to update the model representing the relationship between temperature and the second amplitude.

[0014] In some embodiments, one of the MEMS scanning mirrors is a dual-phase scanning mirror. The dual-phase scanning mirror scans the laser beam back and forth bidirectionally during a scanning cycle, and the point projected in the forward direction during the scanning cycle and the corresponding point projected in the backward direction during the scanning cycle are consistent in the dual-phase scanning direction. The phase of the bidirectional scanning may move slightly at different temperatures, causing the MEMS mirror and the laser projector to be out of phase. When the MEMS mirror and the laser projector are out of phase, the point projected in the forward direction and the corresponding point projected in the backward direction will no longer be consistent. When the points projected in the two directions are inconsistent, the (one or more) projected images may become blurred or distorted. The phase shift of the bidirectional scanning can also be modeled by one of the one or more models that represent the relationship between temperature and the phase shift of the bidirectional scanning.

[0015] In order to update the model representing the temperature and the phase shift of the bidirectional scanning, at least one of the one or more projected predetermined patterns is configured to show whether a point projected during scanning in the forward direction and a corresponding point projected during scanning in the backward direction substantially coincide with each other. When the points projected during scanning in the forward direction and the backward direction do not completely coincide with each other, the model representing the relationship between the temperature and the phase shift of the bidirectional scanning is updated.

[0016] In some embodiments, one or more predetermined patterns may include a first pattern and a second pattern, each of which includes a first line of points and a second line of points. Each of the first line and the second line in the first pattern and the second pattern is in a direction that intersects the scanning dimension of the dual-phase scanning mirror (for example, orthogonal to the scanning dimension). The first line of points in the first pattern and the second line of points in the second pattern are projected during scanning in a forward direction; and the second line of points in the first pattern and the first line of points in the second pattern are projected during scanning in a backward direction. Each of the first line and the second line of the first pattern corresponds to the corresponding first line and second line of the second pattern. The projected first pattern and the projected second pattern are compared to determine whether they overlap. In response to determining that the projected first pattern and the projected second pattern do not overlap, the MEMS scanning device adjusts the phase of the laser driver so that the projected first pattern and the projected second pattern overlap. Based on the adjustment of the phase of the laser driver, the MEMS scanning device is able to update a temperature model representing the relationship between temperature and the phase shift of the dual-phase scanning mirror.

[0017] In some embodiments, the one or more MEMS scanning mirrors include: a fast scanning mirror configured to scan in a first dimension; and a slow scanning mirror configured to scan in a second dimension. The first dimension and the second dimension intersect (e.g., are orthogonal to each other). In some embodiments, the fast scanning mirror is a dual-phase scanning mirror that is controlled by a sinusoidal signal to scan in two directions; and the slow scanning mirror is a single-phase scanning mirror that is controlled by a sawtooth signal to scan in only one direction. In such a case, the one or more temperature models may include at least: (1) a model representing the relationship between temperature and the amplitude of the fast scanning mirror, (2) a model representing the relationship between temperature and the amplitude of the slow scanning mirror, and / or (3) a model representing the relationship between temperature and the phase shift of the fast scanning mirror.

[0018] Additional features and advantages will be set forth in the following description, and in part will be apparent from the description, or may be learned through practice of the teachings herein. The features and advantages of the present invention may be realized and obtained through the instruments and combinations particularly pointed out in the appended claims. The features of the present invention will become more apparent from the following description and the appended claims, or may be understood through practice of the present invention as described below. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to describe the manner in which the above and other advantages and features can be obtained, a more particular description of the subject matter briefly described above will be rendered by reference to specific embodiments illustrated in the accompanying drawings. Understanding that these drawings depict only typical embodiments and are therefore not to be considered limiting in scope, the embodiments will be described and explained with additional specificity and detail through the use of the accompanying drawings, in which:

[0020] Figure 1A An exemplary architecture of a MEMS scanning device 100 is illustrated;

[0021] Figure 1B An exemplary MEMS scanning device including two one-dimensional scanning mirrors is illustrated;

[0022] Figure 1C An exemplary sequence of raster scans is illustrated;

[0023] Figure 1D illustrates an exemplary control signal for a fast scanning mirror and an exemplary control signal for a slow scanning mirror;

[0024] Figure 2 illustrates an exemplary MEMS mirror actuated by an electromagnetic field;

[0025] Figure 3 illustrates an exemplary MEMS mirror driven by a piezoelectric element;

[0026] Figure 4 illustrates a line graph showing a substantially linear relationship between drive current or voltage and deflection angle of a MEMS mirror;

[0027] Figure 5A illustrates a line graph showing a substantially linear relationship between temperature and coil resistance of an electromagnetic MEMS mirror;

[0028] Figure 5B illustrates a line graph showing a substantially linear relationship between temperature and magnetic force generated by a constant current flowing through a coil;

[0029] Figure 5C illustrates a line graph showing a substantially linear relationship between temperature and deflection angle of a MEMS mirror;

[0030] Figure 6 illustrates an exemplary machine learning network that may be implemented to learn the relationship between temperature and amplitude of a MEMS mirror;

[0031] Fig. 7A illustrates a diagram representing an exemplary bi-phase scanning cycle;

[0032] Figure 7B illustrates a graph representing an exemplary phase shift of a bi-phase scanning cycle;

[0033] Figure 8 illustrates an exemplary set of patterns that may be used to detect phase shifts of a dual phase scanning mirror;

[0034] Fig. 9 illustrates an exemplary machine learning network that can be used to learn the relationship between temperature and phase shift of a dual-phase scanning mirror;

[0035] Fig.10 An exemplary head mounted device including two MEMS scanning devices is illustrated;

[0036] Fig.11 A flow chart illustrating an exemplary method for building and / or updating one or more temperature models of a MEMS scanning device;

[0037] Fig.12 A flow chart illustrating an exemplary method for updating one or more existing temperature models of a MEMS scanning device; and

[0038] Fig.13 An exemplary computing system is illustrated in which the principles described herein may be employed. DETAILED DESCRIPTION

[0039] The embodiments described herein are directed to building (one or more) custom temperature models for a MEMS scanning device, and using the (one or more) custom temperature models to adjust (one or more) control parameters of the MEMS scanning device to achieve better display results. The embodiments described herein also allow the (one or more) custom temperature models to be updated during the life of the MEMS scanning device.

[0040] Figure 1A An exemplary architecture of a MEMS scanning device 100 (hereinafter also referred to as a "device") is illustrated, which is a computing system including one or more processors 110 and a computer-readable hardware storage device 120. The device 100 also includes one or more laser projectors 148, one or more MEMS scanning mirrors 136 and / or 138, a display 160, and a thermometer 150.

[0041] One or more laser projectors 148 are controlled by the laser driver 140 to project a laser beam onto one or more MEMS scanning mirrors 136 and / or 138. In some embodiments, only one laser projector is implemented to project (one or more) grayscale images. In some embodiments, one or more laser projectors 148 include multiple laser projectors, each of which projects a laser beam of a different color. Multiple color laser beams are merged into a single color beam for projecting (one or more) color images. For example, one or more laser projectors 148 may include a red laser projector 148R, a green laser projector 148G, and a blue laser projector 148B. The red laser projector 148R is configured to project a red laser beam, the green laser projector 148G is configured to project a green laser beam, and the blue laser projector 148B is configured to project a blue laser beam. The laser driver 140 can use three different control signals or parameters Vr 142, Vg 144 and Vb 146 to control the intensity (i.e., brightness) of each of the laser projectors 148R, 148G and 148B at different times. Three different colored laser beams (each of which is projected at a specific intensity) are merged into a single laser beam to produce the desired color. For example, if each of the red, green and blue projectors 148R, 148G and 148B has 256 intensity levels, a total of 16,777,216 (=256×256×256) colors can be produced.

[0042] The single laser beam is projected onto one or more MEMS scanning mirrors 136 and / or 138. The one or more MEMS scanning mirrors 136 and / or 138 are controlled by a MEMS actuator 130. The MEMS actuator 130 causes each of the MEMS actuators to deflect back and forth with an amplitude and frequency. The amplitude and frequency of each MEMS mirror 136 or 138 can be controlled by control signals or parameters V1 132 and V2 134. The deflection of the MEMS mirrors 136 and / or 138 causes the single laser beam to project a raster scan on the display 160.

[0043] In some embodiments, each of the MEMS mirrors 136 and 138 is a single-dimensional scanning mirror that scans in a single dimension. The MEMS mirror 136 is configured to scan in a first dimension (e.g., horizontally), while the MEMS mirror 138 is configured to scan in a second dimension (e.g., vertically). The first dimension intersects (e.g., is orthogonal to) the second dimension. A single laser beam is first projected onto one of the MEMS mirrors 136 or 138. The laser beam is then reflected from one MEMS mirror 136 or 138 to the other MEMS mirror 138 or 136. The laser beam is then reflected from the other MEMS mirror 138 or 136 to the display 160 to generate a two-dimensional raster scan.

[0044] Figure 1B Also illustrated is an exemplary MEMS scanning device 100B that projects a raster scan onto a display 160 using two one-dimensional scanning mirrors 136 and 138. Figure 1B As illustrated in FIG. 1 , the laser projector 148 first projects a single light beam onto the MEMS mirror 136. The MEMS mirror 136 is configured to deflect about the axis 133 to scan a one-dimensional line. The one-dimensional line is reflected from the MEMS mirror 136 onto the MEMS mirror 138 deflected about the axis 135. The axis 133 intersects the axis 135 (e.g., is orthogonal to each other), so that the one-dimensional line received by the MEMS mirror 138 is scanned into a two-dimensional raster scan image, which is ultimately projected onto the display 160.

[0045] Figure 1C An exemplary sequence of raster scanning 100C is shown. Figure 1C As shown in FIG. 1 , by scanning back and forth in a first dimension with a first mirror (e.g., Figure 1B The first mirror 136 of the display 160 first draws the first cycle 102C at the top of the display 160. At the same time, the second mirror (e.g., Figure 1B The second mirror 138 of the display is gradually deflected so that the line scanned by the first mirror is gradually moved in the second dimension. Similarly, the second period 104C and the third period 106C are drawn continuously by the first mirror under the first period 102C and the gradually deflected second mirror, and so on until the last period or line 108C is drawn on the display. Then, the second mirror is deflected all the way back to the starting position of the period 102C to redraw the next frame of the image. As shown in Figure 1C As shown in the figure, the first mirror scans at a much faster frequency (horizontally) than the second mirror (vertically), because the second mirror has just completed the first cycle after the first mirror has scanned all cycles 102C-108C. In this way, the first mirror is also called a fast scanning mirror, and the second mirror is also called a slow scanning mirror.

[0046] MEMS mirrors often have a resonant frequency that is determined by their mass, structure, and spring constant. In some embodiments, the deflection frequency of a fast mirror is controlled to be close to the resonant frequency of the mirror, thereby obtaining a large mirror deflection angle with a small current.

[0047] Figure 1D An exemplary control signal V1 132 for the fast scan mirror and an exemplary control signal V2 134 for the slow scan mirror are shown. Figure 1D , the fast scanning mirror can be controlled by a basic sinusoidal signal at a first frequency, while the slow scanning mirror can be controlled by a basic sawtooth signal at a second frequency. The first frequency is much faster than the second frequency. Likewise, in some embodiments, the fast scanning mirror can be a dual-phase scanning mirror that scans back and forth bidirectionally to project lines, so that an image is drawn by lines projected in both directions. The slow scanning mirror is often a single-phase scanning mirror that scans in only one direction, so that the laser beam is projected only during the first edge 102D of the sawtooth signal 134. During the second edge 104D of the sawtooth signal 134, the laser projector 148 can be turned off so that no laser beam is projected onto the display 160.

[0048] In some embodiments, two-dimensional raster scanning can also be achieved by a single two-dimensional scanning mirror. The two-dimensional scanning mirror scans in two dimensions (i.e., the first dimension and the second dimension). The first dimension intersects (e.g., orthogonal) with the second dimension. The MEMS driver 130 can use a first control signal or parameter V1 132 to control the deflection of the mirror in the first dimension, and use a second control signal V2 134 to control the deflection of the mirror in the second dimension. In such a case, a single laser beam is projected onto a single two-dimensional scanning MEMS mirror, and the single MEMS mirror itself can reflect the laser beam onto the display 160 to generate a two-dimensional raster scan. The principles described herein are applicable to embodiments that implement two single-dimensional mirrors and / or embodiments that implement a single two-dimensional mirror.

[0049] Note that the laser projector 148 and the MEMS mirror(s) 136 and / or 138 must always be substantially synchronized to project a sharp image of the raster scan. In other words, each of the control parameters (one or more) V1 132, V2 134, Vr 142, Vg 144, Vb 146 must always be substantially synchronized.

[0050] However, because each MEMS mirror 136 or 138 and / or specific (one or more) other components of the device 100 may have various unique temperature characteristics, the MEMS mirror 136 or 138 and / or (one or more) other components may behave slightly differently when the temperature changes, causing the raster scan to be distorted or causing the laser projector 148 and (one or more) MEMS mirrors 136 and / or 138 to be out of sync.

[0051] The principles described herein solve the above-mentioned problems by using machine learning to build one or more temperature models 122 customized for each device 100 and storing the one or more temperature models 122 in the storage device 120 of the device 100. Each of the one or more temperature models 122 is custom built using machine learning to identify the relationship between the temperature of a particular MEMS scanning device 100 and (1) the amplitude of at least one of the MEMS scanning mirrors 136 or 138 or (2) the phase shift of at least one of the MEMS scanning mirrors 136 or 138. In some cases, the one or more temperature models 122 can be built at the time of manufacturing and stored in the device 100 during the manufacturing process. Alternatively or additionally, the one or more temperature models 122 can be built by the MEMS device 100 itself when the user controls the MEMS device 100.

[0052] Back to reference Figure 1A , the device 100 also includes a thermometer 150. The thermometer 150 is configured to detect the current temperature of the device 100. The thermometer 150 can be installed next to the MEMS scanning mirror 136 or 138, or installed anywhere in the housing of the device 100. In some embodiments, a separate thermometer 150 can be implemented for each MEMS scanning mirror 136 or 138 to be able to more accurately obtain the current temperature of the corresponding MEMS scanning mirror 136 or 138.

[0053] Based on the one or more temperature models 122 and the current temperature, the processor 110 of the device adjusts one or more control parameters (including but not limited to: V1 132, V2 134, Vr 142, Vg 144, VbB 146) of the MEMS driver 130 and / or the laser driver 140 to calibrate the device 100 based on the current temperature. In some embodiments, the thermometer 150 updates the current temperature at a predetermined time interval (e.g., 5 minutes). When the current temperature is sufficiently different from the previous temperature (i.e., the difference between the current temperature and the previous temperature is greater than a threshold), the one or more control parameters can be adjusted based on the temperature model(s).

[0054] Additionally, as briefly mentioned above, certain temperature characteristics may gradually change during the lifetime of the MEMS scanning device 100. The principles described herein also allow the MEMS scanning device 100 to update the temperature model(s) 122 during its lifetime using feedback obtained from the display viewing camera 170.

[0055] As in Figure 1A As illustrated in , in some embodiments, the device 100 also includes a display viewing camera 170, which is configured to capture (one or more) images of a predetermined area of ​​the display 160. The device 100 is configured to project one or more predetermined patterns 124 (which can be stored at the storage device 120 of the device 100) at a predetermined area of ​​the display 160. At the same time, the thermometer 150 also obtains the current temperature. Therefore, one or more predetermined patterns 124 are projected based on one or more temperature models 122. Then, the display viewing camera 170 captures one or more images of (one or more) projected patterns, which are used to extract one or more features. The one or more features include (but are not limited to) one or more points or one or more lines. Then, the captured one or more features are compared with one or more predicted ideal features to determine whether there is a sufficient difference between the (one or more) features extracted from the captured (one or more) images and the predicted (one or more) ideal features. In response to identifying sufficient differences between features extracted from the captured image(s) and the ideal feature(s), the processor 110 of the device 100 may cause the MEMS actuator 130 and / or the laser driver 140 to adjust the control parameter(s) V1 132, V2 134, Vr 142, Vg 144, and / or Vb 146 to mitigate the differences. Based on the adjustments, the processor 110 may then cause at least one of the temperature model(s) 122 to be adjusted.

[0056] In some cases, the update of the temperature model(s) may be manually triggered by a user. In some cases, the device 100 automatically attempts to update its temperature model 122 at a predetermined frequency (e.g., 5 minutes, 5 days, etc.). In some embodiments, the predetermined frequency may be adjusted based on the adjustment(s) made in the current update. For example, when the adjustment is greater than a predetermined threshold, the processor 110 of the device 100 may increase the update frequency; and when the adjustment is less than a predetermined threshold, the processor 110 of the device may decrease the update frequency.

[0057] Since the principles described in this article are related to MEMS mirrors, reference Figure 2A brief description of a MEMS mirror is provided in Figure 5. A MEMS mirror is a miniature mirror that incorporates MEMS technology. Several different MEMS technologies can be implemented to accurately control the mirror, including (but not limited to) using electromagnetic fields, piezoelectric elements, and / or electrostatic forces.

[0058] Figure 2 An exemplary MEMS mirror 200 is illustrated that is actuated by an electromagnetic field 210. Figure 2 As illustrated in FIG. 1 , the MEMS mirror 200 includes a mirror 220 suspended in an electromagnetic field 210 by a torsion bar 222. The electromagnetic field 210 may be generated by a magnet (not shown). Within the generated electromagnetic field 210, a current is caused to flow in a coil 230 surrounding the mirror 220. Based on the Fleming rule, the current (represented by arrow 232) generates a Lorentz force 242 at the left edge of the mirror 220; at the same time, the current (represented by arrow 236) generates a Lorentz force 244 in the opposite direction at the right edge of the mirror 220. The opposing forces 242 and 244 drive the mirror 220 to deflect. In this way, the deflection of the mirror 220 can be controlled by the current flowing through the coil 230 and / or the voltage V 250 applied to the coil 230.

[0059] MEMS mirror 200 is a one-dimensional mirror that includes a single torsion bar 222 that allows mirror 220 to deflect along a single line. In some embodiments, a second torsion bar may be implemented to allow the mirror to deflect in two dimensions.

[0060] Figure 3 Another exemplary MEMS mirror 300 is illustrated that is actuated by a piezoelectric element 320. When a voltage 330 is applied to two opposing sides of the piezoelectric element 320, the piezoelectric element 320 deforms. The mirror 310 attached to the piezoelectric element 320 will deflect as the piezoelectric element 320 deforms.

[0061] Figure 4 A line graph 400 showing a substantially linear relationship between a drive current or voltage and a deflection angle of a MEMS mirror is illustrated. The drive current or voltage may correspond to Figure 2 The driving voltage 250 and / or the driving current 232, 236 and / or Figure 3 The driving voltage is 330. Figure 4 As illustrated in FIG. 4 , the horizontal axis 410 represents the drive current or voltage, and the vertical axis 420 represents the deflection angle of the MEMS mirror. Line 430 represents the substantially linear relationship between the drive current and the deflection angle. Generally speaking, the greater the drive current or voltage, the greater the deflection angle. When the drive current or voltage changes its direction (e.g., becomes negative), the deflection angle also changes its direction.

[0062] The MEMS mirror 200 or 300 may correspond to Figure 1AThe voltage V 250, 330 or the current 232, 236 may correspond to the control parameter(s) V1 132 and / or V2 134. Thus, the MEMS driver 130 may be configured to adjust the voltage V 250 or 330 or the current 232 or 236 to control the deflection of the MEMS mirror 220.

[0063] In the MEMS mirror 200 or 300, the coil resistance, magnetic force of the magnet, resonant frequency, piezoelectric elements, and certain other components may have their own temperature characteristics. The principles described herein take into account the temperature characteristics of each individual MEMS mirror to allow for more accurate control of the optical deflection angle of the mirror over a wide temperature range.

[0064] For example, in electromagnetic MEMS mirrors (e.g. Figure 2 In the MEMS mirror 200 of FIG. 1 , when the temperature increases, the coil resistance (e.g., the resistance of coil 230) increases, which reduces the current flowing through the coil. The reduced current, in turn, reduces the optical deflection angle. In addition, even if the current flowing through coil 230 remains unchanged, the magnetic force decreases when the temperature decreases, which in turn reduces the optical deflection angle.

[0065] Figure 5A A line graph 500A showing a substantially linear relationship between temperature and coil resistance is illustrated. Figure 5A The horizontal axis represents temperature, and Figure 5A The vertical axis in FIG. 5 represents coil resistance. Line 510A represents a substantially linear relationship between temperature and coil resistance. The coil resistance increases as temperature increases. For example, as Figure 5A As illustrated in , the temperature T2 is greater than the temperature T1 , and the coil resistance at the temperature T2 is greater than the coil resistance at the temperature T1 .

[0066] Figure 5B Illustrated is a line graph 500B showing a substantially linear relationship between temperature and magnetic force when the current flowing through the coil remains constant. Figure 5B The horizontal axis in FIG. 5 represents temperature, and the vertical axis represents magnetic force. Line 510B represents a substantially linear relationship between temperature and coil resistance. Figure 5B As illustrated in , the temperature T2 is greater than the temperature T1, and the magnetic force at the temperature T2 is greater than the magnetic force at the temperature T1.

[0067] Coil resistance and magnetic field are just two exemplary factors that affect the deflection angle or amplitude of the mirror. Additional factors may also have an effect. For example, the resonant rate of the mirror may also change when the temperature changes. As another example, in a piezoelectric mirror, the piezoelectric effect may also be affected by temperature.

[0068] Due to these various temperature-related factors, the deflection angle of a MEMS mirror generally decreases as temperature increases (assuming the control parameter(s) such as drive current or voltage remain constant). Figure 5C A line graph 500C showing a substantially linear relationship between temperature and the deflection angle of the MEMS mirror is illustrated. Figure 5C The horizontal axis in represents temperature, and the vertical axis represents deflection angle. Line 510C represents a substantially linear relationship between temperature and deflection angle of the MEMS mirror when (one or more) control parameters remain constant. Figure 5C As illustrated in , the temperature T2 is greater than the temperature T1, and the deflection angle at the temperature T2 is smaller than the deflection angle at the temperature T1.

[0069] When the MEMS mirror is deflected at a larger angle, the laser beam reflected by the MEMS mirror will cover a larger width or area on the display. Therefore, the deflection angle of the MEMS mirror can also be called the amplitude. Figure 5C As shown in , the amplitude (or deflection angle) of the MEMS mirror has a substantially linear relationship with temperature. However, since each MEMS mirror is slightly different from the other mirrors, there may be a different linear relationship for each MEMS mirror.

[0070] In order to take into account the individuality of each MEMS mirror, the principles described in this article implement machine learning to observe each individual MEMS scanning device to obtain the amplitude-temperature relationship of each MEMS mirror. In some embodiments, one or more predetermined patterns are projected onto the MEMS scanning device. One or more predetermined patterns may include (one or more) patterns with multiple lines in the scanning direction (e.g., horizontal or vertical). For example, a predetermined pattern may include multiple horizontal white lines; and another predetermined pattern may include multiple vertical white lines. In some embodiments, a single set of patterns is projected, and one or more features (e.g., one or more points or lines) are extracted from the projection pattern. In some embodiments, (one or more) projection patterns may extend across the width of the display. In some embodiments, (one or more) projection patterns may be contained in a predetermined area of ​​the display.

[0071] The current temperature and the projection pattern(s) are monitored over a period of time (e.g., 20-25 minutes). The current temperature may be obtained by the thermometer 150 of the device 100. The projection pattern(s) may be monitored by the display viewing camera 170 of the device 100 or by an external camera (not shown). In some embodiments, the temperature and the projection image(s) are collected at a predetermined frequency (e.g., every second). In some embodiments, the projection image(s) may be collected only when the temperature has changed sufficiently.

[0072] Figure 6An exemplary machine learning network 600 that can be implemented to "learn" the relationship between temperature and the amplitude of the MEMS mirror 136 or 138 is illustrated. Temperature data 610 and (one or more) captured images of (one or more) projection patterns 620 are fed into a linear regression network 630. For each captured image of the projection pattern 620, there is a first width 622 (e.g., horizontal dimension) of the image and a second width 624 (e.g., vertical dimension) of the image. Each first width 622 or second width 624 and corresponding temperature 610 form a data pair. All data pairs including a first width 622 and a corresponding temperature 610 form a first data set; and all data pairs including a second width 624 and its corresponding temperature 610 form a second data set.

[0073] Based on the first data set and the second data set, a linear relationship between temperature and the first width 622 and the second width 624 can be obtained. Since each of the first width 622 and the second width 624 corresponds to the first amplitude or the second amplitude of the corresponding MEMS mirror 136 or 138, a linear relationship 640 between temperature and the first amplitude and a linear relationship 650 between temperature and the second amplitude can also be obtained. Such relationships 640 and 650 can then be stored as two temperature models in the (one or more) temperature model 122 at the storage device 120 of the MEMS scanning device 100. The MEMS scanning device 100 can then use the (one or more) temperature model 122 to adjust its control signals or parameters V1 132 and / or V2 134 to obtain the appropriate amplitude of each MEMS mirror 136 or 138 based on the current temperature.

[0074] In addition to the temperature characteristics related to the amplitude, when dual-phase scanning is implemented, the synchronization of the dual-phase scanning is also related to the temperature. Figures 1B to 1D As briefly described, there may be a fast scan mirror 136 and a slow scan mirror 138. The fast scan mirror 136 can use a substantially sinusoidal signal to draw a line in the horizontal direction. Bi-phase scanning refers to using these two phase scans (i.e., forward scan and backward scan) to draw a line on the display. When bi-phase scanning is implemented, both the line drawn in the forward scan and the line drawn in the backward scan are used to project an image on the display.

[0075] Fig. 7A A diagram representing an exemplary bi-phase scanning cycle 700A is illustrated. Horizontal axis 710 represents the amplitude of a fast scanning mirror (e.g., mirror 136), while vertical axis 720 represents time. During a bi-phase scanning cycle, the bi-phase scanning mirror scans back and forth in a horizontal direction. Line 730 may correspond to Figure 1CAny cycle in cycles 102C to 108C of . Note that any point on line 730 corresponds to two points drawn during the same cycle: (1) one point drawn during the forward scan, and (2) one point drawn during the backward scan. Although the two corresponding points should be considered different and can typically be drawn using different color codes (i.e., different intensities of the laser), if the biphasing is correct, the two corresponding points should be consistent on the horizontal axis, but still separated along the other axis of the other mirror. When the biphasing is incorrect, such corresponding points will be separated along these two axes. For example, when the biphasing is correct, two corresponding points 732 are drawn at times T1 and T2.

[0076] Ideally, the amplitude of the scanning mirror corresponds closely to the control signal, so that when the sinusoidal control signal is applied to the scanning mirror, a corresponding sinusoidal amplitude is generated. However, in reality, there is always a phase shift or delay between the control signal and the amplitude of the scanning mirror. The amount of phase shift is temperature dependent. Generally speaking, the higher the temperature, the greater the phase delay that may occur. The laser driver 140 must be synchronized with the phase of the mirror so that the projected image is clear and not distorted.

[0077] Figure 7B An example of the phase shift of a MEMS scanning mirror is shown. Figure 5B As illustrated in , the dashed line 750 illustrates the amplitude of the MEMS mirror shifted from the amplitude 740 (drawn in solid line). Assuming that the laser driver 140 is projecting a laser beam based on the amplitude 740, in order to draw a corresponding point at position 732, the laser beam is projected at times T1 and T2. However, if in fact the amplitude has moved to the dashed line 750, then at time T1, the first point will be drawn at point 734; and at time T2, the second point will be drawn at 736. Obviously, in this exaggerated example, the two points 734 and 736 (assuming they are consistent on the horizontal axis) are far apart. This will obviously affect the resulting image projected on the display. The same principle applies when there is a slight phase shift when the temperature changes, and even when there is a slight phase shift, the dual-phase scanning may cause two corresponding points to be projected in two slightly different places in the dual-phase scanning direction, resulting in a blurred and / or deformed image.

[0078] The relationship between temperature and the phase shift of the scanning mirror can also be learned via a machine learning network by projecting one or more predetermined patterns on the display 160 of the device and monitoring the projected images for a period of time. The one or more predetermined patterns here are different from the patterns used to learn the relationship between temperature and amplitude. Here, the one or more predetermined patterns may include two lines of points, each of which is parallel to the slow scanning direction (i.e., intersecting the fast scanning direction). In the first pattern, the first line is drawn only by forward scanning, and the second line is drawn only by backward scanning. In the second pattern, the first line is drawn only by backward scanning, and the second line is drawn only by forward scanning. A collection of (one or more) images is taken for each of the projected first and second patterns. When the phase of the amplitude and the phase of the laser driver are synchronized, the two images will overlap. However, when there is a phase shift, the two images will not overlap. Specifically, in one image, the two lines are farther apart; while in the other image, the two lines are closer to each other.

[0079] Figure 8 An exemplary set of first pattern 810 and second pattern 820 that can be projected on display 160 to identify a phase shift of scanning mirror 136 or 138 is illustrated. An external camera or internal camera 170 of device 100 can take a picture of the two images and compare the two images to determine if a phase shift exists. Figure 8 As illustrated in , the two lines projected in the first pattern 810 are farther apart than the two lines projected in the second pattern 820, which indicates a phase shift between the amplitude of the MEMS mirror and the phase of the laser driver. At the same time, the thermometer 150 also obtains the current temperature.

[0080] The process of collecting temperature data and image data is repeated for a sufficient period of time (e.g., 20-25 minutes) to collect a sufficient data set. The collected data set is then fed into the machine learning network. Fig. 9An exemplary machine learning network 900 is illustrated. Temperature data 910 and captured image(s) of the projection pattern(s) (e.g., patterns 810, 820) are fed into a linear regression model 930. For each set of captured first and second patterns, a corresponding phase shift can be determined. Based on the phase shift and corresponding temperature 910 of each set of captured images 920, a linear relationship 950 between temperature and phase shift can be obtained. Such a relationship 950 can then be stored at the storage device 120 of the MEMS scanning device 100 as one of the temperature models 122. The MEMS scanning device 100 can then use the temperature model(s) 122 to adjust its control signals or parameters Vr 142, Vg 144, Vb 146 to synchronize the proper phase of the laser projector 148 with the phase of the MEMS mirror based on the current temperature.

[0081] Furthermore, as discussed above, the temperature characteristics of each MEMS scanning device 100 may gradually change during the life of the device 100. Thus, over time, previously stored temperature model(s) may become obsolete. The principles described herein also address this issue by receiving feedback from the display viewing camera 170 and updating the temperature model(s) based on the received feedback.

[0082] The process of updating the temperature model(s) is similar to but not identical to the process of building the temperature model(s). When updating the temperature model(s), the current temperature is obtained by a thermometer, and one or more predetermined image patterns are projected onto the display 160 based on the current temperature and the existing temperature model. Then, the display viewing camera 170 captures (one or more) images of the (one or more) projected patterns. Note that when updating the temperature model(s), it is not necessary to obtain a large data set across the entire temperature range as during model building. Instead, the captured (one or more) images are used here to extract one or more features, and the extracted features are compared with one or more ideal features. If the extracted (one or more) features are substantially the same as the (one or more) ideal features, the (one or more) existing temperature model(s) 122 are still quite accurate. If the extracted (one or more) features are not substantially the same as the (one or more) ideal features, the (one or more) existing temperature model(s) 122 need to be adjusted accordingly.

[0083] In some embodiments, the predetermined pattern(s) are projected only at a small portion of the display 160 and the camera 170 is configured to observe only a small portion of the display, so that a smaller camera can be implemented and, as such, the update process will not significantly interfere with the user's use of the device 100.

[0084] In some embodiments, the update process can be manually triggered by a user. In some embodiments, the update process can be performed at a predetermined frequency (e.g., every 5 minutes, every day, every month, etc.). Depending on the results of the update, the device 100 can change the predetermined frequency to be higher or lower. For example, during the update process, if no adjustment is made (or the adjustment is less than a predetermined threshold), the device 100 can reduce the update frequency; on the other hand, if a significant adjustment is made (or the adjustment is greater than a predetermined threshold), the device 100 can increase the update frequency.

[0085] Furthermore, the principles described herein are applicable not only to any MEMS scanning device, but also to any device that includes one or more MEMS scanning devices. For example, a single MEMS scanning device can be implemented in a stand-alone projector; alternatively, more than one MEMS scanning device can be embedded in a single device (e.g., a head-mounted device).

[0086] Fig.10 An exemplary head mounted device 1000 including two MEMS scanning devices 1010 and 1020 is illustrated. Each of the MEMS scanning devices 1010 and 1020 is configured to project (one or more) images onto a corresponding display 1012 or 1022. Each of the devices 1010 and 1020 may correspond to the MEMS scanning device 100 of FIG. 1 and have its own temperature characteristics. Thus, for each of the devices 1010 and 1020, a separate set of (one or more) custom temperature models may be constructed and stored at the head mounted device 1000. Each of the MEMS scanning devices 1010 and 1020 is not only able to use its own set of (one or more) temperature models to adjust (one or more) control parameters based on the current temperature, but is also able to update its own set of (one or more) temperature models during the life of the head mounted device 1000. In some embodiments, a shared storage device, processor, and / or thermometer may be used to store and update the temperature model(s) for both devices 1010 and 1020. Alternatively, separate storage devices, processors, and / or thermometers may be coupled to and dedicated to each device 1010 or 1020.

[0087] The following discussion now relates to many methods and method actions that can be performed. Although the method actions may be discussed in a particular order or illustrated in a flow chart as occurring in a particular order, unless otherwise specified, no particular order is required or a particular order is required because an action depends on another action being completed before the action is performed.

[0088] Fig.11A flow chart of an exemplary method 1100 for building and updating one or more temperature models of a MEMS scanning device (e.g., the MEMS scanning device 100 of FIG. 1 ) is illustrated. In some embodiments, the method 1100 can be performed using an external camera and computing resources during a manufacturing process. In some embodiments, the method 1100 can be performed by the MEMS scanning device using its internal display viewing camera (e.g., the camera 170 of FIG. 1 ) and its internal processor 110.

[0089] Method 1100 includes projecting one or more predetermined patterns on a display of a MEMS scanning device (1110). The displayed pattern(s) are then captured by a camera as one or more images (1120). At the same time, a current temperature is obtained by a thermometer (e.g., thermometer 150 of FIG. 1 ) (1130). Actions 1120 and 1130 may be repeated multiple times so that multiple sets of images (one or more) corresponding to different temperatures may be obtained. The obtained data set is then fed into a machine learning network for use in building one or more temperature models (1140).

[0090] The machine learning network can correspond to Figure 6 Machine Learning Network 600 and / or Fig. 9 The machine learning network 900 of FIG. 10 is a device that can be used to scan a MEMS scanning device using a temperature model. The one or more temperature models include, but are not limited to, a model that represents a relationship between temperature and at least one of the following: (1) an amplitude of at least one of the MEMS scanning mirrors, or (2) a phase shift of at least one of the MEMS scanning mirrors. The temperature model(s) are then stored at the MEMS scanning device (1150) so that the MEMS driver or the laser driver of the MEMS scanning device adjusts one or more control parameters (1160) based on the stored one or more temperature models.

[0091] In some embodiments, method 1100 may also include causing one or more temperature models to be updated (1170). In some embodiments, updating one or more temperature models may be manually triggered by a user. Alternatively or additionally, updating one or more temperature models may be automatically performed at predetermined time intervals.

[0092] Fig.12 A flow chart illustrating an exemplary method 1200 for updating one or more existing temperature models of a MEMS scanning device, which may correspond to Fig.11Action 1170. Method 1200 may be performed by the MEMS device itself using an internal display viewing camera (e.g., camera 170 of FIG. 1 ). Method 1200 includes obtaining a current temperature (1210) and projecting one or more predetermined patterns (1220) onto a predetermined area of ​​a display of the MEMS scanning device based on the current temperature and one or more temperature models. The device's internal display viewing camera is then used to capture one or more images of the (one or more) projected patterns (1230). Next, one or more features are extracted from the captured one or more images (1232). The one or more features may include, but are not limited to, one or more points or one or more lines.

[0093] The features extracted from the captured image(s) are then compared to the one or more ideal features to determine if there is a sufficient difference between the one or more features extracted from the captured image(s) and the ideal feature(s) (1240). If there is a sufficient difference, one or more control parameters are adjusted to mitigate the difference (1250). The one or more parameters include, but are not limited to, an amplitude of one of the (one or more) MEMS scanning mirrors and / or a phase shift of a laser projector. Based on the (one or more) adjustments to the control parameter(s), at least one of the one or more temperature models is adjusted (1260).

[0094] In some embodiments, when the adjustment is greater than a predetermined threshold, the time interval for updating the (one or more) temperature models is reduced or the frequency is increased; on the other hand, when the adjustment is less than the predetermined threshold, the time interval for updating the (one or more) temperature models is increased or the frequency is reduced (1270). Similarly, when there is not enough difference between the (one or more) features extracted from the captured (one or more) images and the (one or more) ideal features, the time interval for updating the (one or more) temperature models can also be increased or the frequency can be reduced (1280).

[0095] Finally, because the principles described herein can be performed in the context of a computing system (e.g., the MEMS scanning device itself is a computing system, and building the temperature model(s) via machine learning can be performed by an external computing system), the following description will be given with respect to Fig.13 to describe some introductory discussions of computing systems.

[0096] Computing systems now increasingly take various forms. For example, a computing system may be a handheld device, an appliance, a laptop computer, a desktop computer, a mainframe, a distributed computing system, a data center, or even a device that is not traditionally considered a computing system, such as a wearable device (e.g., glasses). In this specification and claims, the term "computing system" is broadly defined to include any device or system (or combination thereof) that includes at least one physical and tangible processor, and a physical and tangible memory capable of having computer executable instructions that can be executed by the processor. The memory may take any form and may depend on the nature and form of the computing system. The computing system may be distributed over a network environment and may include multiple constituent computing systems.

[0097] As in Fig.13 As illustrated in , in its most basic configuration, computing system 1300 typically includes at least one hardware processing unit 1302 and memory 1304. Processing unit 1302 may include a general-purpose processor, and may also include a field programmable gate array (FPGA), an application-specific integrated circuit (ASIC), or any other special-purpose circuit. Memory 1304 may be physical system memory, which may be volatile, non-volatile, or some combination of the two. The term "memory" may also be used herein to refer to a non-volatile mass storage device such as a physical storage medium. If the computing system is distributed, then processing, memory, and / or storage capabilities may also be distributed.

[0098] The computing system 1300 also has a number of structures thereon that are often referred to as "executable components." For example, the memory 1304 of the computing system 1300 is shown as including an executable component 1306. The term "executable component" is a name for a structure that is well understood by those of ordinary skill in the computing arts to be a structure that may be software, hardware, or a combination thereof. For example, when implemented in software, one of ordinary skill in the art will understand that the structure of an executable component may include a software object, routine, method, etc. that is executable on the computing system, whether such an executable component is present in the computing system's stack or whether the executable component is present on a computer-readable storage medium.

[0099] In this case, one of ordinary skill in the art will recognize that the structure of the executable component exists on a computer-readable medium so that when interpreted by one or more processors of the computing system (e.g., by a processor thread), the computing system is caused to perform a function. Such a structure can be read directly on the computer by the processor (as is the case if the executable component is binary). Alternatively, the structure can be constructed to be interpretable and / or compiled (whether in a single level or in multiple levels) to generate a binary that can be directly interpreted by the processor. When the term "executable component" is used, this understanding of the exemplary structure of the executable component is well within the understanding of one of ordinary skill in the computing arts.

[0100] The term "executable component" is also well understood by those of ordinary skill to include structures, such as hard-coded or hard-wired logic gates, that are implemented exclusively or nearly exclusively in hardware, such as in a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or any other dedicated circuit. Therefore, the term "executable component" is a term for a structure that is well understood by those of ordinary skill in the computing arts, whether the structure is implemented in software, hardware, or a combination. In this description, the terms "component", "agent", "manager", "service", "engine", "module", "virtual machine", etc. may also be used. As used in this specification and in this example, these terms (whether expressed with or without modifying clauses) are also intended to be synonymous with the term "executable component" and therefore also have structures that are well understood by those of ordinary skill in the computing arts.

[0101] In the above description, embodiments are described with reference to actions performed by one or more computing systems. If such actions are implemented in software, one or more processors (of the associated computing system that performs the action) direct the operation of the computing system in response to the executed computer executable instructions that constitute the executable component. For example, such computer executable instructions may be embodied in one or more computer readable media that form a computer program product. Examples of such operations involve the manipulation of data. If such actions are implemented exclusively or nearly exclusively in hardware, such as in an FPGA or ASIC, the computer executable instructions may be hard-coded or hard-wired logic gates. The computer executable instructions (and the manipulated data) may be stored in a memory 1304 of the computing system 1300. The computing system 1300 may also include a communication channel 1308 that allows the computing system 1300 to communicate with other computing systems, such as a network 1310.

[0102] Although not all computing systems require a user interface, in some embodiments, computing system 1300 includes a user interface system 1312 for interfacing with a user. User interface system 1312 may include output mechanism 1312A and input mechanism 1312B. The principles described herein are not limited to precise output mechanism 1312A or input mechanism 1312B, and therefore will depend on the nature of the device. However, output mechanism 1312A may include, for example, a speaker, a display, a tactile output, a hologram, etc. Examples of input mechanism 1312B may include, for example, a microphone, a touch screen, a hologram, a camera, a keyboard, a mouse or other pointer input, any type of sensor, etc.

[0103] The embodiments described herein may include or utilize a dedicated or general-purpose computing system including computer hardware, such as one or more processors and system memory, as discussed in more detail below. The embodiments described herein also include physical media and other computer-readable media for carrying or storing computer-executable instructions and / or data structures. Such computer-readable media can be any available media accessed by a general-purpose or special-purpose computing system. The computer-readable medium storing computer-executable instructions is a physical storage medium. The computer-readable medium carrying computer-executable instructions is a transmission medium. Therefore, by way of example and not limitation, embodiments of the present invention may include at least two significantly different computer-readable media: a storage medium and a transmission medium.

[0104] Computer-readable storage media include RAM, ROM, EEPROM, CD-ROM or other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices, or any other physical and tangible storage media that can be used to store the desired program code means in the form of computer-executable instructions or data structures and can be accessed by a general-purpose or special-purpose computing system.

[0105] A "network" is defined as one or more data links capable of transmitting electronic data between computing systems and / or modules and / or other electronic devices. When information is transmitted or provided to a computing system via a network or another communication connection (hardwired, wireless, or a combination of hardwired or wireless), the computing system properly treats the connection as a transmission medium. Transmission media may include networks and / or data links, which may be used to carry the required program code elements in the form of computer-executable instructions or data structures and may be accessed by general or special computing systems. Combinations of the above should also be included within the scope of computer-readable media.

[0106] In addition, upon reaching various computing system components, program code units in the form of computer executable instructions or data structures may be automatically transferred from a transmission medium to a storage medium (or vice versa). For example, computer executable instructions or data structures received over a network or data link may be buffered in RAM within a network interface module (e.g., a "NIC") and then ultimately transferred to computing system RAM and / or a less volatile storage medium at the computing system. Thus, it should be understood that a storage medium may be included in a computing system component that also (or even primarily) utilizes a transmission medium.

[0107] Computer executable instructions include, for example, instructions and data that, when executed at a processor, cause a general purpose computing system, a special purpose computing system, or a special purpose processing device to perform a certain function or a set of functions. Alternatively or additionally, the computer executable instructions may configure a computing system to perform a certain function or a set of functions. Computer executable instructions may be, for example, binary files, or even instructions that undergo some translation (e.g., compilation) before being directly executed by a processor, for example, intermediate format instructions such as assembly language, or even source code.

[0108] Although the subject matter has been described in language specific to structural features and / or methodological acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the features or acts described above. Rather, the described features and acts are disclosed as exemplary forms of implementing the claims.

[0109] Those skilled in the art will appreciate that the present invention can be implemented in a network computing environment with many types of computing system configurations, including personal computers, desktop computers, laptop computers, message processors, handheld devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, network PCs, minicomputers, mainframe computers, mobile phones, PDAs, pagers, routers, switches, data centers, wearable devices (e.g., glasses), etc. The present invention can also be implemented in a distributed system environment, where local and remote computing systems linked by a network (by a hardwired data link, a wireless data link, or by a combination of hardwired and wireless data links) all perform tasks. In a distributed system environment, program modules can be located in local and remote memory storage devices.

[0110] Those skilled in the art will also appreciate that the present invention may be implemented in a cloud computing environment. A cloud computing environment may be distributed, although this is not required. When distributed, a cloud computing environment may be distributed internationally within an organization and / or have components owned across multiple organizations. In this specification and the claims below, "cloud computing" is defined as a model for enabling on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services). The definition of "cloud computing" is not limited to any of the other numerous advantages that can be obtained from such a model when properly deployed.

[0111] The remaining figures may discuss various computing systems that may correspond to the computing system 1300 described above. The computing systems of the remaining figures include various components or functional blocks that may implement the various embodiments disclosed herein, as will be explained. The various components or functional blocks may be implemented on a local computing system, or may be implemented on a distributed computing system that includes elements residing in the cloud or implementing aspects of cloud computing. The various components or functional blocks may be implemented as software, hardware, or a combination of software and hardware. The computing systems of the remaining figures may include more or fewer components than those shown in the figure, and some components may be combined as needed. Although it is not necessary to show, the various components of the computing system may access and / or utilize processors and memories (e.g., processor 1302 and memory 1304) as needed to perform their various functions.

[0112] For the processes and methods disclosed herein, the operations performed in the processes and methods may be implemented in different orders. In addition, the operations outlined are provided only as examples, and some operations may be optional, combined into fewer steps and operations, supplemented with further operations, or expanded into additional operations without detracting from the essence of the disclosed embodiments.

[0113] The present invention may be implemented in other specific forms without departing from its spirit or characteristics. The described embodiments are considered to be illustrative and not restrictive in all respects. Therefore, the scope of the present invention is indicated by the appended claims rather than by the preceding description. All changes within the equivalent meaning and scope of the claims should be included within their scope.

Claims

1. A MEMS scanning device, include: one or more laser projectors controlled by a laser driver to project a laser beam; one or more MEMS scanning mirrors that are controlled by a MEMS actuator to scan and reflect the laser beam to generate a raster scan when scanning; a display configured to receive the raster scan generated by the one or more MEMS scanning mirrors; a thermometer configured to detect a temperature of the MEMS scanning device; a display viewing camera configured to capture an image of a predetermined area of ​​the display; one or more processors; as well as One or more computer-readable media having stored thereon one or more temperature models and computer-executable instructions configured to, when executed by the one or more processors, cause the MEMS scanning device to: The thermometer detects the current temperature; projecting one or more predetermined patterns at the predetermined area of ​​the display based on the current temperature and the one or more temperature models, wherein each of the one or more temperature models is custom built using machine learning and represents a relationship between temperature and at least one of: (1) an amplitude of at least one of the one or more MEMS scanning mirrors, or (2) a phase shift of at least one of the MEMS scanning mirrors; capturing, by the display viewing camera, one or more images of the projected one or more predetermined patterns; extracting one or more features from the captured image or images; comparing the one or more features extracted from the captured one or more images to one or more ideal features to determine whether there is a sufficient difference between the one or more features extracted from the captured one or more images and the one or more ideal features; and In response to determining that the sufficient difference exists, updating at least one of the one or more temperature models by: In response to identifying the sufficient difference between the one or more features extracted from the captured one or more images and the one or more ideal features, adjusting at least one of: (1) one or more control parameters of the MEMS driver, or (2) one or more control parameters of the laser driver to mitigate the difference; and At least one of the one or more temperature models is updated based on the current temperature and the adjustment to at least one of: (1) the one or more control parameters of the MEMS driver, or (2) the one or more control parameters of the laser driver.

2. The MEMS scanning device according to claim 1, in, The updating of at least one of the one or more models is performed at predetermined time intervals.

3. The MEMS scanning device according to claim 2, in: The predetermined time interval decreases when an adjustment to at least one of (1) the one or more control parameters of the MEMS driver or (2) the one or more control parameters of the laser driver is greater than a threshold value; or The predetermined time interval increases when an adjustment to at least one of (1) the one or more control parameters of the MEMS driver or (2) the one or more control parameters of the laser driver is less than a threshold value.

4. The MEMS scanning device according to claim 1, in: The one or more MEMS scanning mirrors include a first single-dimensional scanning mirror and a second single-dimensional scanning mirror; The first single-dimensional scanning mirror is configured to scan in a first dimension with a first amplitude; The second single-dimensional scanning mirror is configured to scan in a second dimension with a second amplitude; The first dimension intersects the second dimension; The first one-dimensional scanning mirror and the second one-dimensional scanning mirror are configured to relay a laser beam to project a raster scan onto the display; as well as The one or more models include at least: (1) a model representing the relationship between temperature and the first amplitude corresponding to the first one-dimensional scanning mirror, or (2) a model representing the relationship between temperature and the second amplitude corresponding to the second one-dimensional scanning mirror.

5. The MEMS scanning device according to claim 4, in: At least one of the one or more features is used to update a model representing the relationship between temperature and the first amplitude; or At least one of the one or more features is used to update a model representing the relationship between temperature and the second amplitude.

6. The MEMS scanning device according to claim 1, in: At least one of the one or more MEMS scanning mirrors is a dual-phase scanning mirror that scans the laser beam back and forth bidirectionally in a scanning cycle, wherein when the dual-phase scanning is correct, a point drawn during a forward scan in a cycle and a corresponding point drawn during a backward scan in the cycle coincide in the dual-phase scanning direction; and The one or more temperature models include a model representing a relationship between temperature and a phase shift of the dual phase scanning mirror.

7. The MEMS scanning device according to claim 6, in: At least one of the one or more predetermined patterns is configured to show whether a point projected during scanning in a forward direction and a corresponding point projected during scanning in a backward direction are substantially consistent in the dual-phase scanning direction; as well as The MEMS scanning device modifies the model representing the relationship between temperature and the phase shift of the dual-phase scanning mirror when corresponding points projected during scanning in the forward direction and the backward direction are not substantially consistent in the dual-phase scanning direction.

8. The MEMS scanning device according to claim 7, in: The at least one pattern includes a first pattern and a second pattern; Each of the first pattern and the second pattern includes a first line of dots and a second line of dots; each of the first lines and the second lines in each of the first pattern and the second pattern intersects a scanning dimension of the dual phase scanning mirror; the first line of points in the first pattern and the second line of points in the second pattern are projected during scanning in the forward direction; the second line of points in the first pattern and the first line of points in the second pattern are projected during scanning in the rearward direction; each of the first and second lines of the first pattern corresponds to a respective first and second line of the second pattern; In response to determining that the projected first pattern and the projected second pattern do not overlap, the MEMS scanning device adjusts the phase of the laser driver so that the projected first pattern overlaps the projected second pattern; as well as Based on the adjustment to the phase of the laser driver, the MEMS scanning device updates the temperature model representing the relationship between temperature and the phase shift of the dual-phase scanning mirror.

9. The MEMS scanning device according to claim 6, in: The one or more MEMS scanning mirrors include: a fast scanning mirror configured to scan in a first dimension at a first amplitude; and a slow scanning mirror configured to scan in a second dimension at a second amplitude, the first dimension intersecting the second dimension; The fast scanning mirror is a dual-phase scanning mirror which is controlled by a sinusoidal signal to scan bidirectionally; and The slow scanning mirror is a single-phase scanning mirror that is controlled by a sawtooth signal to scan in one direction.

Images