AR glasses control method and device

Abstract

The invention discloses a control method and device of AR glasses. The vehicle-mounted terminal is provided with a pre-trained light transmittance prediction model. Sensing data collected by a plurality of sensors on the vehicle are obtained; and then, inputting the sensing data into the light transmittance prediction model, and predicting the light transmittance of the photochromic lenses of the AR glasses. And finally, the light transmittance of the photochromic lenses is sent to the AR glasses, so that the AR glasses adjust the photochromic lenses based on the light transmittance of the photochromic lenses. Therefore, when the user is in the driving scene, the light transmittance of the electrochromic lens is accurately adjusted under the condition that the illumination intensity of the exterior and the illumination intensity of the interior of the vehicle are obviously different, and then the imaging quality of the AR glasses is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and more particularly to a control method and device for AR glasses. Background Art

[0002] Augmented Reality (AR) is a technology that combines virtual reality with the real world. AR glasses provide users with an enhanced visual experience by superimposing digital information on the real environment. Users can see the real world around them through AR glasses, and digital content related to the real environment, such as images, videos, text, and 3D models, is displayed in the field of vision.

[0003] Electrochromism refers to the phenomenon that the optical properties of a material (reflectivity, transmittance, absorptivity, etc.) undergo stable and reversible color changes under the action of an external electric field, which is manifested in the form of reversible changes in color and transparency. Lenses with electrochromic properties are called electrochromic lenses.

[0004] Currently, AR glasses can adjust the electrochromic lenses of AR glasses according to light intensity to improve the imaging quality of AR glasses in different usage scenarios.

[0005] However, when the user is in a driving scenario, there may be a significant difference in light intensity between the outside and inside of the vehicle, which may cause the AR glasses to be unable to accurately adjust the transmittance of the electrochromic lenses, thereby reducing the imaging quality of the AR glasses. Summary of the invention

[0006] The present specification provides a control method, device, storage medium and electronic device for AR glasses to accurately adjust the transmittance of electrochromic lenses and improve the imaging quality of AR glasses.

[0007] This manual adopts the following technical solutions:

[0008] This specification provides a control method for AR glasses, the method is applied to a vehicle-mounted terminal, and the vehicle-mounted terminal deploys a pre-trained transmittance prediction model; the method includes:

[0009] Acquire sensor data collected by multiple sensors on the vehicle;

[0010] Inputting the sensing data into the transmittance prediction model to predict the transmittance of the photochromic lenses of the AR glasses;

[0011] The transmittance of the photochromic lens is sent to the AR glasses, so that the AR glasses adjust the photochromic lens based on the transmittance of the photochromic lens.

[0012] Optionally, the vehicle-mounted terminal deploys a pre-trained target detection model; the method further includes:

[0013] Inputting the sensing data into the target detection model, detecting risk objects on the driving path of the vehicle, and determining detection information of at least one risk object; wherein the detection information at least includes the category of the risk object, the location of the risk object, and the outline of the risk object;

[0014] The detection information of the at least one risk object is sent to the AR glasses, so that the AR glasses display the detection information of the at least one risk object.

[0015] Optionally, sending the detection information of the at least one risk object to the AR glasses so that the AR glasses display the detection information of the at least one risk object includes:

[0016] Obtaining the current field of view of the AR glasses;

[0017] Based on the detection information of the at least one risk object and the current field of view of the AR glasses, determining whether the at least one risk object is located within the current field of view of the AR glasses;

[0018] If yes, sending the detection information of the risk object in the current field of view of the AR glasses to the AR glasses, so that the AR glasses display the detection information of the risk object in the current field of view of the AR glasses based on the position of the risk object in the current field of view of the AR glasses;

[0019] If not, generate a warning sign corresponding to the risk object outside the current field of view of the AR glasses, and send the warning sign and detection information of the risk object outside the current field of view of the AR glasses to the AR glasses, so that the AR glasses display the warning sign based on the position of the risk object outside the current field of view of the AR glasses.

[0020] Optionally, the method further comprises:

[0021] Get navigation information generated by the navigation application;

[0022] Based on the navigation information, generating a navigation icon corresponding to the navigation information;

[0023] A navigation icon corresponding to the navigation information is sent to the AR glasses, so that the AR glasses display the navigation icon corresponding to the navigation information.

[0024] This specification provides a control method for AR glasses, which is applied to AR glasses; the method includes:

[0025] Obtaining the transmittance of the photochromic lenses of the AR glasses sent by the vehicle-mounted terminal; wherein the transmittance is predicted by the vehicle-mounted terminal by inputting sensor data collected by multiple sensors on the vehicle into a pre-trained transmittance prediction model;

[0026] The photochromic lens is adjusted based on the light transmittance of the photochromic lens.

[0027] Optionally, adjusting the photochromic lens based on the light transmittance of the photochromic lens comprises:

[0028] The photochromic lens is adjusted based on the light transmittance of the photochromic lens and a preset adjustment rate.

[0029] Optionally, the method further comprises:

[0030] Acquire detection information of at least one risk object sent by the vehicle-mounted terminal; wherein the detection information is obtained by the vehicle-mounted terminal inputting sensor data collected by multiple sensors on the vehicle into a pre-trained target detection model for detection; the detection information at least includes the category of the risk object, the location of the risk object, and the outline of the risk object;

[0031] The detection information of the at least one risk object is displayed.

[0032] Optionally, displaying the detection information of the at least one risk object includes:

[0033] Based on the detection information of the at least one risk object and the current field of view of the AR glasses, determining whether the at least one risk object is located within the current field of view of the AR glasses;

[0034] If yes, based on the position of the risk object within the current field of view of the AR glasses, displaying detection information of the risk object within the current field of view of the AR glasses;

[0035] If not, a warning sign corresponding to the risk object outside the current field of view of the AR glasses is generated, and the warning sign is displayed based on the position of the risk object outside the current field of view of the AR glasses.

[0036] Optionally, the method further comprises:

[0037] Get navigation information generated by the navigation application;

[0038] Based on the navigation information, a navigation icon corresponding to the navigation information is generated, and the navigation icon corresponding to the navigation information is displayed.

[0039] Optionally, the AR glasses are provided with an audio acquisition device; and the method further includes:

[0040] Acquire the user's voice control instructions collected by the audio collection device;

[0041] Based on the voice control command, the AR glasses are controlled.

[0042] Optionally, the AR glasses are provided with an image acquisition device; and the method further includes:

[0043] Acquire the user's gesture control instructions acquired by the image acquisition device;

[0044] Based on the gesture control instruction, the AR glasses are controlled.

[0045] This specification provides a control device for AR glasses, the device is applied to a vehicle-mounted terminal, and the vehicle-mounted terminal deploys a pre-trained transmittance prediction model; the device includes:

[0046] An acquisition module, used to acquire sensor data collected by multiple sensors on the vehicle;

[0047] A prediction module, used to input the sensing data into the transmittance prediction model to predict the transmittance of the photochromic lenses of the AR glasses;

[0048] A sending module is used to send the transmittance of the photochromic lens to the AR glasses, so that the AR glasses adjust the photochromic lens based on the transmittance of the photochromic lens.

[0049] This specification provides a control device for AR glasses, the device is applied to AR glasses, and the device includes:

[0050] An acquisition module, used for acquiring the transmittance of the photochromic lens of the AR glasses sent by the vehicle-mounted terminal; wherein the transmittance is predicted by the vehicle-mounted terminal by inputting the collected sensor data into a pre-trained transmittance prediction model;

[0051] The adjustment module is used to adjust the photochromic lens based on the light transmittance of the photochromic lens.

[0052] This specification provides an electronic device, including a communication interface, a processor, a memory and a bus, wherein the communication interface, the processor and the memory are interconnected via the bus;

[0053] The memory stores machine-readable instructions, and the processor executes the control method of the AR glasses by calling the machine-readable instructions.

[0054] This specification provides a machine-readable storage medium, which stores machine-readable instructions. When the machine-readable instructions are called and executed by a processor, the control method of the above-mentioned AR glasses is implemented.

[0055] At least one of the above technical solutions adopted in this specification can achieve the following beneficial effects:

[0056] In this specification, a pre-trained transmittance prediction model is deployed in the vehicle terminal. The sensor data collected by multiple sensors on the vehicle are obtained. Then, the sensor data is input into the transmittance prediction model to predict the transmittance of the photochromic lenses of the AR glasses. Finally, the transmittance of the photochromic lenses is sent to the AR glasses so that the AR glasses adjust the photochromic lenses based on the transmittance of the photochromic lenses. Thus, when the user is in a driving scene, when there is a significant difference in light intensity between the outside and inside of the vehicle, the transmittance of the electrochromic lenses is accurately adjusted, thereby improving the imaging quality of the AR glasses. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The drawings described herein are used to provide a further understanding of this specification and constitute a part of this specification. The illustrative embodiments and descriptions of this specification are used to explain this specification and do not constitute an improper limitation on this specification. In the drawings:

[0058] Figure 1 is a flow chart of a control method of AR glasses shown in an exemplary embodiment;

[0059] Figure 2 is a schematic diagram of a navigation icon shown in an exemplary embodiment;

[0060] Figure 3 is a flowchart of another method for controlling AR glasses shown in an exemplary embodiment;

[0061] Figure 4 is a structural diagram of an electronic device in which a control device of AR glasses is located, shown in an exemplary embodiment;

[0062] Figure 5 is a structural diagram of a control device for AR glasses shown in an exemplary embodiment;

[0063] Figure 6 It is a structural diagram of another control device of AR glasses shown in an exemplary embodiment. DETAILED DESCRIPTION

[0064] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this specification.

[0065] It should be noted that: in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0066] Currently, AR glasses can adjust the electrochromic lenses of AR glasses according to light intensity to improve the imaging quality of AR glasses in different usage scenarios.

[0067] However, when the user is in a driving scenario, there may be a significant difference in light intensity between the outside and inside of the vehicle, which may cause the AR glasses to be unable to accurately adjust the transmittance of the electrochromic lenses, thereby reducing the imaging quality of the AR glasses.

[0068] Based on this, this specification proposes a technical solution for predicting the transmittance of the electrochromic lenses of AR glasses based on the sensor data collected by multiple sensors on the vehicle, and adjusting the electrochromic lenses based on the transmittance of the electrochromic lenses. Thus, when the user is in a driving scene, when there is a significant difference in the light intensity outside and inside the vehicle, the transmittance of the electrochromic lenses is accurately adjusted, thereby improving the imaging quality of the AR glasses.

[0069] The technical solutions provided by the embodiments of this specification are described in detail below in conjunction with the accompanying drawings.

[0070] Figure 1 FIG. 1 is a flow chart of a control method of AR glasses shown in an exemplary embodiment, and the method specifically includes the following steps:

[0071] S100: Acquire sensor data collected by multiple sensors on the vehicle.

[0072] S102: Input the sensing data into the transmittance prediction model to predict the transmittance of the photochromic lenses of the AR glasses.

[0073] In the embodiment of this specification, the control method of AR glasses is applied to a vehicle-mounted terminal. The vehicle-mounted terminal deploys a pre-trained transmittance prediction model.

[0074] The vehicle terminal can obtain sensor data collected by multiple sensors on the vehicle. The sensors mentioned here include: LiDAR, camera, light intensity sensor, temperature sensor, humidity sensor, etc. Among them, LiDAR can collect point cloud data to create a three-dimensional model of the vehicle's surroundings. The camera can collect image data to identify pedestrians, vehicles, traffic signs and signals on the road through image processing technology. The light intensity sensor can collect light intensity to determine the light intensity of the object in front of the vehicle at the current moment. The temperature sensor and humidity sensor can collect temperature and humidity to predict the weather conditions of the vehicle's environment.

[0075] Then, the vehicle terminal can input the sensor data into the transmittance prediction model to predict the transmittance of the photochromic lenses of the AR glasses. For example, the light intensity collected by the light intensity sensor is high, which means that the vehicle is in strong sunlight in clear weather at the current moment. The transmittance of the photochromic lenses of the AR glasses needs to be reduced to reduce glare and make the AR glasses present a sunshade effect. For another example, the light intensity collected by the light intensity sensor is high, and the laser radar and camera collect the appearance of a tunnel ahead, indicating that the vehicle is about to enter a tunnel environment where the light suddenly decreases. In this case, the transmittance of the photochromic lenses of the AR glasses needs to be increased in advance to help the user's eyes adapt to the upcoming light changes and avoid visual discomfort caused by suddenly entering a dark environment.

[0076] It should be noted that users may encounter more complex situations while driving. The transmittance prediction model needs to comprehensively consider factors such as point cloud data, image data, light intensity, temperature and humidity to predict the transmittance of the photochromic lenses of AR glasses to ensure that AR glasses can provide excellent visual effects and comfort in various environments.

[0077] Furthermore, different users' glasses have different sensitivities to light. The transmittance prediction model can also take into account the user's personal preferences and predict a more accurate transmittance of the photochromic lenses of AR glasses. Of course, users can also adjust the transmittance of AR glasses according to their own needs.

[0078] Next, the method for training the transmittance prediction model is as follows:

[0079] First, the sensor data collected by multiple sensors on the vehicle and the transmittance actually adjusted by the user on the vehicle based on the current environment are obtained.

[0080] Then, the sensor data is used as training samples, and the transmittance actually adjusted by the user on the vehicle based on the current environment is used as the training sample label to construct the training samples.

[0081] Finally, the training samples are input into the transmittance prediction model, so that the transmittance of the photochromic lenses of the AR glasses can be predicted by the transmittance prediction model based on the training samples, and the transmittance prediction model is subjected to supervised training calculation with the minimization of the deviation between the transmittance and the training sample label as the optimization goal.

[0082] It should be noted that the training samples may also include personalized data of users.

[0083] S104: Sending the transmittance of the photochromic lens to the AR glasses, so that the AR glasses adjust the photochromic lens based on the transmittance of the photochromic lens.

[0084] In the embodiments of this specification, the AR glasses can establish a wireless communication connection with the vehicle's onboard terminal. The wireless communication connection methods mentioned here may include Bluetooth, Near Field Communication (NFC), Ultra Wide Band (UWB), WiFi, etc. This specification does not limit the wireless communication connection method.

[0085] After the AR glasses establish a wireless communication connection with the vehicle's onboard terminal, the onboard terminal can send the transmittance of the photochromic lens to the AR glasses, so that the AR glasses adjust the photochromic lens based on the transmittance of the photochromic lens.

[0086] In actual applications, users may encounter various potential risks when driving a vehicle, which may affect the safe driving of the vehicle. For example, other vehicles may overtake the vehicle from the side, causing potential collision risks. In the blind spot of the vehicle, pedestrians or animals may suddenly break into the road. In addition, serious potholes in the road ahead may also cause the vehicle to bump or even be damaged.

[0087] However, it is difficult for users to stay alert at all times when driving a vehicle, thereby reducing driving safety.

[0088] In the embodiment of this specification, the vehicle terminal deploys a pre-trained target detection model. The vehicle terminal can input the sensor data into the target detection model, detect the risk object on the vehicle's driving path, and determine the detection information of at least one risk object. The detection information includes at least the category of the risk object, the location of the risk object, and the outline of the risk object.

[0089] Among them, risk objects may refer to objects that pose a safety risk to the normal driving of the vehicle. Risk objects may include: other vehicles, pedestrians, potholes on the road, obstacles, etc.

[0090] Then, the vehicle terminal can send the detection information of at least one risk object to the AR glasses, so that the AR glasses can display the detection information of at least one risk object. For example, when a pedestrian appears in front of the vehicle, the AR glasses will mark the outline of the pedestrian in the field of view, and display relevant text information (such as pedestrian) and distance information next to the outline. For another example, when a pothole appears in front of the vehicle, the AR glasses will mark the outline of the pothole on the road surface in the field of view, and display relevant text information (such as pothole) and distance information next to the outline.

[0091] It should be noted that the target detection model can be trained through autonomous driving datasets such as nuScenes and KITTI.

[0092] It can be seen from the above embodiments that the target detection model can accurately identify key information such as the category, location and outline of the risk object, providing users with comprehensive driving assistance.

[0093] In actual applications, the field of view of AR glasses is limited and may not be able to fully display all the risk objects around the vehicle. For example, other vehicles may overtake the vehicle from the side, or pedestrians may suddenly appear in the blind spot. If these risk objects are outside the field of view of AR glasses, the driver may not be able to receive timely warnings, thereby increasing safety risks.

[0094] In the embodiments of this specification, the vehicle-mounted terminal can obtain the current field of view of the AR glasses.

[0095] Then, the vehicle-mounted terminal can determine whether the at least one risk object is located within the current field of view of the AR glasses based on the detection information of the at least one risk object and the current field of view of the AR glasses.

[0096] If so, the vehicle-mounted terminal can send the detection information of the risk objects within the current field of view of the AR glasses to the AR glasses, so that the AR glasses display the detection information of the risk objects within the current field of view of the AR glasses based on the position of the risk objects within the current field of view of the AR glasses.

[0097] If no, the vehicle-mounted terminal can generate a warning sign corresponding to the risk object outside the current field of view of the AR glasses, and send the warning sign and detection information of the risk object outside the current field of view of the AR glasses to the AR glasses, so that the AR glasses display the warning sign based on the position of the risk object outside the current field of view of the AR glasses.

[0098] Among them, the representation form of the warning sign can be multiple. Taking the example of another vehicle overtaking on the left side of the vehicle, the representation form of the warning sign can be to display red, yellow or other colors at the left border of the left lens of the AR glasses. The representation form of the warning sign can also be to display an icon for indicating vehicle overtaking at the left border of the left lens of the AR glasses. When necessary, this specification does not limit the representation form of the warning sign.

[0099] Furthermore, the color of the warning sign can also indicate different risk levels, which is used to remind users of the urgency of the current risk, so that users can take different measures to avoid the risk. For example, when the distance between the risk object and the vehicle is far, the risk level is low, and the color of the warning sign is yellow. When the distance between the risk object and the vehicle is close, the risk level is medium, and the color of the warning sign is orange. When the distance between the risk object and the vehicle is extremely close, the risk level is high, and the color of the warning sign is red.

[0100] In actual applications, traditional voice navigation systems often have difficulty providing concise and clear guidance when faced with complex road conditions. This limitation may cause users to distract their attention and check the navigation map on the in-vehicle terminal to obtain clearer direction information during driving. This behavior undoubtedly increases safety risks during driving.

[0101] In the embodiment of the present specification, the vehicle-mounted terminal can obtain navigation information generated by the navigation application.

[0102] Then, the vehicle-mounted terminal may generate a navigation icon corresponding to the navigation information based on the navigation information.

[0103] Then, the vehicle-mounted terminal can send the navigation icon corresponding to the navigation information to the AR glasses, so that the AR glasses display the navigation icon corresponding to the navigation information. Figure 2 shown.

[0104] Figure 2 It is a schematic diagram of a navigation icon shown in an exemplary embodiment.

[0105] exist Figure 2 In the navigation information, the navigation icons corresponding to the navigation information include: left lane change, right lane change, left turn, right turn, U-turn, straight ahead, etc. The above navigation icons can be combined to show the user the navigation trajectory of the vehicle at a future moment. Furthermore, the object detection model can detect the lane the vehicle is currently in and display the navigation icon on the corresponding lane.

[0106] It can be seen from the above embodiments that by displaying navigation icons corresponding to navigation information through AR glasses, users do not need to divert their eyes or operate the device during driving, which effectively avoids distraction caused by viewing the traditional vehicle-mounted terminal screen.

[0107] Figure 3 FIG. 1 is a flow chart of a control method of AR glasses shown in an exemplary embodiment, and the method specifically includes the following steps:

[0108] S300: Obtaining the transmittance of the photochromic lenses of the AR glasses sent by the vehicle-mounted terminal; wherein the transmittance is predicted by the vehicle-mounted terminal by inputting sensor data collected by multiple sensors on the vehicle into a pre-trained transmittance prediction model.

[0109] S302: Adjusting the photochromic lens based on the light transmittance of the photochromic lens.

[0110] In the embodiments of this specification, the control method of AR glasses is applied to AR glasses. AR glasses can obtain the transmittance of the photochromic lenses of AR glasses sent by the vehicle terminal. The transmittance is predicted by the vehicle terminal by inputting the sensor data collected by multiple sensors on the vehicle into a pre-trained transmittance prediction model.

[0111] The AR glasses can then adjust the photochromic lenses based on the light transmittance of the photochromic lenses.

[0112] Furthermore, in order to prevent the user from feeling uncomfortable when the transmittance of the photochromic lens changes suddenly, the AR glasses can adjust the photochromic lens based on the transmittance of the photochromic lens and a preset adjustment rate. The preset adjustment rate can be set based on expert experience.

[0113] In actual applications, users may encounter various potential risks when driving a vehicle, which may affect the safe driving of the vehicle. However, it is difficult for users to stay alert at all times when driving a vehicle, thereby reducing the safety of driving.

[0114] In the embodiments of this specification, the AR glasses can obtain detection information of at least one risk object sent by the vehicle terminal. The detection information is obtained by the vehicle terminal inputting the sensor data collected by multiple sensors on the vehicle into the pre-trained target detection model. The detection information includes at least the category of the risk object, the location of the risk object, and the outline of the risk object.

[0115] Then, the AR glasses can display detection information of at least one risk object.

[0116] It can be seen from the above embodiments that AR glasses can display key information such as the category, location and outline of at least one risk object, providing users with comprehensive driving assistance.

[0117] In actual applications, the field of view of AR glasses is limited and may not be able to fully display all the risk objects around the vehicle. If there are risk objects outside the field of view of AR glasses, the driver may not be able to receive timely warnings, which increases safety risks.

[0118] In the embodiments of the present specification, the AR glasses may determine whether at least one risk object is within the current field of view of the AR glasses based on the detection information of the at least one risk object and the current field of view of the AR glasses.

[0119] If so, the AR glasses may display detection information of the risk object within the current field of view of the AR glasses based on the position of the risk object within the current field of view of the AR glasses.

[0120] If not, the AR glasses may generate a warning sign corresponding to the risk object outside the current field of view of the AR glasses, and display the warning sign based on the position of the risk object outside the current field of view of the AR glasses.

[0121] In actual applications, traditional voice navigation systems often have difficulty providing concise and clear guidance when faced with complex road conditions. This limitation may cause users to distract their attention and check the navigation map on the in-vehicle terminal to obtain clearer direction information during driving. This behavior undoubtedly increases safety risks during driving.

[0122] In the embodiments of this specification, the AR glasses can obtain navigation information generated by the navigation application.

[0123] Then, the AR glasses can generate a navigation icon corresponding to the navigation information based on the navigation information, and display the navigation icon corresponding to the navigation information.

[0124] It can be seen from the above embodiments that by displaying navigation icons corresponding to navigation information through AR glasses, users do not need to divert their eyes or operate the device during driving, which effectively avoids distraction caused by viewing the traditional vehicle-mounted terminal screen.

[0125] In actual applications, users need to concentrate on observing road conditions while driving. Traditional touch screens or button operations can easily distract users and may lead to traffic accidents caused by distracted operation of the equipment.

[0126] In the embodiments of this specification, an audio acquisition device is provided on the AR glasses. The AR glasses can obtain the user's voice control instructions collected by the audio acquisition device. Then, the AR glasses can control the AR glasses based on the voice control instructions. For example, the user can control the AR glasses to adjust the transmittance of the photochromic lenses through voice control instructions (such as reducing the transmittance of the photochromic lenses). For another example, the user can control the display content in the AR glasses through voice control instructions (such as shielding obstacles on the left side of the vehicle).

[0127] Furthermore, the AR glasses are also provided with an image acquisition device. The AR glasses can obtain the gesture control instructions of the user acquired by the image acquisition device. Then, the AR glasses can control the AR glasses based on the gesture control instructions. For example, the user can directly click on the display content in the AR glasses to control the display content. The display content can be "turning to white", "turning to brown", "darker", "lighter", "increasing transparency", "decreasing transparency", "transparency becoming 50%", "transparency becoming 80%", etc. For another example, a specific gesture associated with the light intensity adjustment operation is pre-set, such as an "OK gesture" or a "shaking gesture".

[0128] As can be seen from the above embodiments, the user can control the AR glasses through voice and gestures without releasing the steering wheel, thereby reducing traffic accidents that may be caused by distracted operation of the device, thereby improving the convenience of the user during driving and enhancing driving safety.

[0129] It can be seen from the above method that the vehicle terminal deploys a pre-trained transmittance prediction model. The sensor data collected by multiple sensors on the vehicle are obtained. Then, the sensor data is input into the transmittance prediction model to predict the transmittance of the photochromic lenses of the AR glasses. Finally, the transmittance of the photochromic lenses is sent to the AR glasses so that the AR glasses adjust the photochromic lenses based on the transmittance of the photochromic lenses. Thus, when the user is in a driving scene, when there is a significant difference in light intensity between the outside and inside of the vehicle, the transmittance of the electrochromic lenses is accurately adjusted, thereby improving the imaging quality of the AR glasses.

[0130] Furthermore, AR glasses can display key information such as the category, location and outline of at least one risk object, providing users with comprehensive driving assistance. In addition, by displaying navigation icons corresponding to navigation information on AR glasses, users do not need to divert their sight or operate the device during driving, effectively avoiding distraction caused by viewing the traditional vehicle terminal screen. Users can also control AR glasses through voice and gestures. Thus, traffic accidents that may be caused by distracted operation of equipment are reduced.

[0131] Corresponding to the embodiment of the control method of the AR glasses described above, this specification also provides an embodiment of a control device of the AR glasses.

[0132] See also Figure 4 , Figure 4 It is a structural diagram of an electronic device in which a control device of an AR glasses is located, which is shown in an exemplary embodiment. At the hardware level, the device includes a processor 402, an internal bus 404, a network interface 406, a memory 408, and a non-volatile memory 410, and of course may also include other required hardware. One or more embodiments of this specification can be implemented based on software, such as the processor 402 reading the corresponding computer program from the non-volatile memory 410 into the memory 408 and then running it. Of course, in addition to software implementation, one or more embodiments of this specification do not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0133] See also Figure 5 , Figure 5 FIG. 1 is a structural diagram of a control device for AR glasses shown in an exemplary embodiment. The control device for AR glasses can be applied to Figure 4 In the electronic device shown in the figure, the technical solution of this specification is implemented. The device may include:

[0134] An acquisition module 500 is used to acquire sensor data collected by multiple sensors on the vehicle;

[0135] A prediction module 502, configured to input the sensing data into the transmittance prediction model to predict the transmittance of the photochromic lenses of the AR glasses;

[0136] The sending module 504 is used to send the transmittance of the photochromic lens to the AR glasses, so that the AR glasses adjust the photochromic lens based on the transmittance of the photochromic lens.

[0137] Optionally, the vehicle-mounted terminal deploys a pre-trained target detection model; the sending module 504 is further specifically used to input the sensor data into the target detection model, detect risk objects on the vehicle's driving path, and determine the detection information of at least one risk object; wherein the detection information includes at least the category of the risk object, the location of the risk object and the outline of the risk object; the detection information of the at least one risk object is sent to the AR glasses, so that the AR glasses display the detection information of the at least one risk object.

[0138] Optionally, the sending module 504 is further specifically used to obtain the current field of view of the AR glasses; based on the detection information of the at least one risk object and the current field of view of the AR glasses, determine whether the at least one risk object is located within the current field of view of the AR glasses; if so, send the detection information of the risk object located within the current field of view of the AR glasses to the AR glasses, so that the AR glasses display the detection information of the risk object located within the current field of view of the AR glasses based on the position of the risk object located within the current field of view of the AR glasses; if not, generate a warning sign corresponding to the risk object located outside the current field of view of the AR glasses, and send the warning sign and the detection information of the risk object located outside the current field of view of the AR glasses to the AR glasses, so that the AR glasses display the warning sign based on the position of the risk object located outside the current field of view of the AR glasses.

[0139] Optionally, the sending module 504 is further specifically used to obtain navigation information generated by a navigation application; based on the navigation information, generate a navigation icon corresponding to the navigation information; and send the navigation icon corresponding to the navigation information to the AR glasses, so that the AR glasses display the navigation icon corresponding to the navigation information.

[0140] See also Figure 6 , Figure 6 FIG. 1 is a structural diagram of another control device of AR glasses shown in an exemplary embodiment. The control device of AR glasses can be applied to Figure 4 In the electronic device shown in the figure, the technical solution of this specification is implemented. The device may include:

[0141] The acquisition module 600 is used to acquire the transmittance of the photochromic lens of the AR glasses sent by the vehicle-mounted terminal; wherein the transmittance is predicted by the vehicle-mounted terminal by inputting the sensor data collected by multiple sensors on the vehicle into a pre-trained transmittance prediction model;

[0142] The adjustment module 602 is used to adjust the photochromic lens based on the light transmittance of the photochromic lens.

[0143] Optionally, the adjustment module 602 is specifically configured to adjust the photochromic lens based on the light transmittance of the photochromic lens and a preset adjustment rate.

[0144] Optionally, the adjustment module 602 is further specifically used to obtain detection information of at least one risk object sent by the vehicle-mounted terminal; wherein the detection information is obtained by the vehicle-mounted terminal inputting sensor data collected by multiple sensors on the vehicle into a pre-trained target detection model for detection; the detection information includes at least the category of the risk object, the location of the risk object and the outline of the risk object; and the detection information of at least one risk object is displayed.

[0145] Optionally, the adjustment module 602 is specifically used to determine whether the at least one risk object is located within the current field of view of the AR glasses based on the detection information of the at least one risk object and the current field of view of the AR glasses; if so, based on the position of the risk object located within the current field of view of the AR glasses, display the detection information of the risk object located within the current field of view of the AR glasses; if not, generate a warning sign corresponding to the risk object located outside the current field of view of the AR glasses, and display the warning sign based on the position of the risk object located outside the current field of view of the AR glasses.

[0146] Optionally, the adjustment module 602 is further specifically used to obtain navigation information generated by a navigation application; based on the navigation information, generate a navigation icon corresponding to the navigation information, and display the navigation icon corresponding to the navigation information.

[0147] Optionally, the AR glasses are provided with an audio collection device; the adjustment module 602 is further specifically used to obtain the user's voice control instructions collected by the audio collection device; and control the AR glasses based on the voice control instructions.

[0148] Optionally, the AR glasses are provided with an image acquisition device; the adjustment module 602 is further specifically used to obtain gesture control instructions of the user acquired by the image acquisition device; and control the AR glasses based on the gesture control instructions.

[0149] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0150] Based on the same concept as the above method, this specification also provides an electronic device, including: a processor; a memory for storing processor executable instructions; wherein the processor implements the steps of the method described in any of the above embodiments by running the executable instructions.

[0151] Based on the same concept as the above method, this specification also provides a computer-readable storage medium on which computer instructions are stored. When the instructions are executed by a processor, the steps of the method described in any of the above embodiments are implemented.

[0152] Based on the same concept as the above method, this specification also provides a computer program product, including a computer program / instruction, which implements the steps of the method described in any of the above embodiments when executed by a processor.

[0153] It should be understood that although the terms first, second, third, etc. may be used to describe various information in one or more embodiments of this specification, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0154] The above description is merely a preferred embodiment of one or more embodiments of the present specification and is not intended to limit one or more embodiments of the present specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of one or more embodiments of the present specification shall be included in the scope of protection of one or more embodiments of the present specification.

Claims

1. A control method for AR glasses, characterized in that: The method is applied to a vehicle-mounted terminal, and the vehicle-mounted terminal deploys a pre-trained transmittance prediction model. The method includes: Acquire sensor data collected by multiple sensors on the vehicle; Inputting the sensing data into the transmittance prediction model to predict the transmittance of the photochromic lenses of the AR glasses; The transmittance of the photochromic lens is sent to the AR glasses, so that the AR glasses adjust the photochromic lens based on the transmittance of the photochromic lens.

2. The method according to claim 1, characterized in that The vehicle terminal deploys a pre-trained target detection model; the method also includes: Inputting the sensing data into the target detection model, detecting risk objects on the driving path of the vehicle, and determining detection information of at least one risk object; wherein the detection information at least includes the category of the risk object, the location of the risk object, and the outline of the risk object; The detection information of the at least one risk object is sent to the AR glasses, so that the AR glasses display the detection information of the at least one risk object.

3. The method according to claim 2, characterized in that Sending the detection information of the at least one risk object to the AR glasses so that the AR glasses display the detection information of the at least one risk object, includes: Obtaining the current field of view of the AR glasses; Based on the detection information of the at least one risk object and the current field of view of the AR glasses, determining whether the at least one risk object is located within the current field of view of the AR glasses; If yes, sending the detection information of the risk object in the current field of view of the AR glasses to the AR glasses, so that the AR glasses display the detection information of the risk object in the current field of view of the AR glasses based on the position of the risk object in the current field of view of the AR glasses; If not, generate a warning sign corresponding to the risk object outside the current field of view of the AR glasses, and send the warning sign and detection information of the risk object outside the current field of view of the AR glasses to the AR glasses, so that the AR glasses display the warning sign based on the position of the risk object outside the current field of view of the AR glasses.

4. The method according to claim 1, characterized in that The method further comprises: Get navigation information generated by the navigation application; Based on the navigation information, generating a navigation icon corresponding to the navigation information; A navigation icon corresponding to the navigation information is sent to the AR glasses, so that the AR glasses display the navigation icon corresponding to the navigation information.

5. A control method for AR glasses, characterized in that: The method is applied to AR glasses, and the method comprises: Obtaining the transmittance of the photochromic lenses of the AR glasses sent by the vehicle-mounted terminal; wherein the transmittance is predicted by the vehicle-mounted terminal by inputting sensor data collected by multiple sensors on the vehicle into a pre-trained transmittance prediction model; The photochromic lens is adjusted based on the light transmittance of the photochromic lens.

6. The method according to claim 5, characterized in that Adjusting the photochromic lens based on the light transmittance of the photochromic lens comprises: The photochromic lens is adjusted based on the light transmittance of the photochromic lens and a preset adjustment rate.

7. The method according to claim 5, characterized in that The method further comprises: Acquire detection information of at least one risk object sent by the vehicle-mounted terminal; wherein the detection information is obtained by the vehicle-mounted terminal inputting sensor data collected by multiple sensors on the vehicle into a pre-trained target detection model for detection; the detection information at least includes the category of the risk object, the location of the risk object, and the outline of the risk object; The detection information of the at least one risk object is displayed.

8. The method according to claim 7, characterized in that Displaying the detection information of the at least one risk object includes: Based on the detection information of the at least one risk object and the current field of view of the AR glasses, determining whether the at least one risk object is located within the current field of view of the AR glasses; If yes, based on the position of the risk object within the current field of view of the AR glasses, displaying detection information of the risk object within the current field of view of the AR glasses; If not, a warning sign corresponding to the risk object outside the current field of view of the AR glasses is generated, and the warning sign is displayed based on the position of the risk object outside the current field of view of the AR glasses.

9. The method according to claim 5, characterized in that The method further comprises: Get navigation information generated by the navigation application; Based on the navigation information, a navigation icon corresponding to the navigation information is generated, and the navigation icon corresponding to the navigation information is displayed.

10. The method according to claim 1, characterized in that The AR glasses are provided with an audio acquisition device; the method further comprises: Acquire the user's voice control instructions collected by the audio collection device; Based on the voice control command, the AR glasses are controlled.

11. The method according to claim 1, characterized in that The AR glasses are provided with an image acquisition device; the method further comprises: Acquire the user's gesture control instructions acquired by the image acquisition device; Based on the gesture control instruction, the AR glasses are controlled.

12. A control device for AR glasses, characterized in that: The device is applied to a vehicle-mounted terminal, and the vehicle-mounted terminal deploys a pre-trained transmittance prediction model; the device comprises: An acquisition module, used to acquire sensor data collected by multiple sensors on the vehicle; A prediction module, used to input the sensing data into the transmittance prediction model to predict the transmittance of the photochromic lenses of the AR glasses; A sending module is used to send the transmittance of the photochromic lens to the AR glasses, so that the AR glasses adjust the photochromic lens based on the transmittance of the photochromic lens.

13. A control device for AR glasses, characterized in that: The device is applied to AR glasses, and the device includes: An acquisition module, used for acquiring the transmittance of the photochromic lens of the AR glasses sent by the vehicle-mounted terminal; wherein the transmittance is predicted by the vehicle-mounted terminal by inputting the collected sensor data into a pre-trained transmittance prediction model; The adjustment module is used to adjust the photochromic lens based on the light transmittance of the photochromic lens.

14. An electronic device, characterized in that: It includes a communication interface, a processor, a memory and a bus, wherein the communication interface, the processor and the memory are interconnected via the bus; The memory stores machine-readable instructions, and the processor executes the method according to any one of claims 1 to 11 by calling the machine-readable instructions.

15. A machine-readable storage medium, characterized in that: The machine-readable storage medium stores machine-readable instructions, and when the machine-readable instructions are called and executed by a processor, the method according to any one of claims 1 to 11 is implemented.

Images