Self-stabilizing imaging method, device and equipment of AR HUD and medium

Abstract

The invention provides an AR HUD self-stabilization imaging method, device and equipment and a medium, and effectively solves the problem of poor effect of an implementation mode for realizing an AR HUD function for a motorcycle. The method comprises the steps that real-time driving sitting posture information and real-time state information are input to a central controller in real time, inclination evaluation modes corresponding to multiple inclination dimensions are called, the real-time state information and the real-time driving sitting posture information are evaluated in real time, and a first inclination result and a second inclination result are obtained; and fusing the first tilt result and the second tilt result to obtain a fusion result, generating a tilt adjustment instruction for the AR HUD module in real time based on the fusion result, controlling the holder to respond to the tilt adjustment instruction, and performing adjustment based on adjustment information in the tilt adjustment instruction. Therefore, the development picture projected to the windshield glass by the projection unit on the holder is self-stabilized.

Description

Technical Field

[0001] The present application relates to a field, and specifically, to a self-stabilizing imaging method, device, equipment and medium for an AR HUD. Background Art

[0002] HUD (Head-Up Display) technology can project key information (such as vehicle speed, navigation instructions, etc.) into the driver's line of sight based on electronic modules and optical modules, enabling the driver to obtain information without lowering their head, thereby improving driving safety and convenience. It has made remarkable progress in recent years and has been applied to more and more vehicle models. In order to achieve the fusion of information such as navigation, ADAS, and vehicle signals, and perform image rendering and virtual-real overlay, AR HUD technology has emerged. AR HUD requires higher-precision hardware devices such as projectors, cameras, and sensors to support the realization of the augmented reality function.

[0003] Restricted by the volume of the hardware facilities of the AR HUD, it cannot be arranged on the motorcycle's headstock, resulting in the motorcycle not having the AR HUD function. However, in order to enable the motorcycle to also have the AR HUD function, two methods have been proposed. One is to implement side-screen image display based on a transparent screen, but it does not form a virtual image in front of the driver's field of view; the other is a windshield integrated with a transparent screen, but it still has a certain impact on the driver's field of view during driving. Summary of the Invention

[0004] In view of this, the purpose of the present application is to provide a self-stabilizing imaging method, device, equipment and medium for an AR HUD, which effectively solves the problem of poor implementation effect in the implementation methods for realizing the AR HUD function for motorcycles.

[0005] In a first aspect, an embodiment of the present application provides a self-stabilizing imaging method for an AR HUD, which is applied to a motorcycle. The motorcycle includes a central controller and a headstock part. The headstock part includes a windshield and an AR HUD module. The AR HUD module includes a pan-tilt head and a projection unit. The method includes:

[0006] Obtain the real-time driving sitting posture information of the driver when wearing a helmet and driving the motorcycle and the real-time state information of the motorcycle, and input the real-time driving sitting posture information and the real-time state information into the central controller in real time to match with a variety of preset tilt dimensions in the central controller to obtain a matching result;

[0007] Based on the matching result, call the tilt evaluation methods corresponding to the various tilt dimensions to respectively perform real-time evaluation on the real-time state information and the real-time driving sitting posture information to obtain a first tilt result and a second tilt result;

[0008] Fuse the first tilt result and the second tilt result to obtain a fused result, so that the central controller generates a tilt adjustment instruction for the AR HUD module in real time based on the fused result, and sends the tilt adjustment instruction to the pan-tilt head in the AR HUD module;

[0009] Control the pan-tilt head to respond to the tilt adjustment instruction, and adjust based on the adjustment information in the tilt adjustment instruction to self-stabilize the imaging picture projected by the projection unit on the pan-tilt head onto the windshield.

[0010] Combined with the first aspect, the embodiment of the present application provides a first possible implementation manner of the first aspect, wherein the real-time driving sitting posture information and the real-time status information are input into the central controller in real time to be matched with multiple preset tilt dimensions to obtain a matching result, including:

[0011] Extract the attributes in the real-time driving sitting posture information and the real-time status information respectively;

[0012] Based on the mapping relationship between the attributes and the tilt dimensions, determine the matching results of the real-time driving sitting posture information and the real-time status information with multiple tilt dimensions respectively.

[0013] Combined with the first aspect, the embodiment of the present application provides a second possible implementation manner of the first aspect, wherein the tilt dimension at least includes a status tilt dimension;

[0014] The real-time evaluation of the real-time status information and the real-time driving sitting posture information respectively to obtain a first tilt result and a second tilt result includes:

[0015] Based on the multiple sub-status information included in the real-time status information, make item-by-item comparisons with the preset standard sub-status information to obtain multiple comparison results;

[0016] Based on the multiple comparison results, generate the first tilt result for the real-time status information.

[0017] Combined with the first aspect, the embodiment of the present application provides a third possible implementation manner of the first aspect, wherein the tilt dimension at least includes a sitting posture tilt dimension;

[0018] The real-time evaluation of the real-time status information and the real-time driving sitting posture information respectively includes:

[0019] Call the comparison network preset in the sitting posture tilt dimension, and determine the comparison angle for the real-time driving sitting posture information;

[0020] Control the comparison network to compare the real-time driving photo where the real-time driving sitting posture information is located with the standard driving sitting posture photo based on a preset comparison angle, and obtain the second inclination result.

[0021] Combined with the first aspect, the embodiments of the present application provide a fourth possible implementation manner of the first aspect, wherein, the fusing the first inclination result and the second inclination result to obtain a fusion result includes:

[0022] Input the first inclination result and the second inclination result into a preset inclination fusion network;

[0023] Based on the inclination fusion network, process the first inclination result and the second inclination result to obtain a fusion result.

[0024] Combined with the first aspect, the embodiments of the present application provide a fifth possible implementation manner of the first aspect, wherein, before the central controller generates an inclination adjustment instruction for the AR HUD module in real time based on the fusion result, it includes:

[0025] Judge whether the fusion result meets the preset inclination adjustment condition for the AR HUD module;

[0026] If so, generate an inclination adjustment instruction for the AR HUD module in real time based on the fusion result.

[0027] Combined with the first aspect, the embodiments of the present application provide a sixth possible implementation manner of the first aspect, wherein, the pan-tilt head includes a motor;

[0028] The adjustment based on the adjustment information in the inclination adjustment instruction includes:

[0029] Control the pan-tilt head to read the adjustment information in the inclination adjustment instruction, and generate an adjustment instruction including an adjustment method based on the adjustment information;

[0030] Drive the motor to adjust the inclination angle of the pan-tilt head according to the adjustment instruction and the adjustment method.

[0031] In a second aspect, the embodiments of the present application provide a self-stabilizing imaging device for an AR HUD, which is applied to a motorcycle. The motorcycle includes a central controller and a front part. The front part includes a windshield and an AR HUD module. The AR HUD module includes a pan-tilt head and a projection unit. The device includes:

[0032] An acquisition module, configured to acquire real-time driving sitting posture information of a driver wearing a helmet when driving a motorcycle and real-time status information of the motorcycle, and input the real-time driving sitting posture information and the real-time status information into a central controller in real time, so as to match with multiple preset tilt dimensions of the central controller to obtain a matching result;

[0033] An evaluation module, configured to call tilt evaluation methods corresponding to the multiple tilt dimensions based on the matching result, so as to respectively perform real-time evaluation on the real-time status information and the real-time driving sitting posture information to obtain a first tilt result and a second tilt result;

[0034] A sending module, configured to fuse the first tilt result and the second tilt result to obtain a fusion result, so that the central controller generates a tilt adjustment instruction for the AR HUD module in real time based on the fusion result, and sends the tilt adjustment instruction to a pan-tilt head in the AR HUD module;

[0035] A self-stabilization module, configured to control the pan-tilt head to respond to the tilt adjustment instruction, and perform adjustment based on adjustment information in the tilt adjustment instruction, so as to self-stabilize an imaging picture projected by a projection unit on the pan-tilt head onto a windshield.

[0036] In a third aspect, an embodiment of the present application provides an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device runs, the processor communicates with the memory through the bus. When the machine-readable instructions are executed by the processor, the steps of any one of the self-stabilized imaging methods of an AR HUD are executed.

[0037] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, the steps of any one of the self-stabilized imaging methods of an AR HUD are executed.

[0038] An auto - stabilizing imaging method for an AR HUD provided by an embodiment of the present application is applied to a motorcycle. The motorcycle includes a central controller and a front part. The front part includes a windshield and an AR HUD module. The AR HUD module includes a pan - tilt head and a projection unit. The method first obtains the real - time driving sitting posture information of the driver wearing a helmet while driving the motorcycle and the real - time state information of the motorcycle, and inputs the real - time driving sitting posture information and the real - time state information into the central controller in real time to match with multiple preset tilt dimensions in the central controller to obtain a matching result. Secondly, based on the matching result, it calls the tilt evaluation methods corresponding to the multiple tilt dimensions to respectively evaluate the real - time state information and the real - time driving sitting posture information in real time to obtain a first tilt result and a second tilt result. Then, it fuses the first tilt result and the second tilt result to obtain a fusion result, so that the central controller generates a tilt adjustment instruction for the AR HUD module in real time based on the fusion result and sends the tilt adjustment instruction to the pan - tilt head in the AR HUD module. Finally, it controls the pan - tilt head to respond to the tilt adjustment instruction and adjust based on the adjustment information in the tilt adjustment instruction to self - stabilize the imaging picture projected by the projection unit on the pan - tilt head onto the windshield. Thus, when the motorcycle tilts, the imaging picture projected by the projection unit on the pan - tilt head onto the windshield tilts equally with the motorcycle, ensuring the driver's vision when driving the motorcycle, effectively solving the problem that the implementation method of the AR HUD function for motorcycles has poor effects. It avoids the driver's vision leaving the road during driving, distracting the driver's attention, and provides more vehicle information, and can adjust the imaging effect according to the left - right tilt angle of the road, and display the left - right tilt angle and the front - rear slope information in real time. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present application and should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0040] Figure 1 FIG. shows a schematic flow chart of the first auto - stabilizing imaging method for an AR HUD provided by an embodiment of the present application;

[0041] Figure 2 FIG. shows a schematic diagram of a motorcycle starting provided by an embodiment of the present application;

[0042] Figure 3 FIG. shows a schematic flow chart of another auto - stabilizing imaging method for an AR HUD provided by an embodiment of the present application;

[0043] Figure 4 Shows a schematic diagram of the AR HUD module provided by an embodiment of the present application;

[0044] Figure 5 Shows a schematic diagram of the tilt adjustment of the AR HUD module provided by an embodiment of the present application;

[0045] Figure 6 Shows a structural block diagram of the first self-stabilizing imaging device of the AR HUD provided by an embodiment of the present application;

[0046] Figure 7 Shows a schematic structural block diagram of the electronic device provided by an embodiment of the present application. Detailed implementation manners

[0047] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. It should be understood that the accompanying drawings in the present application are only for the purposes of illustration and description, and are not used to limit the protection scope of the present application. In addition, it should be understood that the schematic drawings are not drawn to actual scale. The flowcharts used in the present application illustrate the operations implemented according to some embodiments of the present application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and the steps without logical context relationships may be reversed or implemented simultaneously. In addition, those skilled in the art may add one or more other operations to the flowchart or remove one or more operations from the flowchart under the guidance of the content of the present application.

[0048] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application described and illustrated in the accompanying drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application claimed, but only represents the selected embodiments of the present application. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative efforts fall within the protection scope of the present application.

[0049] It should be noted that the term "including" will be used in the embodiments of the present application to indicate the existence of the features stated thereafter, but does not exclude adding other features.

[0050] Currently, in order to enable motorcycles to also have the AR HUD function, two methods have been proposed. One is to implement screen-side image display based on a transparent screen, but no virtual image is formed in front of the driver's field of view; the other is a windshield integrated with a transparent screen, but it still has a certain impact on the driver's field of view during driving.

[0051] Based on this, the embodiments of the present application provide a self-stabilizing imaging method, device, equipment and medium for an AR HUD, which will be described below through embodiments.

[0052] Embodiment 1

[0053] For the convenience of understanding this embodiment, first, a self-stabilizing imaging method for an AR HUD disclosed in the embodiments of the present application will be introduced in detail. As Figure 1 shown in the flowchart of a self-stabilizing imaging method for an AR HUD, a self-stabilizing imaging method for an AR HUD provided by the present application is applied to a motorcycle, and the motorcycle includes a central controller and a front part. The front part includes a windshield and an AR HUD module. The AR HUD module includes a gimbal and a projection unit. The method includes:

[0054] S101. Obtain the real-time driving sitting posture information of the driver wearing a helmet when driving the motorcycle and the real-time state information of the motorcycle, and input the real-time driving sitting posture information and the real-time state information into the central controller in real time to match with multiple preset tilt dimensions in the central controller to obtain a matching result;

[0055] S102. Based on the matching result, call the tilt evaluation methods corresponding to the multiple tilt dimensions to respectively evaluate the real-time state information and the real-time driving sitting posture information in real time to obtain a first tilt result and a second tilt result;

[0056] S103. Fuse the first tilt result and the second tilt result to obtain a fusion result, so that the central controller generates a tilt adjustment instruction for the AR HUD module in real time based on the fusion result, and sends the tilt adjustment instruction to the gimbal in the AR HUD module;

[0057] S104. Control the gimbal to respond to the tilt adjustment instruction to adjust based on the adjustment information in the tilt adjustment instruction to self-stabilize the imaging picture projected by the projection unit on the gimbal onto the windshield.

[0058] In step S101, when the driver starts to drive the motorcycle, as Figure 2The schematic diagram of motorcycle starting is shown as follows. First, a vehicle starting signal is sent using a remote control key. At this time, the central controller in the motorcycle starts the motorcycle based on the starting signal. The central controller in the motorcycle starts and establishes a communication connection with the cloud platform. The central controller in the motorcycle sends a wake-up signal based on CAN. The wake-up signal is respectively sent to the high-voltage ECU, the drive ECU, and the AR HUD module. The high-voltage ECU, the drive ECU, and the AR HUD module send self-check OK feedback signals based on the wake-up signal. At the same time, after the central controller receives the self-check OK feedback signal, it outputs a high-voltage power-on signal to the high-voltage ECU. The drive ECU outputs signals such as vehicle speed and driving mileage. Then the central controller outputs signals such as vehicle speed, driving mileage, etc., image information of the navigation, and signals such as vehicle speed, available driving mileage, remaining power / oil quantity, etc. to the AR HUD module. When the driver wears a helmet and drives the motorcycle, real-time driving sitting posture information of the driver when wearing the helmet and driving the motorcycle and real-time state information of the motorcycle are obtained. The real-time state information is obtained by real-time detection based on an inclination angle sensor and a slope sensor. The real-time state information includes multiple sub-state information. The multiple sub-state information includes lateral inclination angle information and longitudinal slope information. The real-time driving sitting posture information exists in the form of pictures and is collected in real time by a camera installed at the front part of the vehicle head, such as Figure 3 shown, and the real-time driving sitting posture information and the real-time state information are input into the central controller in real time. After the central controller receives the real-time driving sitting posture information and the real-time state information, it matches them with multiple preset inclination dimensions of the central controller to obtain a matching result, that is, the real-time driving sitting posture information and the real-time state information respectively have corresponding inclination dimensions. The matching result includes the inclination dimensions corresponding to the real-time driving sitting posture information and the real-time state information respectively. Thus, corresponding processing is performed on the real-time driving sitting posture information and the real-time state information based on the obtained matching result.

[0059] The AR HUD module is as Figure 4 shown, and includes a projection unit. The projection unit includes an imaging unit, a Diffuser screen, a microlens, and a microLED. The microLED displays the display image output by the central controller on the motorcycle windshield through the microlens and the Diffuser screen.

[0060] In the specific implementation process of step S101, there is an embodiment: the real-time input of the real-time driving sitting posture information and the real-time state information into the central controller to match with multiple preset inclination dimensions of the central controller to obtain a matching result includes:

[0061] S1011. Extract the attributes in the real-time driving sitting posture information and the real-time state information respectively;

[0062] S1012: Based on the mapping relationship between the attributes and the tilt dimensions, determine the matching results of the real-time driving posture information and the real-time status information with multiple tilt dimensions.

[0063] In steps S1011-S1012, the present application pre-extracts attributes from the real-time driving posture information and the real-time status information respectively. The attributes can be posture, status, or driver and vehicle, which can be determined specifically according to actual conditions. The extraction method is based on a deep neural network, which is trained based on historical driving posture information and status information, and pre-establishes a mapping relationship between the attributes and the tilt dimension. The tilt dimension includes a posture tilt dimension and a status tilt dimension. It can be that the posture attribute is mapped to the posture tilt dimension, and the status attribute is mapped to the status tilt dimension, or the driver attribute is mapped to the posture tilt dimension, and the vehicle attribute is mapped to the status tilt dimension. Based on the above mapping relationship, the matching results of the real-time driving posture information and the real-time status information with multiple tilt dimensions can be determined, thereby achieving that the real-time driving posture information and the real-time status information have corresponding tilt dimensions.

[0064] In step S102, the present application sets corresponding tilt evaluation methods for different tilt dimensions. After obtaining the matching result, the tilt evaluation methods corresponding to the multiple tilt dimensions are called based on the matching result. The tilt evaluation methods corresponding to the multiple tilt dimensions are obtained after multiple experiments. After setting the tilt evaluation methods corresponding to the multiple tilt dimensions, the real-time status information and the real-time driving posture information are respectively evaluated in real time based on the tilt evaluation methods corresponding to the multiple tilt dimensions to obtain a first tilt result and a second tilt result; the first tilt result is used to characterize whether the driver's body is tilted, and the second tilt result is used to characterize whether the motorcycle the driver is on is tilted.

[0065] In the specific implementation process of step S102, there is an embodiment in which: the tilt dimension at least includes a state tilt dimension;

[0066] The real-time evaluation of the real-time state information and the real-time driving posture information is performed respectively to obtain a first tilt result and a second tilt result, including:

[0067] S10211. Compare the multiple sub-state information included in the real-time state information with the preset standard sub-state information item by item to obtain multiple comparison results;

[0068] S10212. Generate the first tilt result for the real-time status information based on the multiple comparison results.

[0069] In steps S10211 - S10212, there are multiple sub - status information of the real - time status information of the present application. The multiple sub - status information includes lateral tilt angle information and longitudinal slope information. Therefore, corresponding standard sub - status information is preset for both the tilt angle information and the slope information. The standard sub - status information can be the threshold of the sub - status information. The tilt angle information and the slope information are compared item - by - item with the corresponding standard sub - status information to obtain multiple comparison results. The comparison results indicate the relationship between the sub - status information and the standard sub - status information. By comparing the sub - status information item - by - item, the performance of the motorcycle in various aspects can be understood more carefully, so as to generate the first tilt result more accurately. Based on the multiple comparison results, the first tilt result for the real - time status information is generated. The first tilt result can be a tilt degree score, and the first tilt result reflects the tilt or deviation degree between the current state of the vehicle and the standard state. It helps the driver or the vehicle management system to timely understand the running condition of the vehicle and take corresponding measures for adjustment or maintenance.

[0070] In the specific implementation process of step S102, there is another embodiment: the tilt dimension at least includes the sitting posture tilt dimension;

[0071] The real - time evaluation of the real - time status information and the real - time driving sitting posture information respectively includes:

[0072] S10221: Invoke the comparison network preset in the sitting posture tilt dimension and determine the comparison angle for the real - time driving sitting posture information;

[0073] S10222: Control the comparison network to compare the real - time driving photo where the real - time driving sitting posture information is located with the standard driving sitting posture photo based on the preset comparison angle to obtain the second tilt result.

[0074] In steps S10221 - S10222, the present application sets up a comparison network for the sitting posture inclination dimension. The comparison network is a pre - set convolutional neural network for image comparison. The comparison network is called, and the comparison angles for the real - time driving sitting posture information are determined. There are two comparison angles set by the present application, namely the central axis of the real - time driving photo and the standard driving sitting posture photo, and the mid - line of the driver's binoculars in the vertical direction in the real - time driving photo and the standard driving sitting posture photo. The standard driving sitting posture photo is pre - taken when the driver is driving a motorcycle and stored in the central controller. The comparison network is controlled to compare the real - time driving photo where the real - time driving sitting posture information is located with the standard driving sitting posture photo for each comparison angle based on the pre - set comparison angles. The comparison process involves steps such as image feature extraction, similarity calculation, and difference analysis, so as to obtain the second inclination result. The second inclination result includes the comparison results of the two comparison angles, and the second inclination result can be an inclination score that fuses the comparison angles.

[0075] In step S103, after obtaining the first inclination result and the second inclination result, the first inclination result and the second inclination result are fused based on a pre - set fusion method to obtain a fusion result. The fusion result comprehensively considers the inclination states of the vehicle and the driver, providing a more accurate basis for subsequent inclination adjustment, so that the central controller can generate an inclination adjustment instruction for the AR HUD module in real - time based on the fusion result. The inclination adjustment instruction includes specific adjustment information, and the adjustment information includes adjustment angle, adjustment method, adjustment direction, and adjustment speed. Generating the inclination adjustment instruction in real - time can ensure that the AR HUD module can respond to the inclination changes of the vehicle and the driver in a timely manner, maintain the accuracy of the projected information, and send the inclination adjustment instruction to the gimbal in the AR HUD module to ensure that the AR HUD module can accurately project the display screen into the driver's line of sight.

[0076] In the specific implementation process of step S103, there is an embodiment: fusing the first inclination result and the second inclination result to obtain a fusion result includes:

[0077] S10311: Input the first inclination result and the second inclination result into a pre - set inclination fusion network;

[0078] S10312: Process the first inclination result and the second inclination result based on the inclination fusion network to obtain a fusion result.

[0079] In steps S10311 - S10312, the present application pre - sets a fusion calculation network for fusing the first tilt result and the second tilt result. The first tilt result and the second tilt result are input into the pre - set tilt fusion network; based on the tilt fusion network, the first tilt result and the second tilt result are processed to obtain a fusion result. For example, if the first tilt result corresponds to a 60 - degree tilt and the second tilt result corresponds to a 30 - degree tilt, the fusion result can be 30 degrees obtained by subtracting the second tilt result from the first tilt result. Other fusion methods such as weighted calculation can also be adopted according to the actual situation, and specific settings can be made according to the actual situation.

[0080] In the specific implementation process of step S103, there is another embodiment: before the central controller generates a tilt adjustment instruction for the AR HUD module in real - time based on the fusion result, it includes:

[0081] S10321. Determine whether the fusion result meets the tilt adjustment condition preset for the AR HUD module;

[0082] S10322. If so, generate a tilt adjustment instruction for the AR HUD module in real - time based on the fusion result.

[0083] In steps S10321 - S10322, before the central controller of the present application generates a tilt adjustment instruction for the AR HUD module in real - time based on the fusion result, it also judges the fusion result to determine whether the fusion result meets the tilt adjustment condition preset for the AR HUD module. The tilt adjustment condition is whether the angle data corresponding to the fusion result is greater than a preset threshold. If so, that is, the angle data corresponding to the fusion result is greater than the preset threshold and tilt adjustment is required, the central controller generates a tilt adjustment instruction for the AR HUD module in real - time based on the fusion result. If not, that is, the angle data corresponding to the fusion result is less than the preset threshold, which means that the tilt angle of the driver when driving the motorcycle is too small and tilt adjustment is not required, then the central controller does not generate the tilt adjustment instruction at this time.

[0084] In step S104, the pan - tilt head in the AR HUD module receives the tilt adjustment instruction from the central controller through a communication interface (such as CAN bus, LIN bus or wireless communication module), controls the pan - tilt head to respond to the tilt adjustment instruction, and the pan - tilt head analyzes the received tilt adjustment instruction to extract adjustment information, so as to make adjustments based on the adjustment information in the tilt adjustment instruction to self - stabilize the imaging picture projected by the projection unit on the pan - tilt head onto the windshield, such as Figure 5As shown, during the adjustment of the pan-tilt head, through built-in sensors (such as gyroscopes, accelerometers, etc.), the tilt state of itself and the position of the projection unit are monitored in real time to obtain monitoring results. The pan-tilt head judges whether the adjustment reaches the expected effect according to the monitoring results. The pan-tilt head continues to monitor the state of the projection unit through the tilt sensor. If there is an offset or jitter, it will automatically perform fine adjustment to keep the display picture stable. This application also establishes a closed-loop control system for the pan-tilt head, compares the actual tilt state of the pan-tilt head with the expected tilt state, and automatically adjusts the actions of the pan-tilt head according to the deviation.

[0085] In the specific implementation process of step S104, there is an embodiment: the pan-tilt head includes a motor;

[0086] The adjustment based on the adjustment information in the tilt adjustment instruction includes:

[0087] S1041. Control the pan-tilt head to read the adjustment information in the tilt adjustment instruction, and generate an adjustment instruction including an adjustment method based on the adjustment information;

[0088] S1042. Drive the motor to adjust the tilt angle of the pan-tilt head according to the adjustment instruction and the adjustment method.

[0089] In steps S1041 - S1042, based on the traditional pan-tilt head, this application sets a motor on the pan-tilt head to adjust the tilt of the pan-tilt head. After receiving the tilt adjustment instruction, the pan-tilt head parses the received instruction, extracts the adjustment information therein, such as the adjustment angle, adjustment method, adjustment direction, adjustment speed, and generates a specific adjustment instruction. The adjustment instruction not only includes basic information such as the target tilt angle, but also contains the adjustment method, such as uniform adjustment, acceleration adjustment, step-by-step adjustment, etc., to ensure the smoothness and accuracy of the adjustment process. The motor controls the pan-tilt head to perform corresponding actions according to the target tilt angle and adjustment method in the adjustment instruction. During the adjustment process, the motor may need to amplify the torque through mechanical structures such as speed reducers and gear sets in the pan-tilt head to ensure that the pan-tilt head can reach the tilt angle smoothly and accurately. Through real-time monitoring and feedback mechanisms, ensure that the tilt angle of the pan-tilt head is accurate and make fine adjustments to the adjustment instruction if necessary. This application also installs a variety of high-precision sensors on the pan-tilt head to comprehensively monitor its tilt state and the operating state of the motor; also set up safety protection mechanisms, such as overcurrent protection, overvoltage protection, overheat protection, etc., to ensure that the pan-tilt head will not be damaged during the adjustment process; also set an emergency stop button or remote stop function to quickly stop the adjustment action of the pan-tilt head in case of abnormal situations.

[0090] Embodiment 2

[0091] This application also provides a self-stabilizing imaging device for an AR HUD, such as Figure 6The following is a block diagram of a self-stabilizing imaging device for an AR HUD. The functions implemented by this self-stabilizing imaging device for an AR HUD correspond to the steps of performing a self-stabilizing imaging method for an AR HUD on a terminal device. This device can be understood as a component of a server including a processor. The self-stabilizing imaging device for an AR HUD described in this application is applied to a motorcycle, which includes a central controller and a front part. The front part includes a windshield and an AR HUD module. The AR HUD module includes a gimbal and a projection unit. The device includes:

[0092] An acquisition module 601, configured to acquire real-time driving sitting posture information of a driver wearing a helmet while driving a motorcycle and real-time state information of the motorcycle, and input the real-time driving sitting posture information and the real-time state information into the central controller in real time, so as to match with multiple preset tilt dimensions in the central controller to obtain a matching result;

[0093] An evaluation module 602, configured to call tilt evaluation methods corresponding to the multiple tilt dimensions based on the matching result, so as to respectively perform real-time evaluation on the real-time state information and the real-time driving sitting posture information to obtain a first tilt result and a second tilt result;

[0094] A sending module 603, configured to fuse the first tilt result and the second tilt result to obtain a fusion result, so that the central controller generates a tilt adjustment instruction for the AR HUD module in real time based on the fusion result, and sends the tilt adjustment instruction to the gimbal in the AR HUD module;

[0095] A self-stabilizing module 604, configured to control the gimbal to respond to the tilt adjustment instruction, and perform adjustment based on the adjustment information in the tilt adjustment instruction, so as to self-stabilize the imaging screen projected by the projection unit on the gimbal onto the windshield.

[0096] In a feasible implementation manner, the evaluation module includes:

[0097] An extraction module, configured to respectively extract attributes in the real-time driving sitting posture information and the real-time state information;

[0098] A determination module, configured to determine the matching results of the real-time driving sitting posture information and the real-time state information with multiple tilt dimensions respectively based on the mapping relationship between the attributes and the tilt dimensions.

[0099] In a feasible implementation manner, the evaluation module further includes:

[0100] A comparison module, configured to perform item-by-item comparison between multiple sub-state information included in the real-time state information and preset standard sub-state information to obtain multiple comparison results;

[0101] A first generation module for generating the first tilt result for the real-time status information based on the multiple comparison results.

[0102] In a feasible implementation manner, the evaluation module further includes:

[0103] A calling module for calling a comparison network preset in the sitting posture tilt dimension and determining a comparison angle for the real-time driving sitting posture information;

[0104] A control module for controlling the comparison network to compare the real-time driving photo where the real-time driving sitting posture information is located with a standard driving sitting posture photo based on the preset comparison angle to obtain the second tilt result.

[0105] In a feasible implementation manner, the sending module includes:

[0106] An input module for inputting the first tilt result and the second tilt result into a preset tilt fusion network;

[0107] A processing module for processing the first tilt result and the second tilt result based on the tilt fusion network to obtain a fusion result.

[0108] In a feasible implementation manner, the sending module further includes:

[0109] A judgment module for judging whether the fusion result meets the tilt adjustment condition preset for the AR HUD module;

[0110] A second generation module for, if so, generating a tilt adjustment instruction for the AR HUD module in real time based on the fusion result.

[0111] In a feasible implementation manner, the self-stabilization module includes:

[0112] A reading module for controlling the gimbal to read adjustment information in the tilt adjustment instruction and generating an adjustment instruction including an adjustment method based on the adjustment information;

[0113] A driving module for driving the motor to adjust the tilt angle of the gimbal according to the adjustment instruction and the adjustment method.

[0114] Embodiment 3

[0115] This application also provides an electronic device, such as Figure 7As shown, it includes a processor 701, a memory 702, and a bus 703. The memory 702 stores machine-readable instructions executable by the processor 701. When the electronic device is running, communication is carried out between the processor 701 and the memory 702 via the bus 703. When the machine-readable instructions are executed by the processor 701, the steps of any one of the self-stabilizing imaging methods of an AR HUD are executed.

[0116] Embodiment 4

[0117] The present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, the steps of any one of the self-stabilizing imaging methods of an AR HUD are executed.

[0118] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described systems and devices can refer to the corresponding processes in the method embodiments, which will not be elaborated herein. In the several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation. For another example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some communication interfaces. The indirect couplings or communication connections of the devices or modules can be in electrical, mechanical, or other forms.

[0119] The modules described as separate components may or may not be physically separated. The components shown as modules may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0120] In addition, in each embodiment of the present application, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.

[0121] When the above-mentioned functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a platform server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, ROM, RAM, magnetic disks, or optical discs that can store program codes.

[0122] The above are only specific implementation manners of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of changes or substitutions, which should all be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A self-stabilizing imaging method for an AR HUD, characterized in that, Applied to a motorcycle, the motorcycle includes a central controller and a front part, the front part includes a windshield, and an AR HUD module, the AR HUD module includes a gimbal and a projection unit, and the method includes: Acquiring real-time driving posture information of a driver wearing a helmet while driving a motorcycle and real-time status information of the motorcycle, and inputting the real-time driving posture information and the real-time status information into a central controller in real time to match them with multiple tilt dimensions preset by the central controller to obtain a matching result; Based on the matching result, calling the tilt assessment methods corresponding to the multiple tilt dimensions to respectively assess the real-time state information and the real-time driving posture information in real time to obtain a first tilt result and a second tilt result; fusing the first tilt result and the second tilt result to obtain a fusion result, so that the central controller generates a tilt adjustment instruction for the AR HUD module in real time based on the fusion result, and sends the tilt adjustment instruction to the gimbal in the AR HUD module; The pan-tilt head is controlled to respond to the tilt adjustment instruction to make adjustments based on the adjustment information in the tilt adjustment instruction, so as to stabilize the display image projected by the projection unit on the pan-tilt head onto the windshield.

2. The method according to claim 1, characterized in that, The real-time input of the real-time driving posture information and the real-time status information to the central controller to match the real-time driving posture information and the real-time status information with the multiple tilt dimensions preset by the central controller to obtain a matching result, including: respectively extracting attributes from the real-time driving posture information and the real-time status information; Based on the mapping relationship between the attributes and the tilt dimensions, matching results between the real-time driving posture information and the real-time status information and the multiple tilt dimensions are determined.

3. The method according to claim 2, wherein The tilt dimension includes at least a state tilt dimension; The real-time evaluation of the real-time state information and the real-time driving posture information is performed respectively to obtain a first tilt result and a second tilt result, including: Comparing the multiple sub-state information included in the real-time state information with the preset standard sub-state information one by one to obtain multiple comparison results; Based on the multiple comparison results, the first tilt result for the real-time status information is generated.

4. The method according to claim 3, wherein The tilt dimension includes at least a sitting tilt dimension; The real-time evaluation of the real-time status information and the real-time driving posture information respectively includes: calling a comparison network preset in the sitting posture tilt dimension and determining a comparison angle for the real-time driving sitting posture information; The comparison network is controlled to compare the real-time driving photo containing the real-time driving posture information with the standard driving posture photo based on a preset comparison angle to obtain the second tilt result.

5. The method according to claim 1, wherein The fusing the first tilt result and the second tilt result to obtain a fusion result includes: Inputting the first tilt result and the second tilt result into a preset tilt fusion network; The first tilt result and the second tilt result are processed based on the tilt fusion network to obtain a fusion result.

6. The method according to claim 1, wherein Before the central controller generates a tilt adjustment instruction for the AR HUD module in real time based on the fusion result, the method includes: Determine whether the fusion result meets the tilt adjustment conditions preset for the AR HUD module; If so, generate a tilt adjustment instruction for the AR HUD module in real time based on the fusion result.

7. The method according to claim 1, characterized in that, The pan-tilt head includes a motor; The adjustment based on the adjustment information in the tilt adjustment instruction includes: Controlling the pan-tilt head to read the adjustment information in the tilt adjustment instruction, and generating an adjustment instruction including an adjustment method based on the adjustment information; Driving the motor to adjust the tilt angle of the pan-tilt head according to the adjustment instruction and the adjustment method.

8. An auto-stabilizing imaging device for an AR HUD, characterized in that, Applied to a motorcycle, the motorcycle includes a central controller and a front part, the front part includes a windshield and an AR HUD module, the AR HUD module includes a pan-tilt head and a projection unit, and the device includes: An acquisition module, configured to acquire the real-time driving sitting posture information of the driver wearing a helmet and driving the motorcycle and the real-time state information of the motorcycle, and input the real-time driving sitting posture information and the real-time state information into the central controller in real time, so as to match with multiple tilt dimensions preset in the central controller to obtain a matching result; An evaluation module, configured to call the tilt evaluation methods corresponding to the multiple tilt dimensions based on the matching result, so as to respectively evaluate the real-time state information and the real-time driving sitting posture information in real time to obtain a first tilt result and a second tilt result; A sending module, configured to fuse the first tilt result and the second tilt result to obtain a fusion result, so that the central controller generates a tilt adjustment instruction for the AR HUD module in real time based on the fusion result, and sends the tilt adjustment instruction to the pan-tilt head in the AR HUD module; A self-stabilization module, configured to control the pan-tilt head to respond to the tilt adjustment instruction, so as to adjust based on the adjustment information in the tilt adjustment instruction, so as to self-stabilize the imaging picture projected by the projection unit on the pan-tilt head onto the windshield.

9. An electronic device, characterized in that, Includes: A processor, a memory and a bus, the memory stores machine-readable instructions executable by the processor, when the electronic device runs, the processor communicates with the memory through the bus, and when the machine-readable instructions are executed by the processor, the steps of a self-stabilizing imaging method of an AR HUD as described in any one of claims 1 to 7 are executed.

10. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, and when the computer program is run by the processor, the steps of a self-stabilizing imaging method of an AR HUD as described in any one of claims 1 to 7 are executed.

Images