Interaction method and apparatus, and vehicle

Abstract

An interaction method and apparatus, and a vehicle, which relate to the technical field of human-machine interaction, and are used for improving the interaction experience of out-of-cabin human-machine interaction. The method comprises: in response to an interaction instruction, projecting an initial interaction interface outside a cabin, wherein the initial interaction interface comprises a plurality of sub-application icons; collecting an out-of-cabin image, and identifying the posture of a user in the out-of-cabin image; on the basis of the posture of the user, selecting a first sub-application icon, and projecting a first sub-application interface; and on the basis of a real-time action of the user outside the cabin, controlling the first sub-application interface to make a corresponding projection response. On the basis of the method, a user can implement human-machine interaction with a plurality of sub-applications outside a cabin, thus providing the user with a good sense of interaction; moreover, out-of-cabin interaction scenarios are relatively diverse, such that the out-of-cabin interaction experience of the user can be optimized, thereby implementing an intelligent out-of-cabin human-machine interaction solution.

Description

An interactive method, device and vehicle Technical Field

[0001] This application relates to the field of human-computer interaction technology, and in particular to an interaction method, device and vehicle. Background Technology

[0002] With the rapid development of intelligent cockpit technology, cars are no longer just a means of transportation for users, but are increasingly becoming a second home integrating entertainment, leisure, and other functions. At the same time, users' functional demands for intelligent cockpits are becoming increasingly diversified. In addition to providing basic in-cabin entertainment functions, they also hope that the cockpit can have a wider range of functions, such as external human-computer interaction and external entertainment functions.

[0003] In response to these demands, the industry has made some efforts to achieve simple external human-computer interaction solutions. However, these solutions require users to interact with the computer on an in-cabin display screen, and then project the interactive content outside the cabin. A common example is a user drawing on an in-cabin display screen, which is then projected onto the outside. Clearly, this solution has limitations; it lacks a sense of interaction for the user, and the scenarios are relatively limited, which is not conducive to improving the user's external interaction experience.

[0004] In summary, improving the interactive experience of human-machine interaction outside the cockpit is a technical problem that urgently needs to be solved in the field of intelligent cockpits.

[0005] Summary of the Invention

[0006] This application provides an interactive method, device, and vehicle for improving the interactive experience of human-computer interaction outside the cabin.

[0007] Firstly, this application provides an interaction method applicable to interactive devices. The interactive device can be a cockpit or its components, such as an in-vehicle infotainment system, domain controller, or vehicle controller; or it can be an external device or component, such as a user terminal, cloud server, roadside unit (RSU), or other vehicle. The method includes: responding to an interaction command by projecting an initial interactive interface outside the cockpit, the initial interactive interface including multiple sub-application patterns; acquiring an external image and recognizing the user's posture in the external image; selecting a first sub-application pattern based on the user's posture and projecting the first sub-application interface; and controlling the first sub-application interface to make a corresponding projection response based on the user's real-time actions outside the cockpit.

[0008] Based on the above methods, multiple sub-application icons can be projected outside the cabin. By sensing and recognizing the user's actions outside the cabin, one of the sub-application icons can be selected, and the corresponding sub-application interaction can be initiated. In this way, users can achieve human-computer interaction with multiple sub-applications outside the cabin, which provides a better interactive experience and offers a richer range of external interaction scenarios. This can optimize the user's external interaction experience and realize an intelligent external human interaction solution.

[0009] In one possible design, interactive commands can be triggered in the following ways: receiving information that the user clicks on the vehicle's infotainment screen or the user's terminal to access an external interactive application; or detecting that the user stands in front of the vehicle for a set duration and / or makes a set gesture.

[0010] Based on the above design, users can trigger interactive commands inside or outside the cabin. The triggering methods can be interface interaction, gesture recognition, or posture recognition, thus improving the flexibility of launching the external interactive entertainment function.

[0011] In one possible design, in response to an interactive command, before projecting the initial interactive interface outside the cabin, it can be determined that the vehicle is in park and its speed is 0. In other words, if the vehicle is not in park or its speed is not 0, even if an interactive command is received, the initial interactive interface will not be projected, i.e., the external interactive entertainment function will not be activated.

[0012] Based on the above design, it can be ensured that the extravehicular interactive entertainment function will not be turned on at inappropriate times, thus improving the safety of extravehicular interaction.

[0013] In one possible design, the initial interactive interface also includes a lock / power-off button and an exit application button. In response to user actions in the external image, if the exit application button in the initial interactive interface is selected, the external projection light is turned off; if the lock / power-off button in the initial interactive interface is selected, the external projection light is turned off, and the cabin lock / power-off is controlled.

[0014] Based on the above design, users can perform corresponding actions outside the cabin to directly shut down the vehicle or exit the external interaction scenario, avoiding the need for users to manually operate the vehicle again after entering the cabin, thus improving the user's external interaction experience.

[0015] In one possible design, the user's posture in the external image is identified, and the first sub-application pattern is selected based on the user's posture. Specifically, this can be done by identifying the user's orientation, hand posture, and key hand position in the external image. When the user's orientation is away from the cockpit, if the hand posture is the first hand posture, the cursor is controlled to move in the initial interactive interface based on the movement trajectory of the key hand position. Based on the second hand posture, the cursor is controlled to select the first sub-application pattern at the current position.

[0016] Based on the above design, different hand gestures and interface interaction methods are proposed when the user is facing away from the cockpit. Based on this interaction method, the user can select different sub-applications in the projection interface by changing the hand gesture and hovering it in the air, giving the user a brand-new extravehicular entertainment interaction experience.

[0017] In one example of the above design, the first hand posture is an open palm, and the second hand posture is a grasping motion.

[0018] Based on the above examples, when a user makes an outstretched hand gesture outside the cabin, the cursor on the projection screen will follow the hand's movement. Conversely, when a user makes a grasping gesture outside the cabin, the cursor on the projection screen will select the control at the current location. These two hand gestures match the user's familiar interface operation methods, making it easier for users to understand and operate, and reducing the difficulty of external interaction.

[0019] In one example of the above design, when the user is facing the cockpit, if the hand posture is the third hand posture, the volume of the external speakers is adjusted; if the hand posture is the fourth hand posture, the projection angle of the external projector is adjusted.

[0020] Based on the above examples, users can adjust the audio volume and projection interface position with gestures outside the cabin without having to manually adjust them inside the cabin, thus reducing the complexity of adjusting the playback volume and interface position during external interaction.

[0021] In a further example, the third hand gesture is a left hand sliding up and down, and the fourth hand gesture is a right hand sliding up and down. Or vice versa, that is, the third hand gesture is a right hand sliding up and down, and the fourth hand gesture is a left hand sliding up and down.

[0022] Based on the above examples, when a user makes a hand swiping motion outside the cabin, it will correspondingly increase or decrease the volume of the external playback, or raise or lower the position of the projection interface on the wall or screen, or move the projection interface on the ground closer or further away. This hand gesture matches the user's conventional understanding of adjustment methods, making it easier for the user to understand and operate, and reducing the difficulty of adjusting the interactive elements outside the cabin.

[0023] In one possible design, the first sub-application interface also includes an exit sub-application button. In response to user actions in the external image, if the exit sub-application button in the first sub-application interface is selected, the first sub-application interface will exit and the initial interactive interface will be projected.

[0024] Based on the above example, users can directly exit the sub-application and return to the initial projection interface by selecting the exit button on the sub-application interface through external actions, so that users can reselect another sub-application, improving the convenience of switching sub-applications.

[0025] In one possible design, after acquiring images of the external environment, the ambient brightness can be determined based on those images. If the ambient brightness is lower than a set brightness value, an external supplementary light is turned on. The brightness of the external supplementary light is lower than that of the external projection light.

[0026] Based on the above design, when the ambient light outside the cabin is insufficient, the external projection area can be actively illuminated to improve the brightness of the acquired external images and enhance the accuracy of user action recognition based on these images. Furthermore, setting the brightness of the external fill light to be lower than that of the external projection light also prevents the fill light process from affecting the projection effect of the external projection light.

[0027] In one possible design, multiple sub-application patterns include extravehicular activity patterns, personalized welcome patterns, hand shadow interactive patterns, and drawing board interactive patterns. Based on this design, multiple types of extravehicular interaction methods can be supported to meet the extravehicular entertainment needs of different users.

[0028] Based on the above design, the possible implementation methods of each sub-application will be explained below.

[0029] Scenario 1: The first sub-application pattern is an extravehicular activity pattern, and the first sub-application interface is an extravehicular activity interface.

[0030] In one possible design of Scenario 1, the first sub-application interface is controlled to make a corresponding projection response based on the user's real-time actions outside the cabin. Specifically, this can be achieved by acquiring inertial sensor information collected by the user terminal, combining the inertial sensor information with images outside the cabin, capturing the user's real-time actions, and then projecting them onto the external motion interface. The user terminal is carried on the user's body.

[0031] Based on the above design, user actions in the external images can be identified first, and then the user actions can be corrected using inertial sensor information to obtain accurate real-time user actions. In this way, even if the recognition results of the external images are inaccurate, the accuracy of detecting real-time user actions can be improved by leveraging the compensatory effect of inertial sensor information, thereby improving the accuracy of external interaction.

[0032] In one example of the above design, the user terminal includes a mobile phone and a fitness tracker. The mobile phone is placed in the user's pants pocket to collect inertial sensor information of the user's leg movements, while the fitness tracker is worn on the user's wrist to collect inertial sensor information of the user's hand movements.

[0033] Based on the above examples, the inertial sensor information of both the user's leg and hand movements can be combined to more accurately correct the user's movements identified in the external images, thereby further improving the accuracy of detecting the user's real-time movements.

[0034] In one example of the above design, capturing the user's real-time actions and projecting them onto the external motion interface can be achieved by combining the user's personalized virtual avatar with the user's real-time actions to generate a real-time motion video of the virtual avatar and projecting it onto the external motion interface.

[0035] Based on the above example, while the user exercises outside the cabin, a virtual avatar corresponding to the exercise is displayed in real time on the projection interface. Therefore, the user can intuitively see the difference between their own movements and the movements in the instruction video, making it easier for them to correct their movements. Furthermore, the presentation of the virtual avatar is more cute and flexible, which can enhance the user's sensory experience when viewing the exercise interface.

[0036] In one example of the above design, the extravehicular activity (EVA) interface includes a start motion button. Before acquiring inertial sensor information collected by the user terminal, in response to user actions in the extravehicular image, if the start motion button is selected, an action guidance user table is projected onto the EVA interface. The action guidance user table includes multiple action guidance items. In response to user actions in the extravehicular image, if the first action guidance item in the action guidance user table is selected, the video of the first action guidance item is projected onto the EVA interface. The video of the first action guidance item is used to guide the user to perform the same action outside the cabin.

[0037] Based on the above examples, users can select desired exercise guidance videos using gestures outside the cabin, meeting the exercise needs of different users.

[0038] In a further example, after capturing the user's real-time actions, motion information can be determined based on the user's real-time actions, similarity information can be determined based on the video of the user's real-time actions and the first action guidance item, and the motion information and similarity information can be projected onto the external motion interface; wherein, the motion information includes one or more of speed, timing, and counting, and the similarity information includes score.

[0039] Based on the above examples, users can intuitively understand the standard of their movements, as well as information such as exercise duration, number of movements, and range of motion on the projection interface, giving users a sense of accomplishment in exercising.

[0040] In a further example, after determining the similarity information based on the user's real-time actions and the video of the first action guidance item, if the similarity information indicates that the similarity between the user's real-time actions and the video of the first action guidance item is lower than the set similarity, then the external speakers are controlled to play action guidance voice.

[0041] Based on the above examples, when there is a significant difference between the user's movements and those in the instruction video, guiding audio can be played to visually inform the user how to correct their actions. In this way, even if the user is not yet proficient in the movement, they can still perform standard movements with the assistance of the guiding audio.

[0042] In one example of the above design, the extravehicular activity interface includes an "End Activity" button and a "Good Moment" button. In response to user actions in the extravehicular image, if the "End Activity" button is selected, a historical activity record is generated. The historical activity record includes a video of the user's real-time movement in this activity with reference to the extravehicular activity interface, as well as a real-time movement video of the user's personalized virtual avatar. In response to user actions in the extravehicular image, if the "Good Moment" button is selected, the historical activity record is projected.

[0043] Based on the above examples, the exercise record can be automatically saved when the exercise ends. In addition, users can also choose to play the current exercise record or the historical exercise record by gesture outside the cabin, so that users can summarize and deduce exercise experience based on the historical exercise record.

[0044] Scenario 2: The first sub-application pattern is a personalized welcome pattern, and the first sub-application interface is a personalized welcome interface.

[0045] In one possible design of Scenario 2, the personalized welcome interface includes an on / off welcome button and a off welcome button. Based on this, according to the user's real-time actions outside the cabin, the first sub-application interface is controlled to make a corresponding projection response. Specifically, in response to the user's actions in the outside image, if the user selects to turn on the personalized welcome button, multiple selectable personalized welcome settings are projected in the personalized welcome interface, and the personalized welcome settings selected by the user are saved; if the user selects to turn off the personalized welcome button, the personalized welcome interface is exited, and the initial interactive interface is projected.

[0046] Based on the above design, users can customize their welcome settings via actions outside the cabin, such as enabling or disabling the personalized welcome function, setting personalized welcome messages, personalized welcome light colors, and enabling or disabling vacant seat indicators. This design is intended for scenarios where users want to modify their personalized welcome settings after disembarking, or is suitable for users accustomed to setting personalized welcome functions outside the cabin.

[0047] In one possible design of scenario two, when the personalized welcome function is enabled in the cockpit, images outside the cockpit can be captured and faces in the images can be identified. If the face matches the first face stored locally in the cockpit, the personalized welcome settings of the first face are obtained, and a welcome guidance pattern is generated based on the personalized welcome settings of the first face and projected outside the cockpit.

[0048] Based on the above design, by recognizing faces in the external images, it is possible to determine which user has arrived at the vehicle, and then use the personalized greeting method set by that user to greet that user, in order to meet the user's personal habits.

[0049] In one example of the above design, if the empty seat indicator is enabled, the cabin interior image can be captured, and the location of the empty seat in the cabin interior image can be identified. Then, the empty seat location and personalized welcome settings are combined to generate a welcome guide pattern, which is then projected outside the cabin.

[0050] Based on the above examples, while greeting users, you can also indicate the location of empty seats in the cabin, so that users can sit directly on the empty seats and improve the efficiency of boarding.

[0051] In one example of the above design, if no face is recognized in the external image after the user leaves the vehicle, but the user terminal establishes a connection with the cabin communication module, a welcome guide pattern is generated based on the empty seat location and the default personalized welcome settings, and then projected onto the outside of the cabin.

[0052] Based on the above example, even if no face is recognized in the external image, the connection between the cockpit and the user terminal can be used to comprehensively detect whether a user has arrived near the vehicle, and then greet the user who arrives. This example can be adapted to scenarios where the user gets into the vehicle in the blind spot of the external camera, such as when the user is not used to coming to the front of the vehicle, but comes to the side or rear of the vehicle to get in, thus avoiding missing the user's need for a greeting.

[0053] In one example of the above design, the cabin usage scenario can also be identified and the corresponding atmosphere configuration information can be obtained. Then, after the door is detected to be open, the external projection light is turned off, and the ambient light inside the car is controlled to project an ambient image according to the ambient configuration information, and the ambient music inside the car is controlled to play ambient music.

[0054] Based on the above examples, specific themed ambient lighting and sound effects can be generated before the user boards the vehicle, and ultimately presented to the user from the moment the user gets in the car through ambient lighting and sound, providing the user with a full sense of atmosphere and improving the user experience.

[0055] Scenario 3: The first sub-application pattern is a hand shadow interactive pattern, and the first sub-application interface is a hand shadow interactive interface.

[0056] In one possible design of scenario three, based on the user's real-time actions outside the cabin, the first sub-application interface is controlled to make a corresponding projection response. Specifically, this can be done by: recognizing the user's gestures and shadows in the image outside the cabin, determining the hand shadow information, determining the first hand shadow category to which the hand shadow information belongs, then obtaining the first scene element that matches the first hand shadow category, projecting the pattern elements and text elements in the first scene element onto the hand shadow interaction interface, and controlling the outside speakers to play the sound effects elements in the first scene element.

[0057] Based on the above design, users can create hand shadows outside the cabin and receive corresponding projections and sound effects, enhancing the fun of hand shadow interaction.

[0058] In one example of the above design, determining the first scene element matched by the first hand shadow category can be specifically done by: querying the correspondence between multiple hand shadow categories and scene elements stored locally in the cockpit, determining the first scene element corresponding to the first hand shadow category, and if the first hand shadow category does not exist in the correspondence, then inputting the first hand shadow category into the preset scene generation algorithm to obtain the first scene element and storing it in the correspondence.

[0059] Based on the above examples, a preset scene generation algorithm can be used to generate scene elements corresponding to multiple hand shadow categories, which can then be stored locally in advance. When needed, directly querying the correspondence will likely retrieve the corresponding scene element, thus improving the efficiency of scene element retrieval. Furthermore, by setting a preset scene generation algorithm locally, it is also possible to adapt to scenes where scene elements cannot be found in the correspondence, achieving comprehensive response of the hand shadow interaction function to various scenes.

[0060] Scenario 4: The first sub-application pattern is a drawing board interactive pattern, and the first sub-application interface is a drawing board interactive interface.

[0061] In one possible design of scenario four, the first sub-application interface is controlled to make a corresponding projection response based on the user's real-time actions outside the cabin. Specifically, it can be: recognizing the user's orientation and hand posture in the image outside the cabin, and when the user's orientation is away from the cabin, the corresponding drawing operation is projected on the drawing board interaction interface according to the hand posture.

[0062] Based on the above design, the user's hand gestures can be captured using the hand posture in the external image, and the corresponding drawing operation can be determined. Then, it can be displayed in real time on the drawing board interactive interface, realizing the user's suspended drawing outside the cabin.

[0063] In one example of the above design, the canvas interaction interface includes a "Stop Drawing" button. Responding to user actions in the external image, if the "Stop Drawing" button is selected, the user's artwork is optimized, and the optimized artwork is projected. Optimization includes one or more of the following: correcting defects, adding filters, and changing styles.

[0064] Based on the above examples, the user's artwork can be optimized when the drawing process ends, presenting the user with a more perfect artwork.

[0065] In one example of the above design, based on the hand gesture, the corresponding drawing operation is projected onto the drawing board interactive interface. Specifically, it can include one or more of the following: if the hand gesture is grasping, the cursor on the drawing board interactive interface is controlled to perform a selection operation; if the hand gesture is waving, the cursor on the drawing board interactive interface is controlled to perform a deselection or return operation; if the hand gesture is open, the cursor on the drawing board interactive interface is controlled to follow the user's hand movement; if the hand gesture is extending a finger, the pen on the drawing board interactive interface is controlled to draw; or, if the hand gesture is clenching a fist, the drawing in the corresponding area on the drawing board interactive interface is controlled to erase.

[0066] Based on the above examples, we have shown the correspondence between various user hand gestures and different drawing operations. These hand gestures are more compatible with users' habitual movements, which can simplify the difficulty of users drawing in mid-air and improve the entertainment experience of users drawing outside the cabin.

[0067] Secondly, this application provides an interactive method applicable to interactive devices, which can be a cockpit or its components, such as an in-vehicle infotainment system, domain controller, or vehicle controller; or, they can be devices or components outside the cockpit, such as a user terminal, cloud server, roadside unit (RSU), or other vehicles, which are connected to the cockpit. The method includes: when the personalized welcome function is enabled in the cockpit, acquiring an external image and identifying a face contained in the external image; if the face matches a first face stored locally in the cockpit, obtaining the personalized welcome settings of the first face, generating a welcome guidance pattern based on the personalized welcome settings of the first face, and projecting it outside the cockpit.

[0068] Based on the above methods, by recognizing faces in the external images, it is possible to determine which user has arrived at the vehicle. Then, a personalized greeting method tailored to that user can be used to greet them. In this way, users can receive corresponding boarding prompts before boarding, which can improve the user's boarding experience.

[0069] In one possible design, a welcome guide pattern is generated based on the personalized welcome settings of the first face. Specifically, this could involve: capturing images inside the cabin and identifying the locations of empty seats inside the cabin, and generating a welcome guide pattern based on the locations of the empty seats and the personalized welcome settings of the first face.

[0070] Based on the above design, while welcoming users, the system can also indicate the location of empty seats in the cabin, making it easier for users to sit directly on the empty seats according to the instructions, thus improving the efficiency of users boarding the vehicle.

[0071] In one example of the above design, before acquiring images of the cabin interior, it can be determined whether the empty seat indicator is enabled. This could be because the cabin's empty seat indicator is enabled, the personalized welcome function has selected to enable the empty seat indicator, or the first person's personalized welcome settings have selected to enable the empty seat indicator. For example, taking the latter as an example, after determining the first person's personalized welcome settings, if the empty seat indicator is enabled in this personalized welcome setting, a welcome guidance pattern can be generated by combining the empty seat locations in the cabin with relevant content in the personalized welcome settings (such as welcome messages, welcome light colors, etc.). The welcome guidance pattern includes both the welcome messages and the empty seat location indicators. Conversely, if the empty seat indicator is not enabled in this personalized welcome setting, a welcome guidance pattern can be generated based solely on the welcome messages and welcome light colors, and this welcome guidance image will not include the empty seat location indicators.

[0072] Based on the above examples, you can also choose to enable or disable the empty seat indicator to meet the different welcoming needs of users.

[0073] In one possible design, acquiring external images can specifically include the following methods one and / or two:

[0074] Method 1: After detecting that the user has left the vehicle, if the user terminal re-establishes a connection with the cockpit communication module, then the acquisition of external images begins. The user terminal can be a car key or mobile phone, etc. Based on Method 1, welcome recognition can only begin when the system senses the user approaching the vehicle with a car key or mobile phone, instead of recognizing the user throughout the entire process after they leave the vehicle, thus saving unnecessary processing resources.

[0075] Method Two involves detecting that the vehicle is in neutral and the user has left the vehicle before starting to collect images of the exterior. Based on Method Two, welcome recognition can be performed even when the user is temporarily parked and does not have their car keys or mobile phone, improving the comprehensiveness of welcome scenarios.

[0076] In one possible design, after the user leaves the vehicle, if no face is recognized in the external image, but the user terminal establishes a connection with the cabin communication module, a welcome guide pattern is generated based on the default personalized welcome settings and projected onto the outside of the cabin. For example, the welcome guide pattern is generated based on the default personalized welcome settings and the location of an empty seat.

[0077] Based on the above design, even if no face is recognized in the external image, the connection between the cockpit and the user terminal can be used to comprehensively detect whether a user has arrived near the vehicle, and then greet the user who arrives. This example can be adapted to scenarios where the user gets into the vehicle in the blind spot of the external camera, such as when the user is not used to coming to the front of the vehicle, but comes to the side or rear of the vehicle to get in, thus avoiding missing the user's need for a greeting.

[0078] In one possible design, the cabin usage scenario can be identified and the corresponding ambient configuration information can be obtained. Then, after detecting that the car door is open, the external projection light is turned off, and the ambient light inside the car is controlled to project an ambient image according to the ambient configuration information, and the ambient music inside the car is controlled to play ambient music.

[0079] Based on the above design, specific themed ambient lighting and sound effects can be generated before the user boards the vehicle, and ultimately presented to the user from the moment the user gets in the car through ambient lighting and sound, providing the user with a full sense of atmosphere and improving the user experience.

[0080] In one possible design, the activation and deactivation of the personalized welcome function, as well as personalized welcome settings for different faces, can be set or entered by the user on the user terminal, the vehicle's infotainment system, other devices in the cabin, or a projection interface outside the cabin. For example, users can perform corresponding actions outside the cabin to select whether to activate the personalized welcome function and enter their personalized welcome settings on the external interactive interface; or, users can click relevant buttons on the vehicle's infotainment system or mobile phone to activate or deactivate the personalized welcome function and enter relevant configuration content to enter their personalized welcome settings; or, users can issue voice commands inside the cabin to directly instruct on or deactivate the personalized welcome function and how to configure their personalized welcome settings, and so on.

[0081] Based on the above design, users can manage information related to the personalized welcome function in multiple ways, satisfying the personal habits of different users in setting up personalized welcome functions and improving versatility.

[0082] In one possible design, the cockpit activates a personalized welcome function, which can be achieved as follows: In response to an interactive command, an initial interactive interface is projected outside the cockpit, including a personalized welcome pattern; external images are captured, and the user's posture in the external images is identified; based on the user's posture, the personalized welcome pattern is selected, and the personalized welcome interface is projected; based on the user's real-time actions outside the cockpit, the personalized welcome interface is controlled to make corresponding projection responses, and the user's personalized welcome information is saved. This personalized welcome information may include enabling or disabling the personalized welcome function, or it may also include the user's personalized welcome settings.

[0083] Based on the above design, users can turn the personalized welcome function on or off, or set their own personalized welcome configuration items, by moving outside the cabin. This method is suitable for users who want to modify the personalized welcome after getting off the vehicle, or for users who are used to setting the personalized welcome function outside the cabin.

[0084] In a further possible design, the personalized welcome interface includes an on / off welcome button. Based on the user's real-time actions outside the cabin, the personalized welcome interface is controlled to make corresponding projection responses and save the personalized welcome information set by the user. Specifically, in response to the user's actions in the external image, if the on / off personalized welcome button is selected, multiple selectable personalized welcome settings are projected on the personalized welcome interface and the selected personalized welcome settings are saved; if the off / off personalized welcome button is selected, the personalized welcome interface is exited and the initial interactive interface is projected.

[0085] Based on the above design, the user's actions can be projected onto the personalized welcome interface in real time, allowing the user to configure personalized welcome information outside the cabin and improving the flexibility of personalized welcome configuration.

[0086] Thirdly, this application provides an interactive device that can be a cockpit, or a component within the cockpit (such as a processor, vehicle infotainment chip, or chip system), or a component outside the cockpit (such as a user terminal, cloud server, RSU, other vehicle, or components thereof). The interactive device may include units or modules for performing the steps of the first aspect or any design or example thereof, or may include units or modules for performing the steps of the second aspect or any design or example thereof.

[0087] For example, in one embodiment, the interactive device may include a projection unit, a recognition unit, and a control unit, which can be used to perform various steps of the first aspect or any of the designs or examples in the first aspect above. Specifically, the projection unit is used to project an initial interactive interface outside the cabin in response to an interactive command, the initial interactive interface including multiple sub-application patterns; the recognition unit is used to acquire images outside the cabin and recognize the user's posture in the images outside the cabin; the projection unit is also used to select a first sub-application pattern according to the user's posture and project the first sub-application interface; the control unit is used to control the projection unit to make a corresponding projection response on the first sub-application interface according to the user's real-time actions outside the cabin.

[0088] For example, in another example, the interactive device may include a recognition unit, a generation unit, and a projection unit, which can be used to perform various steps of the second aspect or any of the designs or examples in the second aspect above. Specifically, the recognition unit is used to acquire an image outside the cabin when the personalized welcome function is activated in the cabin, and to recognize a face contained in the image outside the cabin; the generation unit is used to obtain the personalized welcome settings of the first face if it is determined that the face matches a first face stored locally in the cabin, and to generate a welcome guidance pattern based on the personalized welcome settings of the first face; the projection unit is used to project the welcome guidance pattern outside the cabin.

[0089] Fourthly, this application provides an interactive device, which can be a cockpit, or a component within the cockpit (such as a processor, vehicle infotainment chip, or chip system), or a component outside the cockpit (such as a user terminal, cloud server, RSU, other vehicle components, or their parts). The interactive device may include a processor, and optionally, may also include a memory (or storage medium). The memory stores program instructions; the processor can read the program instructions from the memory, causing the interactive device to execute the methods provided in the first aspect or any of the designs or examples in the first aspect, or to execute the methods provided in the second aspect or any of the designs or examples in the second aspect.

[0090] Optionally, there may be one or more processors and one or more memories.

[0091] Optionally, the memory can be integrated with the processor, or the memory can be set up separately from the processor.

[0092] In one possible design, the interactive device may further include a transceiver. The transceiver is used to receive and transmit signals; the processor, in response to signals received by the transceiver, executes program instructions to cause the interactive device to perform the methods provided in the first aspect or any of the designs or examples in the first aspect, or to perform the methods provided in the second aspect or any of the designs or examples in the second aspect. Optionally, the transceiver may include a transmitter and a receiver.

[0093] In another possible design, the interactive device also includes a communication interface, to which the processor is coupled. The processor reads program instructions from memory, invokes the communication interface to communicate with other devices, and executes the methods provided in the first aspect or any of the designs or examples in the first aspect, or executes the methods provided in the second aspect or any of the designs or examples in the second aspect. Optionally, the communication interface can be a transceiver, or an input / output interface. Optionally, the transceiver can be a transceiver circuit. Optionally, the input / output interface can be an input / output circuit.

[0094] Optionally, when the interactive device is a chip or chip system, the communication interface can be an input / output interface, interface circuit, output circuit, input circuit, pin, or related circuit on the chip or chip system. The processor can also be manifested as a processing circuit or logic circuit.

[0095] Fifthly, this application provides an interactive system including an interactive device, such as the interactive device described in any of the third or fourth aspects above, which is used to perform a method as described in any of the designs or examples in the first aspect above, or to perform a method as described in any of the designs or examples in the second aspect above.

[0096] In one possible design, the interactive system also includes onboard sensors, a vehicle control unit, and human-machine interaction elements. The onboard sensors are used to collect sensor information inside and outside the cabin and send it to the interactive device. The interactive device is used to send control signals to the vehicle control unit based on this sensor information. The control signals are used to control the vehicle control unit to drive the human-machine interaction elements to achieve the above-mentioned interactive methods.

[0097] Optionally, onboard sensors include, but are not limited to: external cameras, internal cameras, external microphones, and internal microphones. Human-machine interface components include, but are not limited to: external projection lights, external welcome lights, internal ambient lighting, external speakers, and internal speakers. Vehicle control units include, but are not limited to: external projection light controllers, external welcome light controllers, internal ambient lighting controllers, and audio controllers.

[0098] In a further possible design, the interactive system may also include a user terminal connected to the interactive device. The user terminal is used to collect user-related information and send it to the interactive device; the interactive device is used to combine the user-related information and sensor information collected by the vehicle's sensors to determine control signals and send them to the vehicle control unit.

[0099] Optionally, the user terminal includes, but is not limited to, mobile phones and fitness trackers.

[0100] Optionally, user-related information includes inertial sensor information during user movement.

[0101] Sixthly, this application provides an electronic device connected to an interactive vehicle for communicating with the interactive vehicle to implement the interactive method as described in the first aspect or any of the designs in the first aspect, or to implement the interactive method as described in the second aspect or any of the designs in the second aspect. The electronic device may include units or modules for implementing the interactive method as described in the first aspect or any of the designs in the first aspect, or may include units or modules for implementing the interactive method as described in the second aspect or any of the designs in the second aspect, such as including the interactive device as described in any of the third or fourth aspects.

[0102] Alternatively, the electronic device may be a user terminal, a cloud server, an RSU, or another vehicle.

[0103] Seventhly, this application provides a means of transportation that includes the interactive device provided in any one of the third or fourth aspects above, or the interactive system provided in the fifth aspect above.

[0104] Eighthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a computer, causes the computer to perform the method provided in the first aspect or any of the designs in the first aspect, or to perform the method provided in the second aspect or any of the designs in the second aspect. Optionally, the computer may be a cockpit or a component thereof.

[0105] Ninthly, this application provides a computer program product that, when run on a computer, causes the computer to perform the method provided in the first aspect or any of the designs in the first aspect, or to perform the method provided in the second aspect or any of the designs in the second aspect. Optionally, the computer may be a cockpit or a component thereof.

[0106] In a tenth aspect, this application provides a chip for reading a computer program stored in a memory and executing the method provided in the first aspect or any of the designs in the first aspect, or executing the method provided in the second aspect or any of the designs in the second aspect.

[0107] Alternatively, the chip can be an in-vehicle infotainment chip.

[0108] Optionally, the chip may include a processor coupled to a memory for reading a computer program stored in the memory to implement the method provided by the first aspect or any of the designs in the first aspect.

[0109] Optionally, the chip may also include components such as memory, communication interface, and power supply module. The memory is used to store computer programs; the communication interface is used to receive and send data; and the power supply unit is used to supply power to the processor.

[0110] In one aspect, this application provides a chip system including a processor for supporting a computer to implement the methods provided in the first aspect or any of the designs in the first aspect, or to implement the methods provided in the second aspect or any of the designs in the second aspect.

[0111] In one possible design, the chip system also includes memory for storing the computer's necessary programs and data. The chip system can consist of chips or include chips and other discrete components.

[0112] The technical effects that can be achieved in the third to eleventh aspects mentioned above can be referred to the descriptions of the beneficial effects in the first and second aspects mentioned above, and will not be repeated here. Attached Figure Description

[0113] Figure 1a illustrates a possible application scenario provided by this application;

[0114] Figure 1b illustrates another possible application scenario provided by this application;

[0115] Figure 1c illustrates another possible application scenario provided by this application;

[0116] Figure 1d illustrates another possible application scenario provided by this application;

[0117] Figure 2 illustrates an exemplary architectural diagram of an interactive system provided in this application;

[0118] Figure 3a exemplarily illustrates a schematic diagram of the exterior of a vehicle provided in this application;

[0119] Figure 3b illustrates an exemplary diagram of the installation location of an in-cabin camera provided in this application;

[0120] Figure 3c illustrates an exemplary schematic diagram of the installation position of an interior ambient light provided in this application;

[0121] Figure 4 illustrates a flowchart of an interactive method provided in this application;

[0122] Figure 5 illustrates a scenario diagram of an initial interactive interface for projection provided in this application;

[0123] Figure 6 is an exemplary schematic diagram of a possible presentation form of an initial interactive interface provided in this application;

[0124] Figure 7 illustrates, for example, a schematic diagram of the internal architecture of an initial interactive interface responding to user actions provided in this application;

[0125] Figure 8a illustrates an exemplary scenario of a user performing actions with their back to the cockpit, as provided in this application.

[0126] Figure 8b illustrates an exemplary interface response method for a conventional operation provided in this application;

[0127] Figure 8c illustrates an exemplary scenario of a user performing actions facing the cockpit, as provided in this application.

[0128] Figure 9 is an exemplary schematic diagram of the presentation form of an extravehicular motion interface provided in this application;

[0129] Figure 10 illustrates an exemplary internal process diagram for implementing external sports and entertainment functions provided in this application;

[0130] Figure 11 is an exemplary schematic diagram of the presentation of a personalized welcome interface provided in this application;

[0131] Figure 12a illustrates an exemplary flowchart of a personalized welcome function provided in this application.

[0132] Figure 12b illustrates an exemplary internal logic diagram of a personalized greeting function provided in this application.

[0133] Figure 12c exemplarily illustrates a projection interface diagram of a personalized welcome function provided in this application;

[0134] Figure 13a is an exemplary schematic diagram of the presentation form of a hand shadow interactive interface provided in this application;

[0135] Figure 13b illustrates an exemplary internal process diagram for implementing hand shadow interaction functionality provided in this application.

[0136] Figure 14a is an exemplary schematic diagram of the presentation form of a drawing board interactive interface provided in this application;

[0137] Figure 14b illustrates an exemplary internal process diagram for implementing a drawing board interactive function according to this application;

[0138] Figure 15 illustrates a schematic diagram of the structure of an interactive device provided in this application;

[0139] Figure 16 illustrates a schematic diagram of another interactive device provided in this application;

[0140] Figure 17 illustrates a schematic diagram of another interactive device provided in this application. Detailed Implementation

[0141] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0142] The following includes various details of embodiments of this application to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of this application. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0143] Furthermore, the data collection, storage, processing, transmission, and use mentioned in the embodiments of this application must comply with local laws.

[0144] It should be noted that if the information processed in this application involves user personal information (such as biometric information such as facial images, fingerprint information, voiceprint information, location trajectory information, and personal contact information), the processing of such user personal information will be based on legality, and the user will be fully informed and authorized, in accordance with the relevant laws and regulations on personal information protection in the country or region.

[0145] First, some terms used in this application will be explained. It should be noted that these explanations are for the convenience of those skilled in the art and do not constitute a limitation on the scope of protection claimed in this application.

[0146] I. Ultra-high performance (UHP) bulbs

[0147] UHP bulbs, also known as ultra-high pressure mercury bulbs, are an ideal cold light source with a long lifespan. Generally, UHP bulbs with a power of 100W to 120W can last up to 8,000 hours, and the longest can even reach 12,000 hours. After 4,000 hours of cumulative working time, their brightness will not show significant decay.

[0148] II. Digital Light Processing (DLP)

[0149] DLP technology is a technology that uses digital micromirror devices (DMDs) to display visual digital information. It first processes the image signal digitally before transmitting it to the DMD chip for projection. The DMD chip contains nearly a million rapidly rotating micromirrors, each corresponding to an optical pixel in the projected image. These micromirrors can rapidly change their angle under the control of digital drive signals. Upon receiving a corresponding signal, the micromirror tilts at a certain angle (e.g., 10°), changing the direction of the incident light reflection. A micromirror in projection mode is indicated as "on" and tilts +10° with the digital signal. If the micromirror is in non-projection mode, it is indicated as "off" and tilts -10°. Simultaneously, in the "on" state, the reflected incident light is projected onto the screen through a projection lens, while in the "off" state, the incident light reflected from the micromirrors is absorbed by a light absorber.

[0150] III. Color Wheel (CW)

[0151] A color wheel is an optical instrument that can form color images. The most common color wheel is a disc mounted on a central axis, on which multiple colors, such as red, green, and blue, are evenly or unevenly coated. When in use, the disc is rotated quickly, so the instantaneous image formed actually contains only one color. However, because the human eye reacts relatively slowly, the different colors appear to mix together when viewed, and the final result is the mixture of colors, which is the color image.

[0152] Next, we will introduce some possible application scenarios of the interactive methods provided in this application.

[0153] The interactive methods provided in this application can be applied to vehicles, especially vehicles with cabins, such as: vehicles (e.g., cars, trucks, smart cars, buses, trains, recreational vehicles, station wagons, vans, amusement park vehicles, construction vehicles, trams, golf carts, sightseeing vehicles, patrol cars, smart cars and digital cars, etc.), ships (e.g., passenger ships, cargo ships, ferries, cruise ships, dredgers, barges, bulk carriers or coal carriers, etc.), airplanes (e.g., passenger airplanes, cargo airplanes, helicopters, agricultural machinery, forest protection aircraft, aerial surveying aircraft, medical evacuation aircraft, sightseeing aircraft or weather aircraft, etc.), subways, high-speed rail, trains, light rail, etc.

[0154] Taking a vehicle as an example, please refer to Figures 1a, 1b, and 1c, which illustrate three possible application scenarios provided by this application. In the application scenario shown in Figure 1a, a user is camping in the wild. A screen is set up on the ground, and the vehicle is parked to one side of the screen with its front facing it. The vehicle's front projection light projects an exercise instruction video onto the screen, and the user stands to one side of the vehicle and follows the exercise instruction video. Figure 1b is also a camping scenario, but the difference is that a tent is used directly as a screen. The user sits sideways in front of the vehicle and makes hand shadows, which are projected onto the tent by the vehicle's front projection light, creating a fun and imaginative camping atmosphere. Figure 1c shows a user sitting in front of the vehicle drawing in the air. The user's drawing actions are projected in real time onto the screen in front of the vehicle, and the vehicle's front projection light, combined with the screen, functions as a drawing board.

[0155] Please refer to Figure 1d, which illustrates another possible application scenario provided by this application: vehicle welcome. As the user approaches the vehicle, a welcoming message is projected onto the ground around the vehicle, instructing the user to take their designated seat or an empty seat inside the vehicle. The welcoming message can be personalized text, such as the illustrative text "Welcome, handsome, come to your designated vehicle," and may also include a directional arrow pointing to the designated vehicle. Alternatively, it can be a default welcoming message or other welcoming messages; the specifics are not limited.

[0156] It should be understood that the four application scenarios above are merely examples, and the interactive methods provided in this application can also be applied to other possible scenarios, not limited to those listed above. For example, a user could play a game on an external game screen projected by a vehicle's projector lights, or watch a movie on a vehicle's projector lights outside the cabin, adjusting the movie volume, viewing position, or fast-forwarding / rewinding the video with gestures. Another example is the application of the interactive method to projectors, where the projector projects the image onto a projection screen, recognizes the user's real-time actions, and responds on the projected image. Yet another example is its application in smart home scenarios, such as LCD TVs, enabling the TV to adjust the volume or image of the video being played in accordance with the user's real-time actions, thus enhancing the user's smart home experience. Furthermore, the interactive method can be applied to road lighting scenarios, adjusting the brightness, color temperature, or color difference of streetlights by detecting the driver's gestures as a vehicle passes, providing suitable ambient light for the driver and reducing traffic accidents. For example, interactive methods can also be applied to public areas such as cinemas, shopping malls, high-speed rail stations, airports, bus stations, schools, parks, communities, squares, churches, scenic areas, and architectural areas. In these public areas, different game visuals are projected in response to the user's real-time actions, providing a comfortable gaming environment and enhancing the user experience. And so on. These are just a few examples.

[0157] It should be noted that the application scenarios described in this application are for the purpose of more clearly illustrating the technical solutions of this application, and do not constitute a limitation on the technical solutions provided in this application.

[0158] As described in the background technology description, current mainstream extravehicular interaction methods have certain limitations, resulting in a poor user experience. These limitations manifest in two main aspects: firstly, the system architecture of extravehicular interaction is not comprehensive enough, and the platform covers a limited range of scenarios; secondly, the platform's capabilities are limited, the algorithms are not sophisticated enough, and the interactivity is insufficient.

[0159] To address these two shortcomings, this application proposes an interactive solution based on the collaboration between cabin hardware and software, taking into account both inside and outside the cabin. This solution can provide users with an intelligent out-of-cabin entertainment experience, enhancing user interaction while covering more interactive scenarios and improving the user's out-of-cabin interactive experience.

[0160] The interactive scheme proposed in this application will be described in detail below with reference to the specific accompanying drawings.

[0161] First, taking the application of an interactive solution to a vehicle as an example, we provide a system for realizing external interaction functions, referred to as the interactive system. This interactive system includes a variety of hardware ecosystems for achieving human-machine interaction. For example, please refer to Figure 2, which shows a possible architectural diagram of the interactive system. This system architecture includes onboard sensors, interactive devices, a vehicle control unit, and human-machine interaction components. Optionally, it also includes a user terminal. Each component is described below with examples.

[0162] Vehicle-mounted sensors, belonging to the hardware input units of the interactive system, include various sensors used to detect the internal and external environments of the vehicle, such as vision sensors, including but not limited to external cameras (e.g., a four-way fisheye camera) and internal cameras. External cameras are typically installed on the roof, next to the license plate light, inside the bumper, on the door, or at the bottom of the rearview mirror, primarily responsible for capturing external images. For example, Figure 3a shows a vehicle exterior view with an external camera mounted on the roof, capable of capturing images within a certain angle range directly in front of the vehicle. Optionally, in other examples, multiple external cameras can be arranged around the vehicle body to capture images within a 360° range around the vehicle, or the capture range may be less than 360°, i.e., with blind spots, without limitation. Internal cameras are typically installed in the center above the windshield, as shown in Figure 3b. In some examples, they can also be installed in other locations inside the cabin, such as the center above the rear windshield, or at the inner door handle. The cabin camera is mainly responsible for collecting images inside the cabin, which can be used to identify the passenger situation, such as the location of empty seats.

[0163] In some scenarios, onboard sensors may also include other visual sensors, such as radar (e.g., lidar, millimeter-wave radar, or ultrasonic radar), as well as non-visual sensors, such as external and internal microphones. Radar can be used to collect point cloud data within a 360° range around the vehicle body, which can assist external cameras in more accurately identifying the external environment. External microphones can be used to collect external voice, while internal microphones are used to collect internal voice. Both internal and external voice can be used to recognize user commands, assisting users in achieving richer external interaction scenarios.

[0164] Interactive devices, as core components of interactive systems, can be devices specifically designed for external interaction or devices that perform other functions in addition to external interaction. For example, in one scenario, the interactive device could be a control unit within the vehicle, such as an in-vehicle infotainment chip, a cockpit domain controller (CDC) or other domain controllers, or a vehicle control unit (VCU). This allows the use of existing vehicle control units to achieve external interaction, improving the utilization rate of in-vehicle components. Alternatively, to reduce the workload of in-vehicle control units, the interactive device could be an additional control unit specifically designed for external interaction, such as a dedicated digital signal processing (DSP) chip. This DSP chip contains all the components necessary for digital signal processing, including but not limited to: power amplifiers, analog-to-digital converters (DACs), digital-to-analog converters (ADCs), and processing units. This DSP chip is independent of the vehicle but can connect to relevant vehicle components to achieve external interaction in conjunction with those components. Alternatively, in another example, the interactive device could be a cloud server, terminal equipment, RSU, or other vehicle. This interactive device can connect to relevant components in the external interactive vehicle via a network, such as the onboard sensors, vehicle control unit, and user terminal shown in Figure 2, to realize an external interactive control scheme for the external interactive vehicle. Understandably, in some examples, the interactive device can also be a combination of the above examples; for instance, some functions of the interactive device may be implemented by a separately set control unit, while other functions may be implemented by a control unit in the vehicle. And so on, without further listing.

[0165] Optionally, taking the vehicle-mounted chip as an example of an interactive device, the chip has algorithm calculation and logic processing functions, and various software modules are locally installed, such as the sensing base, recognition algorithm, user profile, and visual effects algorithm shown in Figure 2. The sensing base has the basic capability of human information detection, which can detect the human body, face, hand, and key point positions in the input information, and is the basis for subsequent recognition algorithms. The recognition algorithm can identify human actions, gestures, scenes, and user identities (IDs), such as Face ID, based on the human information detected by the sensing base. The user profile is used to maintain the personal exclusive information of one or more users. It stores each user's personalized settings, habits, biometric information, and other personal exclusive information under the corresponding account. Based on the user ID identified by the recognition algorithm, the personal exclusive information of a specified user can be provided. The visual effects algorithm can use known user, scene, and recognition information to generate personalized visual effects, voiceprint effects, and language effects through artificial intelligence (AI) and other methods to provide users with a more realistic interactive atmosphere.

[0166] The vehicle infotainment chip can also contain configuration information for an "external interaction" application (APP). The vehicle infotainment system provides information and control through the external interaction APP interface. The external interaction APP includes multiple sub-APPs and their corresponding configuration information. The vehicle infotainment system can provide a user-visualized and operable real-time platform for each sub-APP's application scenarios, displayed to the user through human-machine interface elements. It can also provide basic computing and storage services for each sub-APP. In some scenarios, the external interaction APP can be simultaneously installed in both the vehicle infotainment chip and the user terminal. Users can activate the external interaction function by clicking on the external interaction APP on the user terminal, and the vehicle infotainment chip recognizes the user's actions outside the cabin and provides corresponding projection responses.

[0167] Human-computer interaction components, belonging to the hardware output units of the external interaction system, include various components that directly interact with the user, such as external projection lights, interior ambient lights, external welcome lights, external speakers, and interior speakers. External projection lights are used to display external projected images to the user. DLP projection lights can be used. Their imaging principle involves focusing the cold light emitted by a UHP bulb into a beam of light through a focusing lens, then homogenizing the beam with a light bar, and finally splitting the uniform light into red-green-blue (RGB) colors using a color wheel. The mixed colored light is then projected onto a DMD chip through a lens, and the colored light reflected from the DMD chip is imaged by a projection lens. In some examples, the DLP projection light is one or more small lights inside the vehicle's headlights, as shown in Figure 3a. The vehicle's headlights also contain other lights, which can serve as supplementary lighting in external interaction scenarios or as illumination in non-external interaction scenarios. Interior ambient lighting includes strip lighting, dot lighting, and projection lighting, typically located on door panels, the edge of the center console, under seats, and the dashboard, as shown in Figure 3c. Ambient lighting can project light onto the floor, ceiling, or sunroof, enhancing the interior atmosphere and personalizing the space. External welcome lights project light onto the ground outside the vehicle, usually located at the bottom of the doors or on the sides, providing clear entry and exit paths for users. External audio systems are primarily used to play music or provide information from outside the cabin; their installation location varies depending on the model, but they are typically installed in the bumper or grille, and sometimes on the side of the vehicle, as shown in Figure 3a. Internal audio systems are used to play music or provide information inside the cabin, usually positioned near the A-pillar triangular window or on the C-pillar, and sometimes on the door trim panels or other locations.

[0168] The vehicle control unit, also known as the vehicle infotainment control unit, is mainly used to implement the interaction schemes and APP interfaces determined by the vehicle infotainment chip onto the human-machine interface elements. For example, it outputs certain control signals to the human-machine interface elements to control them to perform corresponding operations. Corresponding to the various human-machine interface elements shown in Figure 2, the vehicle control unit may include a projection light controller (such as a projection light switch controller and a projection light angle controller), an ambient light controller, a welcome light controller, an exterior audio controller (such as a switch controller and a volume controller), and an interior audio controller. When the exterior projection light is a DLP projection light, the projection light controller can specifically be a DLP digital projection headlight controller. The DLP digital projection headlight controller can communicate with the vehicle's left front headlight module and right front headlight module to control the headlights and one or more internal small lights.

[0169] The user terminal can include various terminal devices carried by the user, such as mobile phones, fitness trackers, tablets, laptops, smartwatches, smart headphones, and other wearable devices. In some scenarios, it may also include a car key. Optionally, the user terminal can register in advance with the in-vehicle communication module to establish a network connection with it. For example, it can interact with the in-vehicle communication module through short-range communication technologies, including but not limited to: Bluetooth, Wi-Fi, Near Field Communication (NFC), and Wi-Fi Aware. Based on this short-range communication technology, when the user moves a certain distance away from the vehicle, the user terminal can automatically disconnect from the in-vehicle communication module, locking and unlocking the vehicle. When the user returns to the vehicle within a certain distance, the user terminal automatically re-establishes a connection with the in-vehicle communication module, powering on and unlocking the vehicle. This achieves the intelligent vehicle management effect of locking upon leaving the vehicle and powering on upon returning.

[0170] Based on the interactive system shown in Figure 2, a multi-dimensional and three-dimensional data acquisition and information dissemination channel can be constructed to realize the interactive method provided in this application. The collaborative process of each component is as follows: The hardware input unit collects the information required for its respective scenario based on the user's current application scenario and inputs the information into the interactive device; the interactive device sequentially passes through the sensing base and recognition algorithm to obtain the user interaction recognition results for each scenario, and selectively performs visual effect algorithm calculations and user profile matching according to the specific scenario to generate a comprehensive interactive scheme with personalized interaction; the interactive device generates the interface of the external interactive APP in real time based on the interactive scheme and applies the interactive scheme and APP interface to the vehicle control unit; the vehicle control unit controls the hardware output unit based on the interactive scheme and APP interface to achieve a closed loop of user interaction.

[0171] It should be understood that the architecture of the above interactive system is only an example. In other examples, the interactive system may include more, fewer, or different components, and each component may include more, fewer, or different elements. Furthermore, the components shown or not shown may be combined or divided in any way. The division of the components shown is only a logical functional division. In actual implementation, they may be fully or partially integrated into a single physical entity or physically separated. This application does not make any specific limitations on this.

[0172] Based on the interactive system shown in Figure 2, please refer to Figure 4, which shows a flowchart of an interactive method provided in this application. This method is applicable to interactive devices, such as the interactive device shown in Figure 2. As shown in Figure 4, the method includes the following steps 401, 402, and 403:

[0173] Step 401: In response to the interactive command, an initial interactive interface is projected outside the cabin. The initial interactive interface includes multiple sub-application icons.

[0174] Here, interactive commands are used to indicate entry into the extravehicular interactive scenario. Interactive commands can be triggered through various human-computer interaction methods, including but not limited to: voice interaction, interface interaction, gesture interaction, application (APP) interaction, button interaction, brainwave interaction, etc.

[0175] Taking voice interaction as an example, in one triggering method, when a user wants to have an interactive entertainment experience outside the cabin, the user can say "start the external interaction" or similar voice content inside or outside the cabin and close to the vehicle. After receiving the voice content, the microphone inside or outside the cabin sends it to the interactive device. The interactive device recognizes the voice content and determines that the interactive command has been received.

[0176] Taking APP interaction as an example, in one triggering method, a control for the "Out-of-Cockpit Interactive APP" is set on the vehicle's infotainment screen. When a user wants to engage in an out-of-cabin interactive entertainment experience, the user can switch the vehicle to park and click the control on the infotainment screen. The infotainment screen then notifies the interactive device that the user has clicked the control, and the interactive device confirms receipt of the interactive command. In another triggering method, a control for the "Out-of-Cockpit Interactive APP" is set on the user's terminal (such as a mobile phone). When a user wants to engage in an out-of-cabin interactive entertainment experience, the user can click the control on the user's terminal within a certain distance from the vehicle. For example, the user can click it before leaving the vehicle or click it near the vehicle after leaving the vehicle to ensure that the user's terminal and the vehicle are connected. In this way, the user's terminal can notify the interactive device that the user has clicked the control, and the interactive device confirms receipt of the interactive command.

[0177] Taking gesture interaction as an example, in one triggering method, when a user wants to engage in an external interactive entertainment experience, the user can stand in front of the vehicle for a set duration and / or make a set gesture. For example, the user leaves the car keys in the car, leaves, closes the door, stands in front of the car for a few seconds, and makes a snapping gesture. The interactive device (or other unit in the vehicle) detects the door opening and closing. If it determines that the car keys have not left the vehicle, it activates the external camera to capture external images and recognizes the user's posture in the external images. When it recognizes that the user has stood in front of the car for a few seconds and made a snapping gesture, it determines that an interactive command has been received. Optionally, it can also be that the external camera captures the user's snapping gesture, and the external speakers also capture the snapping sound, before determining that an interactive command has been received, thereby improving the accuracy of starting the external interactive scene.

[0178] Optionally, after receiving the interaction instruction, the interaction device can directly turn on the external projection lamp and control the external projection lamp to project the initial interaction interface. Alternatively, it can first determine whether the current state of the vehicle meets the preset safety conditions. After determining that the preset safety conditions are met, it then turns on the external projection lamp and controls the external projection lamp to project the initial interaction interface. Taking the latter as an example, the preset safety conditions may include, for example: the vehicle is in the parking gear and the vehicle speed is 0. When the current state of the vehicle meets the preset safety conditions, it indicates that the current state of the vehicle is suitable for performing the external interaction entertainment function. The interaction device can turn on the external projection lamp and project the initial interaction interface. On the contrary, it indicates that the vehicle may be in motion. In this case, projection is not possible. The interaction device does not need to turn on the external projection lamp. That is, the external interaction entertainment function is locked and cannot be opened, so as to ensure that the external interaction entertainment function will not be opened at an inappropriate time and improve the safety of external interaction.

[0179] Further, taking the external projection lamp as a DLP projection lamp as an example, when controlling the external projection lamp to project the initial interaction interface, the interaction device can obtain the configuration information of the initial interaction interface from the APP information stored locally and send a control instruction to the DLP digital projection headlight controller. The control instruction contains the positions of the microlenses in the DLP projection that need to be lit to present the initial interaction interface. The DLP digital projection headlight controller controls the microlenses at the corresponding positions to turn through the corresponding angles, so that these microlenses are switched to "on", and the other microlenses remain "off". In this way, the light beams emitted by the "on" microlenses pass through the light wheel to form a colored initial interaction interface.

[0180] In some scenarios, the initial interaction interface can be imaged on the ground or on a screen. The screen can be a wall, or a projection screen pre-built by the user, or a tent, or other materials that can display images. For example, FIG. 5 is an example of projecting the initial interaction interface onto the ground. The vehicle uses the DLP digital projection headlights in the left headlight module and the right headlight module to project the initial interaction interface on the ground in front of the vehicle head. The position of the initial interaction interface is related to the projection angle of the DLP digital projection headlights.

[0181] In some scenarios, the initial interaction interface can be displayed only outside the vehicle and not on the in-vehicle screen to save power. Or, in some other scenarios, it can also be displayed both outside the vehicle and on the in-vehicle screen to form a linkage between the in-vehicle screen and the external projection. This application does not make any restrictions on this.

[0182] Optionally, the initial interactive interface includes multiple sub-application icons, which include at least two of the following: extravehicular activity icons, personalized welcome icons, hand shadow interaction icons, and drawing board interaction icons. For example, taking the simultaneous inclusion of these four sub-application icons as an example, please refer to Figure 6, which shows a possible presentation of the initial interactive interface. In this example, each sub-application icon includes an icon and text to indicate the function of the sub-application. In this way, even if the user cannot understand the icon, they can select the desired sub-application through the text.

[0183] In one example, as shown in Figure 6, the initial projected interface may also include a cursor (the blank arrow in the lower right corner of the figure), also known as a mouse cursor, used to indicate the current position of the mouse. The cursor can have various presentation styles, such as the arrow shown in the figure, a short vertical line, or other forms, or different presentation styles depending on the situation. The cursor can be static or dynamic, such as flashing regularly. The cursor can be displayed or hidden. A hidden cursor does not exist when the interface is static; once the interface changes, the hidden cursor is displayed, and it is hidden again when the interface becomes static. This application does not limit the content related to the cursor.

[0184] Step 402: Acquire external images and identify user postures in the external images. Select the first sub-application pattern based on the user posture and project the first sub-application interface.

[0185] Optionally, after controlling the external projection light to project the initial interactive interface, the interactive device can also control the external camera to capture and obtain external images. The external images are related to the field of view of the external camera. For example, in the layout shown in Figure 3a, the external camera can capture images within a certain angle range directly in front of the vehicle. Therefore, only users within this angle range directly in front of the vehicle will be captured in the external image. This layout is designed based on the user's typical actions in an external interactive scenario. If the interface is to be projected onto the front of the vehicle using the headlights, the user will inevitably go to the front of the vehicle. Therefore, only images within a certain range in front of the vehicle can be captured, reducing the requirement for the external camera's field of view capability.

[0186] Alternatively, in other examples, to ensure more comprehensive acquisition of external images, multiple external cameras can be installed on the roof or body of the vehicle, capturing 360° environmental images around the vehicle. This way, even if some users are not accustomed to interacting with the vehicle from the front, their posture can be detected, allowing for flexible adaptation to assist them in completing the external interaction.

[0187] Furthermore, after acquiring the external image, the interactive device can also identify the ambient brightness based on the image. For example, it can input the external image into the sensing base and obtain the ambient brightness output by the sensing base. If the ambient brightness is lower than a set value, it indicates that the external scene is too dark. In this case, the external supplementary lighting can be turned on. The external supplementary lighting is located around the external projection light. For example, when the external projection light is a DLP projection light in a vehicle's headlight module, the external supplementary lighting can be considered as any other light in the headlight module besides the DLP projection light. When turning on the external supplementary lighting, its brightness must be lower than that of the external projection light to avoid affecting the projection effect. In some scenarios, to achieve better supplementary lighting, the external camera can be controlled to acquire external images at a certain acquisition cycle. Based on the ambient brightness in the acquired external images, the brightness of the external supplementary lighting can be adjusted to a suitable level for adaptive supplementary lighting.

[0188] Understandably, other methods can also be used to identify the ambient brightness outside the cabin. For example, in another example, an ambient light sensor could be installed outside the cabin. After the interactive device controls the external projector to project the initial interactive interface, it could control the ambient light sensor to collect the ambient brightness outside the cabin and compare it with a set brightness value. If it is lower than the set brightness value, the external supplementary light would be turned on. Alternatively, in yet another example, brightness recognition could be skipped initially, and user posture recognition could be performed directly on the collected external image. If the recognition result is poor, such as failing to recognize the user posture, it could then be determined whether the recognition error is due to insufficient ambient brightness outside the cabin. If so, the external supplementary light would be turned on to provide additional illumination. There are many other possible methods, which will not be listed here.

[0189] Furthermore, when the ambient light outside the cabin exceeds a set brightness value, or after supplemental lighting is provided to ensure the ambient light exceeds the set brightness value, the interactive device can recognize the user's posture in the external image. This user posture specifically refers to the user's orientation, hand posture, and the position of key hand points. For example, please refer to Figure 7, which shows the internal architecture diagram of an initial interactive interface responding to user actions provided in this application. In this architecture, after the external image captured by the external camera is input to the interactive device, the interactive device first calls the sensing base to perform key point detection on the external image of the human body (including face and body) and hands to determine the key points of the human body and hands. Then, it calls the recognition algorithm to identify the key points of the human body and hands to determine the user's orientation and hand posture. Based on the user's orientation, it can be determined whether the user is facing away from or towards the cabin. Based on the hand posture, it can be determined whether the user is currently performing a grasping action, an outstretched hand action, a sliding action, or other actions. Based on the detection results of the hand key points, the movement trajectory of the hand key point positions can be determined.

[0190] Optionally, based on the user's orientation, hand posture, and the movement trajectory of key hand points, there are two common operations.

[0191] In the first standard operation, if the user is facing away from the cockpit (or facing the initial interactive interface), when the hand posture is the first hand posture, the interactive device can control the cursor to move within the initial interactive interface based on the movement trajectory of the key hand points. When the hand posture changes to the second hand posture, the cursor is controlled to select the first sub-application icon at the current position. For example, referring to Figures 8a and 8b (A), assuming the first hand posture is an open palm and the second hand posture is a grasping motion, when the user is identified as facing away from the cockpit, if the user opens their left hand and moves it to the upper left, the cursor on the currently projected initial interactive interface will also move to the upper left. When the cursor moves to the "Extravehicular Activity" icon, and the user makes a grasping motion, the cursor selects the "Extravehicular Activity" icon and projects the interface of the "Extravehicular Activity" sub-application.

[0192] In one example, as shown in Figure 8b above, the initial interactive interface may also include a "Lock / Power Off" button. Based on conventional operation one, if the user selects the "Lock / Power Off" button, it indicates that the user intends to end the current external interaction and shut down the vehicle. In this case, the interactive device can turn off the external projection lights and control the cabin to lock / power off. For example, it can control the external projection light switch controller to rotate all microlenses to the angle corresponding to "off" and cut off the vehicle's power supply. In this way, the user can directly perform the corresponding action outside the cabin to directly shut down the vehicle, avoiding the user having to re-enter the cabin, thus improving the user's driving experience.

[0193] In one example, as shown in Figure 8b above, the initial interactive interface can also include an "Exit" button. Based on conventional operation one, if the user selects the "Exit" button, it means the user intends to end the current external interaction without shutting down the vehicle. In this case, the interactive device can simply turn off the external projection lights without controlling the cabin lock and power off; in other words, the cabin remains powered on. Combining these two examples, users can flexibly choose how to exit the external interaction to meet the exit needs of different users, demonstrating good versatility.

[0194] Understandably, exiting the external interaction scenario by selecting the relevant button in the initial interactive interface is just one example. Other examples might involve the user getting back into the vehicle and clicking to close the external interaction app on the vehicle's infotainment screen; the user removing the car key from the vehicle to lock and power off the car; or the in-vehicle communication module detecting the disconnection after the user moves a certain distance from the vehicle, automatically locking and powering off the car, which would then automatically close the external interaction app. Alternatively, there might be other exit methods; specific methods are not limited.

[0195] The second standard procedure involves the following: If the user is facing the cockpit (or has their back to the initial interactive interface), the interactive device adjusts the volume of the external speakers when the hand gesture is the third gesture, and adjusts the projection angle of the external projector when the hand gesture is the fourth gesture. For example, referring to Figures 8c, 8b(B), and 8b(C), assuming the third hand gesture is a left-hand up-and-down swipe and the fourth hand gesture is a right-hand up-and-down swipe, when the user is identified as facing the cockpit, if the user slides their left hand up or down, volume buttons are projected onto the initial interactive interface. The volume increases as the user slides their left hand up and decreases as they slide it down. If the user slides their right hand up or down, the initial interactive interface moves accordingly to a higher or lower position, assuming the projection is on a screen. If the projection is on the ground, the initial interactive interface will move towards or away from the cockpit. In other words, swiping up with your right hand adjusts the angle, making the projection on the ground farther or the projection on the wall higher; swiping down with your right hand adjusts the angle, making the projection on the ground closer or the projection on the wall lower.

[0196] It's understandable that users might be facing the cockpit, turning their backs to the cockpit, or even turning to the side. If the cockpit is detected to be turning to the side, no action needs to be taken. However, in some scenarios, there might be other actions, such as indicating that the user is turning to the side to exit the external interaction scene, or indicating that the user is turning to the side to turn on the external music, etc. There are no specific limitations.

[0197] It should be noted that "facing the cockpit" in this application does not only refer to the user's orientation directly facing the cockpit, but includes all orientations in which the user faces the cockpit and the side angle is less than the first angle threshold. Similarly, "facing away from the cockpit" does not only refer to the user's orientation with their back to the cockpit, but includes all orientations in which the user's back is to the cockpit and the side angle is less than the second angle threshold. The first angle threshold and the second angle threshold may be the same or different.

[0198] For example, assuming both the first and second angle thresholds are 30°, and the default angle for a user facing directly into the cockpit is 0°, while the default angle for a user with their back to the cockpit is 180°, then: if the external image determines that the user's current orientation, compared to their orientation facing directly into the cockpit, has a side angle within the range of -30° to +30° (excluding -30° and +30°), then the user is considered to be facing the cockpit; if the side angle is within the range of -150° to +210° (excluding -150° and +210°), then the user is considered to be facing away from the cockpit. In other words, if the user is facing directly into the cockpit and turns within 30° to the left or right, they are considered to be facing the cockpit; if the user is facing away from the cockpit and turns within 30° to the left or right, they are considered to be facing away from the cockpit. The user's side angle in the external image can be directly identified by the sensing base, which is a function of the sensing base itself and will not be discussed in detail here.

[0199] Optionally, in other scenarios, the sensing base can also output the confidence level of whether the user's current orientation in the external image is facing or away from the cockpit, assigning it the orientation with the higher confidence level. Alternatively, the sensing base can directly output the orientation to which the current orientation belongs, and so on. There are many other possible recognition methods, which will not be listed here.

[0200] Furthermore, regarding the user's left and right hands mentioned above, whether facing or away from the cockpit, the user's left and right hands refer to their actual left and right hands, as seen from the user's perspective. If the user is facing away from the cockpit, the sensing base can directly use the hand recognition algorithm to mark hand recognition boxes on the external image; the left hand recognition box indicates the left hand, and the right hand recognition box indicates the right hand. If the user is facing the cockpit, the sensing base needs to first mirror the external image, essentially swapping the left and right sides, and then use the hand recognition algorithm to mark hand recognition boxes on the mirrored image. In this case, the left hand recognition box still indicates the left hand, and the right hand recognition box still indicates the right hand. This approach considers the user's perspective; the left and right hands are simply the user's own, without needing to additionally learn the correspondence with the left and right sides of the cockpit, thus reducing the learning difficulty for the user.

[0201] Alternatively, in other scenarios, the left hand in the cockpit can be considered the left hand and the right hand in the cockpit can be considered the right hand, or other definitions can be given. However, as long as the definitions of the left and right hands remain consistent throughout the entire external interaction process, this application does not impose any specific limitations on this.

[0202] Optionally, referring to Figure 7 above, in the first conventional operation, the projected content on the initial interactive interface can be controlled by the external projection lamp switch controller. In the second conventional operation, the overall movement of the initial interactive interface can be controlled by the external projection lamp angle controller, and the volume of the external speakers can be controlled by the external speaker controller. For example, taking a DLP projection lamp as an example, if the projected content is changed, the interactive device can send a control signal to the DLP digital projection lamp controller to control the DLP digital projection lamp controller to change the switching state of the multiple microlenses contained in the DLP projection lamp, thereby changing the projected content on the initial interactive interface. If the speaker volume is changed, the interactive device can send a control signal to the external speaker controller to control the external speaker controller to change the volume of the external speakers. If the projection position is changed, the interactive device can send a control signal to the DLP angle controller to control the DLP angle controller to change the rotation angle of the multiple microlenses contained in the DLP projection lamp, thereby changing the projection position of the initial interactive interface.

[0203] It should be noted that the two standard operations described above are the basic functions of the "Extravehicular Interaction" app and are applicable throughout its entire use. In other words, whether on the initial interactive interface, a sub-application interface, or a sub-application interface at a lower or higher level, users can control the cursor movement on the current projection screen by facing away from the cockpit and extending their palm, select the cursor on the current projection screen by making a grasping motion with their hand while facing away from the cockpit, and adjust the volume of the external speakers and the projection angle of the external projector by swiping up and down with their left and right hands while facing the cockpit. There are no restrictions on the user's body posture when facing away from or towards the cockpit; for example, it can be a standing or sitting posture with the user facing away.

[0204] Step 403: Based on the user's real-time actions outside the cabin, control the first sub-application interface to make a corresponding projection response.

[0205] Optionally, taking a DLP projector as an example, after the user selects the first sub-application icon, the interactive device can retrieve the locally stored APP information, find the configuration information of the first sub-application, and then send a control command to the DLP digital projection lamp controller. This control command includes the positions of the microlenses that need to be illuminated to display the interface of the first sub-application. The DLP digital projection lamp controller controls the corresponding microlenses to switch to the on state according to the control command, and controls the microlenses in other positions to switch to the off state. The beam of light emitted from the on microlenses passes through the light wheel to form the colored interface of the first sub-application.

[0206] Furthermore, after projecting the first sub-application interface, the interactive device can also control the external camera to collect external images at a certain acquisition frequency. In some scenarios, it can also control the internal camera to collect external images and obtain relevant information sent by the user terminal. Then, it combines this information to identify the user's real-time actions outside the cabin and controls the first sub-application interface to make corresponding projection responses based on the identified real-time actions of the user.

[0207] To facilitate understanding of the specific response methods of each sub-application, the response process of each sub-application will be described below.

[0208] Extravehicular activity sub-application

[0209] Referring to Figure 8b above, assuming that based on conventional operation one, the user selects the "Extravehicular Activity" pattern in the initial interactive interface, the interactive device can control the extravehicular projection lamp controller to drive the extravehicular projection lamp to project the extravehicular activity interface, and the current projection interface switches from the initial interactive interface to the extravehicular activity interface.

[0210] As an example, please refer to Figure 9(A), which shows a schematic diagram of the initial presentation of an extravehicular activity interface provided by this application. The extravehicular activity interface includes a "Start Activity" button, a "Good Moment" button, and an "Exit" button. Based on this extravehicular activity interface, please refer to Figure 10, which shows a schematic diagram of the internal process for realizing extravehicular activity entertainment functions provided by this application. The process includes the following steps one to nine.

[0211] Step 1: The interactive device responds to the user's actions outside the cabin by selecting the "Start Exercise" button on the external exercise interface.

[0212] For example, based on the above conventional operation one, after projecting the external motion interface shown in Figure 9(A), the user can come to the front of the vehicle and face away from the cabin, extend their hand and move their palm. The external camera captures the external image and inputs it into the interactive device. The interactive device calls the sensing base and recognition algorithm to identify that the user in the external image is facing away from the cabin and has their hand outstretched. Therefore, it controls the external projection light controller to change the on / off state of each microlens in the external projection light, so that the cursor in the external motion interface moves with the user's hand. When the cursor moves to the "Start Motion" button, the user makes a grasping motion with their hand. The interactive device detects that the user's hand posture has changed to grasping and controls the external projection light controller to change the on / off state of each microlens in the external projection light, so that the cursor selects the "Start Motion" button, as shown in Figure 9(B).

[0213] Step two: The interactive device controls the projection action of the external projection light, providing guidance to the user.

[0214] For example, as shown in Figure 10, the interactive device can find the action library from the relevant information of the extravehicular activity sub-application, retrieve the action guidance user table from the action library, and then control the extravehicular projection lamp controller to project the action guidance user table onto the extravehicular activity interface, as shown in Figure 9(C). The action guidance user table includes multiple action guidance items, such as the beginner action 1, intermediate action 2, advanced action 3, etc., as shown in the figure.

[0215] Step 3: In response to the user's actions outside the cabin, select the first action guidance item in the action guidance user table and project the video of the first action guidance item onto the external motion interface.

[0216] For example, based on the above conventional operation one, assuming the first action guidance item is intermediate action 2 as shown in Figure 9 (C), the user can extend their hand and move their palm while facing away from the cabin in front of the vehicle. The interactive device recognizes the user's actions in the external image and controls the cursor in the external motion interface to move with the user's hand. When the cursor moves to the position of "intermediate action 2", the user's hand makes a grasping motion, the interactive device controls the cursor to select "intermediate action 2", and projects the video corresponding to intermediate action 2, as shown in Figure 9 (D). This video is used to guide the user to perform the same action outside the cabin.

[0217] Optionally, while projecting the video of the first action guidance item onto the external exercise interface, pre-configured text information can also be projected, such as "Timing," "Speed," "Count," and "Score" as shown in Figure 9(D), as well as "Tips" about the action guidance video. This information can be considered as guidance information for the user and statistical information about the user's exercise, making it easier for the user to understand the exercise status in a timely manner.

[0218] Step four: The user, wearing a user terminal, exercises outside the cabin. The interactive device controls the external camera to capture images at a certain frequency and obtains inertial sensor information (INS information) collected by the user terminal. Then, combining the INS information and the external images, the user's real-time movements are captured. The user terminal can include a mobile phone and a fitness tracker. The mobile phone is placed in the user's pants pocket to collect INS information of the user's leg movements, while the fitness tracker is worn on the user's wrist to collect INS information of the user's hand movements. By combining the INS information from both leg and hand movements, the user's movements identified in the external images can be accurately corrected to obtain accurate real-time user motion data.

[0219] For example, after projecting the video of the first action guidance item shown in Figure 9(D) onto the external motion interface, as shown in Figure 1a above, the user can place their phone in their pants pocket, wear a fitness tracker, and follow the video of the first action guidance item to perform the same actions. The phone and fitness tracker collect inertial navigation information of the user's hand and leg movements, while the external camera collects external images. The inertial navigation information is directly fed into the recognition algorithm in the interactive device, while the external images are fed into the sensing base in the interactive device. The sensing base detects the human body key point information in the motion image and outputs it to the recognition algorithm. The recognition algorithm determines the user's actions based on the human body key point information identified by the sensing base, and then uses the inertial navigation information to correct the user's actions, obtaining real-time human motion fusion information. Based on this human motion fusion information, motion capture is performed to obtain the user's real-time actions.

[0220] Step 5: The interactive device combines the user's personalized virtual avatar with the user's real-time actions to generate a real-time motion video of the virtual avatar (i.e., a real-time avatar), which is then projected onto the external motion interface, such as the small human figure in the lower center of Figure 9 (E).

[0221] The virtual avatars that users personalize are pre-configured in the user profile of the interactive device. For example, the avatars can be configured by the user on the vehicle screen after the vehicle leaves the factory, for use in connecting to the vehicle system or realizing other functions. Alternatively, the avatars can be configured by the user before exercise, such as logging into the smart vehicle system with their face or account before exercise and configuring a unique virtual avatar, which is specifically used for outdoor exercise demonstrations.

[0222] Optionally, as shown in Figure 10, the interactive device stores multiple user profiles, each corresponding to a FaceID. Each user profile contains user information, historical records, and personal habits of the user to which the FaceID belongs. After acquiring an external image, the interactive device, in addition to calling the sensing base to detect human key point information in the external image for motion recognition, can also call the sensing base to detect face bounding boxes and obtain facial key point information from the external image. Then, the facial key point information is output to the recognition algorithm to determine the FaceID of the face based on the facial key point information. Next, the interactive device matches this FaceID with the FaceIDs corresponding to the multiple user profiles stored locally to find the corresponding user profile and retrieve user information, historical records, and personal habits from that user profile. Based on this information, the interactive device determines whether the user has previously stored a virtual avatar. If so, it uses this virtual avatar and the user's real-time movements obtained through motion capture to generate a real-time motion video of the virtual avatar, which is then projected onto the external motion interface. Conversely, if the user profile does not contain a virtual avatar, the intelligent character generation technology in the visual effects algorithm is invoked to generate a brand-new AI virtual avatar, which is then stored in the user profile. At the same time, the newly generated virtual avatar and the user's real-time actions are used to generate a real-time motion video of the virtual avatar, which is then displayed on the external motion interface.

[0223] Step six: The interactive device determines motion information based on the user's real-time actions, determines similarity information between the user's real-time actions and the video of the first action guidance item, and projects the motion information and similarity information onto the external motion interface, as shown in Figure 9(E). The motion information includes one or more of speed, timing, and counting, and the similarity information includes the score.

[0224] For example, taking a scenario that simultaneously includes speed, timing, counting, and scoring, as shown in Figure 10, after the user selects the first action guidance item, the interactive device not only projects the video of the first action guidance item from the action library onto the external motion interface, but also inputs the video of the first action guidance item to the sensing base. The sensing base detects the human key point information in the video of the first action guidance item and outputs it to the recognition algorithm, which identifies the human motion information corresponding to the video of the first action guidance item. The interactive device compares the human motion information with the human motion fusion information obtained from detecting the user's real-time actions to obtain similarity information, evaluates the human motion fusion information obtained from detecting the user's real-time actions to obtain motion information, and inputs the similarity information and motion information to the information processing module of the external motion sub-application. The information processing module scores the user's real-time motion based on the similarity information, analyzes the user's real-time motion speed, timing, and counting based on the motion information, and uses this information to update the external motion interface. Furthermore, as the user moves in real time, this information in the external motion interface will continuously change.

[0225] Optionally, the interactive device can also determine the prompt message based on the comparison between the user's real-time actions and the first action guidance video, and control the external projection light to project the prompt message, as shown in Figure 9(E) "Pay attention to speed".

[0226] Step 7: When the similarity information indicates that the similarity between the user's real-time actions and the video of the first action guidance item is lower than the set similarity value, the interactive device can also control the external speakers to play action guidance voice.

[0227] For example, as shown in Figure 10, when the score is below a set value, such as 50 points, the interactive device can also find a corpus from relevant information in the extravehicular activity sub-application, and based on the features provided by the similarity algorithm, find matching voice information from the corpus. This voice information is then sent to the extravehicular speaker, which is instructed to play the voice message to provide the user with voice prompts about which actions need improvement. This voice information, combined with the projected action guidance video, enables the user to perform actions that more closely resemble the guided movements.

[0228] Step eight, as shown in Figure 9 (D) or (E), when projecting the video of the first action item, a "End Movement" button can also be projected on the extravehicular motion interface. If the user wants to end the movement, they can select the "End Movement" button based on the above conventional operation one, so that the current projection interface is restored to the initial extravehicular motion interface, as shown in Figure 9 (F).

[0229] Optionally, in response to the user selecting the "End Exercise" button in the external image, the interactive device can also generate a historical exercise record and store it locally. For example, as shown in Figure 10, the interactive device can combine external images captured by the external camera with real-time motion videos of the virtual avatar to generate a historical exercise record. Therefore, the historical exercise record includes the video of the user's real-time movement with reference to the external exercise interface during this exercise, as well as the real-time motion video of the virtual avatar customized by the user.

[0230] Optionally, after generating historical motion records, the interactive device can also use a video generation algorithm to generate video recordings of memorable moments (video blogs, Vlogs), or simply "moments of joy," and store these moments in the configuration information of the extravehicular activity (EVA) sub-application. Correspondingly, as shown in Figure 9(F), the initial EVA interface can also project a "Moments of Joy" button. If the user wants to view historical motion records, they can select the "Moments of Joy" button based on the above-mentioned conventional operation. Responding to this user action in the extravehicular image, the interactive device can find the Vlog related to the "Moments of Joy" from the relevant information in the EVA sub-application and control the extravehicular projection light to project the Vlog. This Vlog contains a video of the user's actual motion and a video of a virtual avatar's motion. The user's actual motion video is a video of their back, while the virtual avatar's motion video is a video of their front view. Displaying both videos simultaneously enhances the user's viewing experience.

[0231] Step nine, as shown in Figure 9(F), can also project an "Exit" button onto the initial external motion interface. If the user wants to end the external motion sub-application, they can select the "Exit" button based on the above conventional operation one. In response to this user action in the external image, the interactive device can exit the external motion interface and project the initial interactive interface. The currently projected interface restores from the external motion interface to the initial interactive interface, as shown in Figure 6. The user can select other sub-applications on the initial interactive interface to continue external interaction, or select the "Exit" button to exit the external interaction scene, or select the "Lock and Power Off" button to directly turn off the vehicle.

[0232] Based on the above, an external sports and entertainment function can be realized. Various hardware modules in the interactive system acquire user movement information, while the software module uses this information to perceive and recognize user actions. It then combines user profiles and visual effects algorithms to determine the interaction plan, which is implemented on various hardware devices outside the cabin via the vehicle control unit. This is then displayed in real-time on the external sports interface, creating a closed loop for user interaction. In this way, users can receive corresponding projection responses when they move outside the cabin, satisfying their sports and entertainment experience.

[0233] Personalized welcome sub-application

[0234] Referring to Figure 8b above, assuming that based on conventional operation one, the user selects the "personalized welcome" pattern in the initial interactive interface, the interactive device can control the external projection light controller to drive the external projection light to project the personalized welcome interface, and the current projection interface switches from the initial interactive interface to the personalized welcome interface.

[0235] As an example, please refer to Figure 11(A), which shows a schematic diagram of the initial presentation of a personalized welcome interface provided in this application. This personalized welcome interface includes an "Open Welcome" button and a "Close Welcome" button. Based on this personalized welcome interface and the above-mentioned conventional operation one, the user can stand in front of the vehicle with their back to the cabin and make corresponding gestures. The external camera captures the external image and sends it to the interactive device. The interactive device responds to the user's actions in the external image:

[0236] In one scenario, if the "Enable Welcome" button is selected in the personalized welcome interface, as shown in Figure 11(B), the interactive device can activate the personalized welcome function. Simultaneously, it can control the external projection light to project multiple selectable personalized welcome settings, such as Face ID, welcome message, welcome light color, and whether to enable vacant seat reminders, as shown in Figure 11(C). When the user faces away from the cabin and makes a corresponding gesture, the interactive device responds to the gesture, selects the corresponding personalized welcome setting, and obtains the user's input information. Then, it stores the association between this information and the personalized welcome setting in the user profile.

[0237] In another scenario, if the "Close Welcome" button is selected in the personalized welcome interface, the interactive device can close the personalized welcome function. At the same time, it can also exit the personalized welcome interface and project the initial interactive interface, as shown in Figure 6.

[0238] It should be noted that enabling or disabling the personalized welcome function via the external projection interface is only one optional implementation method. This method is designed for scenarios where users want to modify the personalized welcome after exiting the vehicle, or to accommodate users who are accustomed to setting the personalized welcome function externally. However, it should be understood that the personalized welcome function can also be enabled or disabled in other ways, such as allowing users to select whether to enable it on the vehicle's infotainment screen, set it on their user terminal, or enable it via voice command within the cabin, etc. Furthermore, personalized welcome settings can also be configured in other ways, such as allowing users to select or input settings on the vehicle's infotainment screen or user terminal, or allowing users to set them via voice command within the cabin, etc., without limitation.

[0239] Optionally, assuming the personalized welcome function is enabled in the cockpit, please refer to Figure 12a, which shows a flowchart illustrating one method for implementing the personalized welcome function provided in this application. It should be noted that the specific implementation of this personalized welcome function may not depend on the aforementioned initial interactive interface or personalized welcome interface, but can be implemented as a separate method. In other words, the user can select to enable the personalized welcome function and set relevant configuration information on the user terminal, vehicle system, or other devices in the cockpit, and the cockpit will automatically enable personalized welcome based on these settings, without requiring the user to configure relevant information through the initial interactive interface or personalized welcome interface.

[0240] As shown in Figure 12a, the method may specifically include the following steps 1201 to 1203.

[0241] Step 1201: Acquire external images and identify faces contained in the external images.

[0242] Please refer to Figure 12b, which shows the internal implementation logic diagram of this personalized welcome function. Combining Figures 12a and 12b, after the personalized welcome function is activated in the cabin, the interactive device can first control the external camera to collect external images, and then input the external images into the sensing base to detect human key point information in the external images. After that, the human key point information is input into the recognition algorithm to identify facial information, i.e., FaceID.

[0243] Optionally, the interactive device can begin collecting external images only when it senses the user's proximity to the vehicle. The method of sensing the user's proximity depends on the specific welcome scenario. For example, in a driver-centric welcome scenario, the driver, carrying the car key, gradually approaches the vehicle when they need it. When the distance between the key and the vehicle becomes close enough, the key automatically establishes a connection with the vehicle's communication module, which then powers on the entire vehicle. Therefore, in this scenario, the interactive device can determine the user's proximity each time it switches from a powered-off state to a powered-on state and control the external camera to collect external images. Conversely, in a non-driver-centric welcome scenario, the non-driver does not carry the car key. This is typically a temporary parking scenario, such as the driver temporarily parking to pick someone up or stopping to buy something. In this scenario, the user can shift the vehicle to neutral when leaving, meaning the vehicle remains powered on, and the external camera continues to collect external images and send them to the interactive device, which then identifies the Face ID in the external images.

[0244] Step 1202: If the face matches the first face stored locally in the cockpit, then obtain the personalized welcome settings of the first face.

[0245] Here, the interactive device can match the faces identified in the external images with multiple faces stored locally. If a first face is matched, the personalized welcome settings for that first face are obtained.

[0246] For example, as shown in Figure 12b, the interactive device can obtain multiple user profiles stored locally and compare the identified FaceID with multiple FaceIDs corresponding to the multiple user profiles. If the similarity between the first FaceID and the identified FaceID is greater than a set threshold, the user profile of the first FaceID is obtained, and personalized welcome settings based on the user's historical habits are obtained from the user profile, such as welcome message, welcome light color, and whether to enable empty seat reminder.

[0247] Optionally, if the face in the external image does not match any of the locally stored FaceIDs, it means that the user currently standing in front of the vehicle is not a user in the vehicle and may be a passerby. In this case, there is no need to greet the user. Therefore, the interactive device can leave it alone and control the external camera to continue collecting external images until it recognizes the FaceID of the user in the vehicle in the external image.

[0248] Step 1203: Generate a welcome guide pattern based on the personalized welcome settings of the first face and project it outside the cabin.

[0249] Optionally, if the personalized welcome settings for the first face do not enable the vacant seat reminder, the interactive device can directly generate a welcome guidance pattern based on other information in the personalized welcome settings for the first face and project it outside the cabin. For example, control information can be sent to the external welcome light controller, causing the external welcome light controller to control the external welcome lights to project the welcome message in the personalized welcome settings onto the ground around the vehicle body according to the welcome light color in the personalized welcome settings, as shown in Figure 12c(A).

[0250] Conversely, if the personalized welcome setting for the first face indicates that an empty seat reminder is enabled, the interactive device can control the in-cabin camera to capture images of the cabin and identify the location of the empty seat within the cabin. For example, the in-cabin image is input into the sensing base to detect the seat detection frame in the image, and then the recognition algorithm is called to perform in-situ detection on the image of the seat detection frame to determine the location of the empty seat. Afterwards, the interactive device can determine the welcome fusion information based on the location of the empty seat and the personalized welcome setting, and generate a welcome guidance pattern, which is then projected outside the cabin. For example, as shown in Figure 12c (B), if the user profile's account ID or nickname is "Xiaoshuai," and the user's historical habits indicate that they usually sit in the left rear seat, and the left rear seat is currently empty, a series of arrows pointing to the left rear seat can be projected on the ground around the vehicle, along with the text "Welcome Xiaoshuai, come to your exclusive ride."

[0251] Understandably, whether or not to enable the empty seat indicator can also be set in the cabin. For example, if the empty seat indicator is enabled, then any user entering the cabin can be shown an empty seat. Alternatively, it can be set in the personalized welcome function. If the personalized welcome function is enabled, and the empty seat indicator is also enabled, then users entering the cabin will be shown an empty seat for the entire period when the function is enabled.

[0252] Furthermore, the above examples all use the identification of faces in external images as an example. However, in other scenarios, faces may not be identified in the external images. In such cases, whether or not to activate the welcome function depends on the specific welcome scenario. For example:

[0253] In the case of a passenger-side welcoming scenario, the trigger condition for capturing the external image is that the vehicle is in neutral and the user temporarily gets out of the car. In this case, if no face is captured, it means that the user has not come to the front of the car and there is no need for welcoming, so as to avoid welcoming errors.

[0254] Conversely, in a driver's seat welcome scenario, although no face is recognized in the external image, the trigger condition for capturing the external image is the establishment of a connection between the user terminal and the vehicle's communication module. This means that a user has indeed arrived near the vehicle, and a welcome experience needs to be provided. Furthermore, if the user terminal is a device like a car key, which doesn't clearly identify the user ID, then since it's impossible to determine which user in the vehicle is carrying the key, the interactive device can use a default welcome method. For example, it can generate a welcome guide pattern based on the location of an empty seat and the default personalized welcome settings, and project it outside the vehicle. For example, as shown in Figure 12c (C), the text "Please get in the car" and a series of arrows pointing to empty seats inside the vehicle are projected onto the ground around the vehicle. Conversely, if the user terminal is a device with a clear user ID, such as a user's mobile phone or computer, the vehicle can obtain the user ID. The interactive device can quickly locate the user's Face ID based on this user ID, and then provide the aforementioned personalized welcome experience based on that Face ID. This method can be adapted to scenarios where users get into the car in the blind spot of the external camera, such as when users are not used to coming to the front of the car, but come to the side or rear of the car to get in. Even if the external camera does not capture the image, it can still provide users with a welcoming experience.

[0255] Based on the above welcoming methods, the vehicle usage needs of users with different habits can be more comprehensively identified, thereby providing a welcoming experience for users with different habits and meeting users' personalized boarding needs.

[0256] Optionally, during the execution of the personalized welcome function, in addition to providing an external welcome experience before the user boards the cabin, an internal atmosphere experience can also be provided during or after the user boards. Specifically, the interactive device can also identify cabin usage scenarios and obtain the corresponding atmosphere configuration information. When the door is detected to be open, the external projection light is turned off, and the interior ambient lights are controlled to project an ambient image according to the atmosphere configuration information, as well as the interior audio system is controlled to play ambient music. There are many ways for the interactive device to identify cabin usage scenarios. For example, it can interact with other modules within the cabin to obtain the cabin usage scenarios perceived by those modules, or it can automatically identify the current cabin usage scenario based on external images captured by the external camera, and so on.

[0257] For example, taking the latter as an example, referring to Figure 12b, in addition to controlling the external camera to capture external images before the user gets into the vehicle, the interactive device also needs to control the external camera to continuously capture external images throughout the journey and input the external images into the recognition algorithm to identify the current cabin usage scenario. The interactive device's local storage has preset correspondences between various scenarios and atmosphere information. By querying this correspondence, the atmosphere information corresponding to the current cabin usage scenario can be obtained. Of course, if the current cabin usage scenario cannot be found in the correspondence, it can also be input into the scenario generation algorithm (a type of visual effects algorithm) to generate new atmosphere information, and the cabin usage scenario and the new atmosphere information are stored in the correspondence. Afterwards, the interactive device can determine the atmosphere fusion information based on the acquired or generated atmosphere information and the user profile, and generate the corresponding theme atmosphere. Based on this theme atmosphere, after detecting that the user has opened the car door, it can send control signals to the cabin ambient light controller and cabin audio system to control the external ambient light to project the corresponding atmosphere pattern, and at the same time control the cabin audio system to play the corresponding ambient music to create the scene atmosphere.

[0258] For example, if the external image indicates the vehicle is on a beautiful country road, the identified usage scenario is a travel scenario. In this scenario, by controlling the ambient lighting and audio system, a light green pattern can be projected inside the vehicle when the user gets in, and soothing music can be played to create a comfortable camping atmosphere. Or, if the external image indicates the vehicle is surrounded by tall buildings, the identified usage scenario is a commuting scenario. In this scenario, by controlling the ambient lighting and audio system, a colorful and vibrant pattern can be projected inside the vehicle when the user gets in, and lively music can be played to motivate the user to get to work. Still, if the external image indicates the vehicle is next to a primary school, the identified usage scenario is picking up children from school. In this scenario, by controlling the ambient lighting and audio system, a sky-blue pattern can be projected inside the vehicle when the user gets in, and children's songs can be played to create a cheerful and playful family atmosphere. As can be seen, based on this solution, specific themed ambient lighting and sound effects can be generated before the user boards the vehicle, and ultimately presented to the user from the moment they get in through ambient lighting and sound, providing the user with a full sense of atmosphere and improving the user experience.

[0259] Based on the above, a personalized welcome function can be implemented. Various hardware modules in the interactive system acquire user information from both outside and inside the vehicle, while the software modules utilize external user information to perceive and recognize user actions, and internal information to detect available seats. Combining user profiles and visual effects algorithms, a welcome plan is determined and implemented on various hardware components outside the cabin via the vehicle control unit. In this way, users receive appropriate boarding prompts before boarding and experience a corresponding atmosphere during boarding, thus optimizing the user's boarding experience.

[0260] Hand Shadow Interactive Sub-application

[0261] Referring to Figure 8b above, assuming that based on conventional operation one, the user selects the "hand shadow interaction" pattern in the initial interactive interface, the interactive device can control the external projection light controller to drive the external projection light to project the hand shadow interaction interface, and the current projection interface switches from the initial interactive interface to the hand shadow interaction interface.

[0262] As an example, the hand shadow interaction interface can be a blank interface. Essentially, after entering the hand shadow interaction sub-application, the external projector lamp projects a blank area onto the outside of the vehicle. For instance, as shown in Figure 1b above, when camping, a user can manually set up a screen or tent in front of the vehicle and select the hand shadow interaction pattern on the initial interaction interface, causing the external projector lamp to project a relatively bright blank area onto the screen or tent. Then, the user sits in front of the vehicle near the screen and makes hand shadow movements, as shown in Figure 13a(A). In this way, the shadow cast by the external projector lamp on the user's hand is clearly and without ghosting onto the screen or tent.

[0263] To further enhance the fun of hand shadow interaction, please refer to Figure 13b, which shows a schematic diagram of the internal process for implementing hand shadow interaction function provided in this application. The process may specifically include the following steps one to three.

[0264] Step 1: The interactive device controls the external camera to capture images outside the cabin, recognizes the user's gestures and shadows in the images, determines the hand shadow information, and determines the first hand shadow category to which the hand shadow information belongs.

[0265] For example, in response to a user's click on a hand shadow interactive pattern, the interactive device can not only control the external projection light to project a blank interface, but also send control signals to the external camera to control the camera to capture external images at a certain acquisition frequency. The interactive device acquires the external images and inputs them into a sensing base. The sensing base detects key points and targets in the external images to identify gesture key points and target key points, and inputs these key points into a recognition algorithm to identify gesture information and shadow information, referred to as hand shadow information. The interactive device also has a classification recognition model (a type of recognition algorithm). The interactive device can input the gesture information and shadow information together into the classification recognition model and obtain the hand shadow category to which the gesture information and shadow information belong, output by the classification recognition model. For example, it outputs the animal category represented by the user's current hand shadow action, such as rabbit, dog, duck, peacock, etc.

[0266] Step two: The interactive device determines the first scene element for the first hand shadow classification and matching.

[0267] In one example, the interactive device's local storage contains multiple mappings between hand shadow categories and scene elements. These mappings can be pre-generated using a preset scene generation algorithm. After recognizing a first hand shadow category, the interactive device first queries the locally stored mappings. If a scene element corresponding to the first hand shadow category can be found, that scene element is used as the first scene element. If no mapping is found, as shown in Figure 13b, the first hand shadow category is then input into the preset scene generation algorithm to call the algorithm to customize a first scene element that matches the first hand shadow category, and the mapping between the first scene element and the first hand shadow category is stored locally.

[0268] The first scene element includes graphic elements and / or text elements, and in some scenes, sound effects elements are also included. For example, when the first hand shadow category is the animal category referred to by the user's hand shadow, the first scene element can be considered as a customized scene that matches the currently referred to animal. The graphic elements may include, for example, patterns of the animal's habitat and patterns of other animals that may exist in the habitat; the text elements may be introductory text describing the animal's appearance and habits; and the sound effects elements may be, for example, the animal's call.

[0269] Step 3: The interactive device controls the external projection light to project the pattern elements and / or text elements from the first scene elements onto the hand shadow interactive interface. Optionally, if the first scene elements also include sound effects, the external speakers are controlled to play the sound effects.

[0270] For example, referring to Figure 13b, the interactive device can send control signals to the external projection lamp controller based on the pattern elements and / or text elements in the first scene elements, controlling the external projection lamp controller to change the on / off state of each microlens in the external projection lamp, so that the external projection lamp projects pattern elements and / or text elements on the hand shadow interactive interface. Also, based on the sound effect elements in the first scene elements, a control signal can be sent to the external speaker controller to control the external speaker controller to turn on the external speaker, so that the external speaker plays sound effect elements. For example, in the example of Figure 13a, the animal category represented by the user's hand shadow is a rabbit. The scene elements corresponding to the rabbit include textual descriptions of the rabbit, a picture of its habitat (grassland), and pictures of other animals on the grassland, such as kangaroos. This information is projected onto the hand shadow interactive interface, as shown in Figure 13a(B). Optionally, the sound of a rabbit can also be played.

[0271] Based on the above, an external hand shadow interactive entertainment function can be realized. Various hardware modules in the interactive system acquire the user's hand shadow information, while the software module uses this information to perceive and recognize the user's hand shadows. It then combines customized scenes and visual effects algorithms to determine the interaction plan, which is implemented on various hardware devices outside the cabin via the vehicle control unit. This is then displayed on the external hand shadow interactive interface, completing the closed loop of the user's external hand shadow interaction. In this way, users can receive corresponding projections and sound effects responses when making hand shadows outside the cabin, enhancing the fun of the hand shadow interaction.

[0272] Drawing board interactive sub-application

[0273] Referring to Figure 8b above, assuming that based on conventional operation one, the user selects the "Paintboard Interaction" pattern in the initial interactive interface, the interactive device can control the external projection light controller to drive the external projection light to project the Paintboard Interaction interface, and the current projection interface switches from the initial interactive interface to the Paintboard Interaction interface.

[0274] As an example, please refer to Figure 14a(A), which shows a schematic diagram of the initial presentation of a drawing board interactive interface provided in this application. This drawing board interactive interface includes a drawing area and various drawing tools, including but not limited to: a brush tool, an eraser tool, and a cursor selection tool. In addition, it also includes a "Stop Drawing" button. Based on this drawing board interactive interface, please refer to Figure 14b, which shows a schematic diagram of the internal process for implementing the drawing board interactive function provided in this application, specifically including the following steps one and two.

[0275] Step 1: The interactive device identifies the user's head and hand postures in the external image. When the user's head posture is facing away from the cockpit, the corresponding drawing operation is projected onto the interactive drawing interface based on the hand posture.

[0276] For example, based on the above conventional operation one, after projecting the interactive drawing interface shown in Figure 14a (A), as shown in Figure 1c above, the user can sit in front of the car with their back to the cabin and extend their hand to draw in the air. The external camera captures the external image and inputs it into the interactive device. The interactive device calls the sensing base to detect the human body key points and hand key points in the external image, and calls the recognition algorithm to identify the human body key points to determine the user's orientation, and identify the hand key points to determine the user's hand posture. When the user's orientation is away from the cabin, the user's gesture is captured according to the hand posture to determine the corresponding drawing operation. In response to the drawing operation, a real-time interface is generated, and the external projection light is controlled to display the drawing on the interactive drawing interface in real time, realizing online drawing, as shown in Figure 14a (B).

[0277] The correspondence between gestures and drawing operations includes, but is not limited to, the following five: a grasping gesture corresponds to a selection operation; a waving gesture corresponds to a deselection or return operation; an open hand gesture corresponds to mouse control; an extended index finger gesture corresponds to a brush operation; and a fist-waving gesture corresponds to an eraser operation. Based on these five correspondences, when drawing in mid-air, in response to the user's grasping gesture facing away from the cockpit, the interactive device can control the cursor on the drawing board interface to select the brush mode. In response to the user's extended index finger gesture facing away from the cockpit, the interactive device captures the hand's drawing trajectory and controls the brush on the drawing board interface to draw the corresponding pattern in real time. In response to the user's fist-waving gesture facing away from the cockpit, the interactive device can control the corresponding pattern on the drawing board interface to be erased in real time. In response to the user's waving gesture facing away from the cockpit, the interactive device can control the cursor on the drawing board interface to return to the previous interface or deselect. In response to the user's open hand gesture facing away from the cockpit, the interactive device can control the cursor on the drawing board interface to move with the user's hand.

[0278] In addition, all user actions on the canvas are recorded and saved locally. Correspondingly, as shown in Figure 14a(A), the initial canvas interaction interface also includes a "historical artwork" button. Based on the above conventional operation one, if the user selects the "historical artwork" button, in response to this action, the interactive device controls the external projection light through the external projection light controller to directly display historical artworks on the canvas interaction interface, such as the artwork saved locally after the last drawing.

[0279] Step two: The interactive device responds to the user's actions in the external image. If the "End Drawing" button is selected, the user's drawing is optimized and the optimized drawing is projected.

[0280] For example, assuming the user finishes drawing, the canvas interface is as shown in Figure 14a(C). The user can then extend their hand with their back to the cockpit, causing the cursor in the canvas interaction interface to move accordingly. When the cursor moves to the "End Drawing" button, the user makes a grabbing motion, and the interactive device controls the cursor in the canvas interaction interface to select the "End Drawing" button. Furthermore, the interactive device can also optimize the completed artwork, such as using image style transfer technology to perform AI re-creation of the completed image, including but not limited to: correcting defects in the drawn image, adding filters to the image, and changing the drawing style according to the scene to which the image belongs. For example, Figure 14a(D) is the artwork after AI re-creation of the original drawn image shown in Figure 14a(C). This re-creation not only automatically connects the broken lines on the dinosaur in the original drawn image but also adds corresponding background patterns, such as trees and stars. In some scenes, appropriate colors can also be added to make the artwork more perfect. After the re-creation is completed, the interactive installation can also control the external projection light through the external projection light controller to display the final work in real time on the interactive interface of the canvas outside the cabin, so as to present it to the user.

[0281] Based on the above, an interactive entertainment function for the external drawing board can be implemented. Users draw while suspended outside the cabin. Various hardware modules in the interactive system acquire the user's gesture information, while the software module uses this information to perceive and recognize the gestures, determining the corresponding drawing action. The vehicle control unit then projects the drawing action onto various hardware devices outside the cabin, displaying the user's actions in real-time on the interactive interface of the external drawing board, thus completing the closed loop of user interaction. This solution essentially moves the drawing board from the in-vehicle infotainment system to an external projection light, enhancing the user's external entertainment experience.

[0282] The above content describes the specific implementation of the interaction method in four out-of-cabin interaction scenarios. This interaction method relies on a comprehensive human-computer interaction software algorithm architecture, which includes perception algorithms (for gesture recognition, action recognition, scene recognition, face recognition, presence detection, gesture classification, etc.), visual effects algorithms (for scene generation, virtual avatar generation, motion capture, and other visual effects-related algorithms), and user profiling, among other multi-dimensional elements. Furthermore, AI algorithms can be used to perceive and generate interaction schemes, thereby enabling functions such as vehicle-mounted system operation, settings adjustment, and application experience outside the cabin, providing users with a brand-new entertainment and interaction scenario and improving their intelligent out-of-cabin interaction experience.

[0283] It is understood that the above content only provides examples of the implementation process for four extravehicular interaction scenarios, but these implementation processes are also applicable to other types of extravehicular interaction scenarios. For example, it can also be applied to scenarios where users play games on an extravehicular projection game interface, or to scenarios where users adjust movie volume and playback speed through gestures on an extravehicular projection movie interface, without limitation.

[0284] Furthermore, this interactive method can be extended to any device or system that requires human-computer interaction. For example, it can be applied to any mobile device with a cockpit, including but not limited to ships, airplanes, high-speed trains, regular trains, helicopters, lawnmowers, etc. Alternatively, it can be applied to smart home scenarios, such as changing the TV screen based on the user's real-time actions during playback to enhance the viewing experience. And so on.

[0285] Furthermore, as system architecture evolves and new scenarios emerge, the interactive methods provided in this application are also applicable to similar technical problems, and this application does not impose any specific limitations on them.

[0286] Based on the interactive method described above, this application can also provide an interactive device that can be used to execute the above interactive method. The relevant features can be found in the above method embodiments, and will not be repeated here.

[0287] In one possible implementation, please refer to Figure 15, which shows a possible structural schematic diagram of the interactive device. The interactive device 1500 may be a cockpit or a module within a cockpit (such as a processor, chip, or chip system), or it may be an apparatus applied to or used in conjunction with a cockpit or its modules to implement the interactive methods executed by the cockpit or its modules. The interactive device 1500 may include various units or modules for implementing the interactive methods in the embodiments shown in Figures 4, 7, 10, 13b, and 14b above.

[0288] As shown in Figure 15, the interactive device 1500 may include a projection unit 1510, a recognition unit 1520, and a control unit 1530. The projection unit 1510, recognition unit 1520, and control unit 1530 can be used to implement the interactive methods in the embodiments shown in Figures 4, 7, 10, 13b, and 14b. For example, when the interactive device 1500 executes the interactive method shown in Figure 4 above, the projection unit 1510 is used to respond to an interactive command by projecting an initial interactive interface outside the cabin, which includes multiple sub-application patterns; the recognition unit 1520 is used to acquire images outside the cabin and recognize the user's posture in the images; the projection unit 1510 is also used to select a first sub-application pattern according to the user's posture and project the first sub-application interface; the control unit 1530 is used to control the projection unit 1510 to make a corresponding projection response on the first sub-application interface according to the user's real-time actions outside the cabin.

[0289] It should be noted that the aforementioned projection unit 1510, recognition unit 1520, and control unit 1530 can be implemented using virtual modules. For example, the projection unit 1510 can be implemented using a software function unit or a virtual device, the recognition unit 1520 can be implemented using a software function or a virtual device, and the control unit 1530 can be implemented using a software function or a virtual device. Alternatively, the projection unit 1510, recognition unit 1520, and control unit 1530 can also be implemented using physical devices. For example, if the interactive device 1500 is implemented using a chip / chip circuit, the projection unit 1510, recognition unit 1520, and control unit 1530 can be integrated processors, microprocessors, or integrated circuits.

[0290] In another possible implementation, please refer to Figure 16, which shows another possible structural schematic diagram of the interactive device. The interactive device 1600 may be a cockpit or a module within a cockpit (such as a processor, chip, or chip system), or it may be an apparatus applied to or used in conjunction with a cockpit or its modules, capable of implementing the interactive methods performed by the cockpit or its modules. The interactive device 1500 may include various units or modules for implementing the interactive methods in the embodiments shown in Figures 12a or 12b above.

[0291] As shown in Figure 16, the interactive device 1600 may include a recognition unit 1610, a generation unit 1620, and a projection unit 1630. The recognition unit 1610, generation unit 1620, and projection unit 1630 can be used to implement the interactive method in the embodiments shown in Figure 12a or Figure 12b. For example, when the interactive device 1600 executes the interactive method shown in Figure 12a above, the recognition unit 1610 is used to acquire images outside the cabin and recognize faces contained in the images; the generation unit 1620 is used to, if it is determined that a face matches a first face stored locally in the cabin, obtain personalized welcome settings for the first face and generate a welcome guidance pattern based on the personalized welcome settings; the projection unit 1630 is used to project the welcome guidance pattern onto the outside of the cabin.

[0292] It should be noted that the aforementioned identification unit 1610, generation unit 1620, and projection unit 1630 can be implemented using virtual modules. For example, the identification unit 1610 can be implemented using a software function unit or a virtual device, the generation unit 1620 can be implemented using a software function or a virtual device, and the projection unit 1630 can be implemented using a software function or a virtual device. Alternatively, the identification unit 1610, generation unit 1620, and projection unit 1630 can also be implemented using physical devices. For example, if the interactive device 1600 is implemented using a chip / chip circuit, the identification unit 1610, generation unit 1620, and projection unit 1630 can be integrated processors, microprocessors, or integrated circuits.

[0293] The unit division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional units in each embodiment of this application can be integrated into a single processor, exist as separate physical units, or two or more units can be integrated into a single module. The integrated module can be implemented in hardware or as a software functional module.

[0294] In another possible implementation, please refer to Figure 17, which shows another possible structural schematic of the interactive device. For example, the interactive device 1700 may be a chip or a chip system, used to implement the functions of the interactive device or its modules (such as processors, chips, or chip systems) described in the foregoing embodiments. Optionally, in the embodiments of this application, the chip system may be composed of chips or may include chips and other discrete devices.

[0295] As shown in FIG17, the interactive device 1700 may include at least one processor 1710, which is coupled to a memory. Optionally, the memory may be located within the interactive device 1700, integrated with the processor, or located outside the interactive device 1700. For example, the interactive device 1700 may also include at least one memory 1720. The at least one memory 1720 stores the computer programs (or instructions) and / or data necessary for implementing any of the above embodiments; the at least one processor 1710 may execute the computer programs (or instructions) and / or data stored in the at least one memory 1720 to complete the methods in any of the above embodiments.

[0296] The interactive device 1700 may also include a communication interface 1730, through which the interactive device 1700 can interact with other devices. For example, the communication interface 1730 may be a transceiver, circuit, bus, module, pin, or other type of communication interface. When the interactive device 1700 is a chip-based device or circuit, the communication interface 1730 in the interactive device 1700 may also be an input / output circuit, capable of inputting information (or receiving information) and outputting information (or sending information). The processor may be an integrated processor, a microprocessor, an integrated circuit, or a logic circuit, and the processor can determine the output information based on the input information.

[0297] The coupling in this embodiment is an indirect coupling or communication connection between devices, units, or modules, which can be electrical, mechanical, or other forms, used for information exchange between devices, units, or modules. The processor 1710 may operate in conjunction with the memory 1720 and the communication interface 1730. This embodiment does not limit the specific connection medium between the processor 1710, the memory 1720, and the communication interface 1730.

[0298] Optionally, referring to Figure 17, the processor 1710, the memory 1720, and the communication interface 1730 are interconnected via a bus. The bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used in Figure 17, but this does not indicate that there is only one bus or one type of bus.

[0299] In the embodiments of this application, the processor 1710 may be a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field-programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or being executed by a combination of hardware and software modules in the processor.

[0300] In this embodiment, the memory 1720 can be a non-volatile memory, such as a hard disk drive (HDD) or a solid-state drive (SSD), or it can be volatile memory, such as random-access memory (RAM). The memory 1720 can be any other medium capable of carrying or storing desired program code in the form of instructions or data structures, and accessible by a computer, but is not limited thereto. The memory 1720 in this embodiment can also be a circuit or any other device capable of implementing storage functions for storing program instructions and / or data.

[0301] Based on the above, this application also provides an interactive system, including the above interactive devices. The interactive device may be the interactive device 1500 in FIG15, or the interactive device 1600 in FIG16, or the interactive device 1700 in FIG17, for executing the interactive method described in the above embodiments.

[0302] Optionally, as shown in Figure 2 above, the interactive system may further include onboard sensors, a vehicle control unit, and human-machine interface elements, which are connected sequentially. The onboard sensors collect sensor information from inside and outside the cabin and send it to the interactive device. The interactive device controls the vehicle control unit to drive the human-machine interface elements based on the sensor information, thereby realizing the above interactive methods.

[0303] Optionally, as shown in Figure 2 above, the vehicle-mounted sensors may specifically include: an external camera, an internal camera, an external microphone, and an internal microphone. Therefore, the sensor information inside and outside the cabin may specifically include: external images, internal images, external voice, and internal voice. Human-machine interface elements may specifically include: an external projection light, an external welcome light, an internal ambient light, an external speaker, and an internal speaker. The vehicle control unit may specifically include: an external projection light controller, an external welcome light controller, an internal ambient light controller, and an audio controller.

[0304] Optionally, as shown in Figure 2 above, the interactive system may further include a user terminal connected to the interactive device. The user terminal is used to collect user-related information, such as inertial sensor information during user movement, and send it to the interactive device. The interactive device is used to combine the user-related information and the sensor information from the vehicle's onboard sensors to determine control signals and send them to the vehicle control unit.

[0305] Optionally, as shown in Figure 2 above, the user terminal may specifically include: a mobile phone and a fitness tracker.

[0306] The above interactive system can be considered a relatively comprehensive and three-dimensional system built on existing capabilities and platforms, and some improvements have been made to the existing content. Specifically, these include: First, adding motion similarity calculation based on human key points to the recognition algorithm to accurately identify user actions and complete real-time interaction between user actions and the interface outside the cabin; Second, developing an "External Interaction" APP in the existing vehicle system, and developing at least four sub-APPs within this APP, namely the aforementioned "External Movement" sub-APP, "Personalized Welcome" sub-APP, "Hand Shadow Interaction" sub-APP, and "Drawing Board Interaction" sub-APP, to adapt to users' interactive needs for various external scenarios; Third, opening up the path from the camera module in the vehicle system chip to the vehicle control unit, enabling the camera module and the vehicle control unit to exchange information to realize the above external interaction solution.

[0307] The above interactive system offers several advantages, including: First, the hardware input units include various vehicle-mounted sensors and user terminals such as external and internal cameras, external microphones, mobile phones, and fitness trackers. This diversifies and enhances the acquisition of input information, resulting in richer information collection. Second, the powerful vehicle chip integrates many existing functions, such as scene generation, scene recognition, and AI re-rendering, allowing the vehicle's interactive capabilities to converge and serve the interactive method provided in this application, achieving diverse external interactive scenarios. Third, the hardware output units include various internal and external components, such as projection lights, welcome lights, ambient lights, and speakers. This diversifies and enhances the interactive output components, particularly the external projection lights, enabling users to experience thoughtful and personalized interactive services even outside the cabin. Based on these advantages, the external interactive solution provided in this application can solve the problems of poor and limited human-computer interaction experiences in existing external cabin systems.

[0308] It should be noted that the functions of each device in the above interactive system, the concepts involved that are related to the technical solutions provided in this application, explanations, detailed descriptions and other steps are described in the foregoing method embodiments, and will not be repeated here.

[0309] Based on the above, this application may also provide a means of transportation that includes the above interactive devices, such as the interactive device 1500 shown in FIG15, the interactive device 1600 shown in FIG16, or the interactive device 1700 shown in FIG17, or includes the above interactive system, such as the interactive system shown in FIG2.

[0310] For example, the means of transportation can be a vehicle, such as a car, truck, motorcycle, bus, recreational vehicle, amusement park vehicle, construction equipment, tram, toy car, golf cart, train, etc., and this application does not impose any particular limitation. In addition, the vehicle can be a new energy vehicle, including electric vehicles, such as two-wheel drive electric vehicles or four-wheel drive electric vehicles, or a fuel-powered vehicle, and this application does not impose any limitation in this regard.

[0311] Based on the above, this application can also provide an electronic device that connects to an external interactive vehicle for communicating with the external interactive vehicle to implement the above-described interactive methods. For example, the electronic device may be a cloud server, terminal equipment, RSU, or other vehicles. The electronic device includes units or modules for implementing the above-described interactive methods, such as the interactive device 1500 shown in Figure 15, the interactive device 1600 shown in Figure 16, or the interactive device 1700 shown in Figure 17.

[0312] Based on the above, this application also provides a computer-readable storage medium storing instructions that, when executed, cause the method provided in any of the above-described method embodiments to be implemented. The computer-readable storage medium may include various media capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory, random access memory, magnetic disk, or optical disk.

[0313] Based on the above, this application also provides a computer program product, which includes: a computer program (also referred to as code or instructions), which, when run on a computer, causes the computer to perform the method provided in any of the above method embodiments.

[0314] In this application, "at least one" means one or more, and "more than one" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. Furthermore, the various numbers involved in the embodiments of this application (such as the numerical numbers "first," "second," "third," etc.) are only for descriptive convenience and are not intended to limit the scope of the embodiments of this application. The order of the sequence numbers of the above processes does not imply the order of execution; the execution order of each process should be determined by its function and internal logic.

[0315] In the various embodiments of this application, unless otherwise specified or in case of logical conflict, the terminology and / or descriptions of different embodiments are consistent and can be referenced by each other. The technical features of different embodiments can be combined to form new embodiments according to their inherent logical relationship.

[0316] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, compact disc read-only memory (CD-ROM), optical storage, etc.) containing computer-usable program code.

[0317] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more flowchart illustrations and / or one or more block diagrams.

[0318] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0319] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

Claims

1. An interactive method, characterized in that, include: In response to an interactive command, an initial interactive interface is projected outside the cabin, the initial interactive interface including multiple sub-application icons; Acquire images outside the cabin, identify the user's posture in the images outside the cabin, select the first sub-application pattern according to the user's posture, and project the first sub-application interface; Based on the user's real-time actions outside the cabin, the first sub-application interface is controlled to make corresponding projection responses.

2. The method as described in claim 1, characterized in that, The step of identifying the user's posture in the extravehicular image and selecting the first sub-application pattern based on the user's posture includes: Identify the user's orientation, hand posture, and key hand positions in the external images; When the user is facing away from the cockpit, if the hand gesture is the first hand gesture, the cursor is controlled to move in the initial interactive interface according to the movement trajectory of the key hand points, and the cursor is controlled to select the first sub-application pattern at the current position according to the second hand gesture.

3. The method as described in claim 2, characterized in that, The method further includes: When the user is facing the cockpit, if the hand gesture is the third hand gesture, the volume of the external speakers is adjusted; if the hand gesture is the fourth hand gesture, the projection angle of the external projector is adjusted.

4. The method as described in claim 3, characterized in that, The first hand gesture is an open palm, the second hand gesture is a grasping gesture, the third hand gesture is a left hand sliding up and down, and the fourth hand gesture is a right hand sliding up and down.

5. The method according to any one of claims 1 to 4, characterized in that, The method further includes: The ambient brightness outside the cabin is determined based on the external image. If the ambient brightness is lower than the set brightness value, the external supplementary light is turned on. The brightness of the external supplementary light is lower than that of the external projection light.

6. The method according to any one of claims 1 to 5, characterized in that, The various sub-application patterns include extravehicular activity patterns, personalized welcome patterns, hand shadow interactive patterns, and drawing board interactive patterns.

7. The method according to any one of claims 1 to 6, characterized in that, The first sub-application pattern is an extravehicular motion pattern, and the first sub-application interface is an extravehicular motion interface. The step of controlling the first sub-application interface to make corresponding projection responses based on the user's real-time actions outside the cabin includes: Acquire inertial sensor information collected by a user terminal, which is carried by the user; By combining the inertial sensor information and the external images, the user's real-time actions are captured and projected onto the external motion interface.

8. The method as described in claim 7, characterized in that, The process of capturing the user's real-time actions and projecting them onto the extravehicular motion interface includes: By combining the user's personalized virtual avatar with the user's real-time actions, a real-time motion video of the virtual avatar is generated and projected onto the external motion interface.

9. The method as described in claim 7 or 8, characterized in that, The extravehicular motion interface includes a start motion button; before acquiring the inertial sensor information collected by the user terminal, it also includes: In response to user actions in the external image, if the start movement button is selected, a movement guidance user table is projected in the external movement interface, and the movement guidance user table includes multiple movement guidance items. In response to user actions in the external image, if the first action guidance item in the action guidance user table is selected, the video of the first action guidance item is projected onto the external motion interface. The video of the first action guidance item is used to guide the user to perform the same action outside the cabin.

10. The method as described in claim 9, characterized in that, After capturing the user's real-time actions, the method further includes: Motion information is determined based on the user's real-time actions, and similarity information is determined based on the user's real-time actions and the video of the first action guidance item. The motion information and the similarity information are then projected onto the extravehicular motion interface. The motion information includes one or more of speed, timing, and counting, and the similarity information includes a score.

11. The method as described in claim 10, characterized in that, After determining the similarity information based on the user's real-time actions and the video of the first action guidance item, the method further includes: If the similarity information indicates that the similarity between the user's real-time actions and the video of the first action guidance item is lower than a set similarity, then the external speakers will play action guidance voice.

12. The method according to any one of claims 7 to 11, characterized in that, The extravehicular activity interface includes an end-of-activity button and a "good moment" button; the method further includes: In response to user actions in the external image, if the End Exercise button is selected, a historical exercise record is generated. The historical exercise record includes a video of the user's real-time movement with reference to the external exercise interface during this exercise, as well as a real-time exercise video of the user's personalized virtual avatar. In response to user actions in the external image, if the "Good Moment" button is selected, the historical motion record is projected.

13. The method according to any one of claims 1 to 6, characterized in that, The first sub-application pattern is a personalized welcome pattern, and the first sub-application interface is a personalized welcome interface, which includes an open welcome button and an close welcome button. The step of controlling the first sub-application interface to make corresponding projection responses based on the user's real-time actions outside the cabin includes: In response to user actions in the external image, if the personalized welcome button is selected, multiple selectable personalized welcome settings are projected into the personalized welcome interface, and the personalized welcome settings selected by the user are saved; if the personalized welcome button is selected to be disabled, the personalized welcome interface is exited, and the initial interactive interface is projected.

14. The method according to any one of claims 1 to 13, characterized in that, When the personalized welcome function is enabled in the cockpit, the method further includes: Collect images outside the cabin and identify faces contained in the images outside the cabin. If the face matches a first face stored locally in the cabin, obtain the personalized welcome settings of the first face. The system acquires images of the cabin interior and identifies the locations of empty seats within the cabin based on these images. Then, based on the locations of the empty seats and the personalized welcome settings, it generates a welcome guide pattern and projects it outside the cabin.

15. The method as described in claim 14, characterized in that, The method further includes: If no face is recognized in the external image after the user leaves the vehicle, but the user terminal establishes a connection with the cabin communication module, the welcome guidance pattern is generated based on the empty seat position and the default personalized welcome settings, and projected outside the cabin.

16. The method as described in claim 14 or 15, characterized in that, The method further includes: Identify cockpit usage scenarios and obtain the corresponding atmosphere configuration information for those scenarios; After detecting that the car door is open, turn off the exterior projection light, control the interior ambient light to project an ambient image according to the ambient configuration information, and control the interior audio system to play ambient music.

17. The method according to any one of claims 1 to 6, characterized in that, The first sub-application pattern is a hand shadow interactive pattern, and the first sub-application interface is a hand shadow interactive interface; The step of controlling the first sub-application interface to make corresponding projection responses based on the user's real-time actions outside the cabin includes: Identify user gestures and shadows in the external images to determine hand shadow information; Determine the first hand shadow category to which the hand shadow information belongs, and determine the first scene element that matches the first hand shadow category; The graphic and text elements from the first scene elements are projected onto the hand shadow interactive interface, and the sound effects elements from the first scene elements are played through the external speakers.

18. The method as described in claim 17, characterized in that, The determination of the first scene element matching the first hand shadow classification includes: The system queries the correspondence between multiple hand shadow categories and scene elements stored locally in the cockpit to determine the first scene element corresponding to the first hand shadow category. If the first hand shadow category does not exist in the correspondence, the first hand shadow category is input into a preset scene generation algorithm to obtain the first scene element and store it in the correspondence.

19. The method according to any one of claims 1 to 6, characterized in that, The first sub-application pattern is a drawing board interactive pattern, and the first sub-application interface is a drawing board interactive interface, which includes a "stop drawing" button. The step of controlling the first sub-application interface to make corresponding projection responses based on the user's real-time actions outside the cabin includes: The system identifies the user's orientation and hand gesture in the external image. When the user's orientation is away from the cockpit, the system projects the corresponding drawing operation onto the interactive drawing interface based on the hand gesture. In response to user actions in the external image, if the "End Drawing" button is selected, the user's drawing is optimized and the optimized drawing is projected. The optimization includes one or more of the following: correcting defects, adding filters, and changing styles.

20. The method as described in claim 19, characterized in that, The step of projecting a corresponding drawing operation onto the interactive drawing interface based on the hand gesture includes one or more of the following: If the hand gesture is grasping, then the cursor on the drawing board interactive interface will be controlled to perform a selection operation; If the hand gesture is a wave, then the cursor on the interactive drawing interface will be controlled to perform a deselection or return operation; If the hand gesture is an open hand, then the cursor on the interactive drawing interface will follow the user's hand movement. If the hand gesture is an extended finger, it controls the brush on the interactive drawing interface to draw; or... If the hand gesture is a fist gesture, then the corresponding area of ​​the drawing on the drawing board interactive interface will be erased.

21. The method according to any one of claims 1 to 20, characterized in that, The initial interactive interface also includes a lock / power off button and an exit application button, and the first sub-application interface also includes an exit sub-application button; The method further includes: responding to user actions in the extravehicular image. If the "Exit Application" button in the initial interactive interface is selected, the external projection light will be turned off. If the "Lock and Power Off" button in the initial interactive interface is selected, the exterior projection light will be turned off, and the cabin will be locked and powered off. If the "Exit Sub-application" button in the first sub-application interface is selected, the first sub-application interface will be exited, and the initial interactive interface will be projected.

22. The method according to any one of claims 1 to 21, characterized in that, The interactive command is triggered in any of the following ways: The system receives information that the user clicks on the external interactive application on the vehicle's infotainment screen or user terminal; or... The system detects that the user has been standing in front of the vehicle for a set duration and / or has made a set gesture.

23. The method according to any one of claims 1 to 22, characterized in that, Before the initial interactive interface is projected externally, the following is also included: Confirm that the vehicle is in park and its speed is 0.

24. An interactive device, characterized in that, Includes units and / or modules for performing the method as described in any one of claims 1 to 23.

25. An interactive device, characterized in that, Includes a processor, which is coupled to memory: The processor is configured to execute a computer program or instructions stored in the memory to cause the interactive device to perform the method as described in any one of claims 1 to 23.

26. A means of transportation, characterized in that, Including the interactive device as described in claim 24 or 25.

27. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program or instructions that, when executed, implement the method as described in any one of claims 1 to 23.

28. A computer program product, characterized in that, The computer program product includes computer program code that, when run on a computer, causes the computer to perform the method as described in any one of claims 1 to 23.

Images