Screen projection implementation method and device of vehicle-mounted mobile equipment and storage medium

Abstract

The invention discloses a screen projection implementation method and device for vehicle-mounted mobile equipment and a storage medium, and the method comprises the steps: obtaining an application list installed in the mobile equipment carried by a person on a vehicle, carrying out the rearrangement of visual information corresponding to each application program, and generating a first display image for display; since each piece of visual information is obtained by sorting the real-time driving scene information, the sequence of each piece of visual information in the first display picture can adapt to the use habits of the on-vehicle personnel on the application program in the driving scene represented by the real-time driving scene information, and the user experience of the on-vehicle personnel can be improved under the condition that the mobile equipment is projected to the vehicle machine system. According to the method, the vehicle-mounted personnel can conveniently and quickly find, start, activate and use the wanted application program, so that the use efficiency of the application program is improved, the time and energy paid by the vehicle-mounted personnel to use the application program are reduced, and the traffic safety is guaranteed. The invention is widely applied to the technical field of automobiles.

Description

Technical Field

[0001] The present invention relates to the field of automobile technology, and in particular to a method, device and storage medium for implementing screen projection of a vehicle-mounted mobile device. Background Art

[0002] The functions of car computer systems are becoming more and more powerful. For example, they are equipped with sophisticated display and interactive functions, and users can operate and use various applications on the car computer system. However, due to factors such as insufficient available computing power and insufficient platform support, the functions of car computer systems are usually not as rich as those of mobile devices such as mobile phones. In addition, mobile devices, as devices that are carried with us daily, have accumulated more data that is not available on the car computer system. Therefore, even if the car computer system can install the same applications as mobile devices, users generally tend to use applications installed on mobile devices. At the same time, because the interaction and display performance of the car computer system is usually better than that of mobile devices, users also tend to use the interaction and display functions of the car computer system.

[0003] In order to resolve the above contradictions, screen projection technology can be used. For example, the display screen data generated by the application running on the mobile phone can be sent to the car system, which will be displayed by the car system. In this way, you can use both the application running on the mobile phone and the interactive and display functions of the car system.

[0004] However, current mobile device projection technology has not been optimized for driving scenarios. For example, during driving, users generally have only a short time (for example, drivers usually have only a short time to operate the car system when waiting for a red light or passing through a traffic jam) to find, open and activate an application, which makes it difficult to activate and use the desired application in a timely manner in the case of in-car projection, which brings inconvenience to the use of the application. Summary of the invention

[0005] In view of the technical problems that the current mobile device screen projection technology has not been optimized for driving scenarios, the purpose of the present invention is to provide a method, device and storage medium for implementing screen projection of a vehicle-mounted mobile device.

[0006] On the one hand, an embodiment of the present invention includes a method for implementing screen projection of a vehicle-mounted mobile device, and the method for implementing screen projection of a vehicle-mounted mobile device includes the following steps:

[0007] Obtaining an application list of a mobile device carried by a person in the vehicle; the application list includes visual information of each of a plurality of application programs running on the mobile device;

[0008] Obtain real-time driving scene information;

[0009] Sorting each of the visualization information according to the real-time driving scene information;

[0010] Generate a first display screen according to the sorted visualization information;

[0011] The first display screen is displayed through the vehicle system.

[0012] Furthermore, the obtaining of the application list of the mobile device carried by the person in the vehicle includes:

[0013] Receiving a first projection screen sent by the mobile device; the first projection screen is generated by the mobile device according to a desktop page running on the mobile device;

[0014] Identify the first projection screen and determine the application icons included in the first projection screen;

[0015] The application list is composed by using the application icons as the visual information.

[0016] Furthermore, the sorting of each of the visualization information according to the real-time driving scene information includes:

[0017] Get AI models;

[0018] Inputting the real-time driving scene information into the artificial intelligence model for processing;

[0019] Obtaining a sequence of application names output by the artificial intelligence model;

[0020] The visualization information is sorted according to the order of each visualization information in the application name sequence.

[0021] Furthermore, the artificial intelligence model is trained through a first training process, and the first training process includes:

[0022] Acquire first historical driving scene information and a first actually used application sequence detected by multiple drivers when each of the drivers drives the car; the first actually used application sequence includes multiple application program names actually used by the driver when driving the car and sorted by actual use intensity;

[0023] The first historical driving scene information is used as the input of the artificial intelligence model, and the first actual application sequence is used as the expected output of the artificial intelligence model to adjust the parameters of the artificial intelligence model.

[0024] Furthermore, the method for implementing screen projection of the vehicle-mounted mobile device also includes:

[0025] Acquire a plurality of second historical driving scene information detected by the vehicle occupant when driving the vehicle and a plurality of corresponding second actually used application sequences; the second actually used application sequences include a plurality of application program names actually used by the vehicle occupant when driving the vehicle and sorted by actual use intensity;

[0026] When the cumulative number of all the second actually used application sequences is greater than a first number threshold, clustering each of the second actually used application sequences;

[0027] When the number of clusters obtained by clustering is less than the second number threshold, the artificial intelligence model is trained in a second training process, and the second training process includes:

[0028] The parameters of the artificial intelligence model are adjusted by using each of the second historical driving scene information as the input of the artificial intelligence model and each of the second actual use application sequences as the expected output of the artificial intelligence model.

[0029] Furthermore, the method for implementing screen projection of the vehicle-mounted mobile device also includes:

[0030] determining at least a portion of said visualization information;

[0031] For the determined visualization information, a first operation instruction is generated; the first operation instruction is used to instruct the mobile device to operate the application corresponding to the visualization information to generate a second projection screen;

[0032] Receiving a second projection screen sent by the mobile device, and recording the second projection screen to obtain a recorded screen;

[0033] The screen recording image is cached.

[0034] Further, the determining at least part of the visualization information includes:

[0035] Obtaining the running real-time performance of the application corresponding to each of the visualization information;

[0036] Get the real-time threshold;

[0037] Filtering out the visualization information corresponding to the operation real-time smaller than the real-time threshold as the determined visualization information;

[0038] or

[0039] Obtain information about future driving scenarios;

[0040] Acquire the correlation between the application corresponding to each of the visualization information and the future driving scene information;

[0041] Get the correlation threshold;

[0042] The visualization information corresponding to the correlation degree greater than the correlation degree threshold is screened out as the determined visualization information.

[0043] Furthermore, the method for implementing screen projection of the vehicle-mounted mobile device also includes:

[0044] detecting a second operation instruction directed to the visual information;

[0045] Reading the screen recording image corresponding to the visual information pointed to by the second operation instruction;

[0046] Generate a second display screen according to the read screen recording screen;

[0047] The first display picture is superimposed on or replaced by the second display picture through the vehicle system.

[0048] On the other hand, an embodiment of the present invention also includes a computer device, including a memory and a processor, the memory is used to store at least one program, and the processor is used to load at least one program to execute the screen projection implementation method of the vehicle-mounted mobile device in the embodiment.

[0049] On the other hand, an embodiment of the present invention also includes a computer-readable storage medium, which stores a program executable by a processor. When the program executable by the processor is executed by the processor, it is used to execute the screen projection implementation method of the vehicle-mounted mobile device in the embodiment.

[0050] The beneficial effect of the present invention is as follows: by executing the screen projection implementation method of the vehicle-mounted mobile device in the embodiment, a list of applications installed in the mobile device carried by the vehicle personnel can be obtained, and the visual information corresponding to each application is rearranged to generate a first display screen for display; since each visual information is obtained by sorting according to the real-time driving scene information, the order of each visual information in the first display screen can adapt to the vehicle personnel's usage habits of the application in the driving scene represented by the real-time driving scene information, so that when the mobile device is projected to the vehicle system, it is convenient for the vehicle personnel to quickly find, start, activate and use the desired application, thereby improving the efficiency of application use, reducing the time and energy required by the vehicle personnel to use the application, and ensuring traffic safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 Schematic diagram of an automobile system to which the method for implementing screen projection of an in-vehicle mobile device can be applied in an embodiment;

[0052] Figure 2 A schematic diagram of the steps of a method for implementing screen projection of a vehicle-mounted mobile device in an embodiment;

[0053] Figure 3 This is a schematic diagram of the principles of steps S101-S103 in the embodiment;

[0054] Figure 4 Schematic diagram of the artificial intelligence model;

[0055] Figure 5 This is a schematic diagram of the second projection screen (recording screen). DETAILED DESCRIPTION

[0056] In this embodiment, the method for implementing screen projection of a vehicle-mounted mobile device can be applied to Figure 1 In the vehicle system shown. Figure 1 The automobile system includes components such as processor, memory, voice sensor (microphone), body movement sensor, environmental information sensor, driving status sensor, communication module and interaction module (touch screen), which constitute the vehicle terminal.

[0057] Among them, the processor is a component with control and data processing functions; the memory can be used for the processor to write and read data; the voice sensor (microphone) can be installed in the passenger compartment of the car to collect the spoken voices of the driver, passengers and other people in the car; the body motion sensor can be a camera installed in the passenger compartment of the car, which detects the body movements such as gestures, head or facial expressions of the people in the car by taking images and analyzing the movements of the people in the car, and recognizes the emotional state of the people in the car based on the body movements such as facial expressions, or the body motion sensor can also be a sensor installed on the steering wheel, accelerator pedal, accelerator pedal, gear lever, air conditioning button, window button, ambient light button and other components of the car to detect the steering of the people in the car. The driving state sensor module can detect the real-time position, driving speed, driving acceleration, turning radius and inclination of the vehicle and other driving state information. The driving state sensor module can specifically include a satellite navigation module, a speed sensor, an acceleration sensor, an inclination sensor, etc.; the environmental information sensor can detect the weather, temperature and other information of the environment in which the vehicle is located; the communication module can connect and communicate with mobile devices such as mobile phones through wireless or wired communication protocols such as Bluetooth, WiFi, USB, etc., and the communication module can also connect and communicate with base stations and other devices through wireless communication protocols such as 5G, thereby accessing the server.

[0058] In this embodiment, Figure 1As shown, a car installed with a car terminal that executes the screen projection method for a vehicle-mounted mobile device is called "this car". Each step in the screen projection method for a vehicle-mounted mobile device can be executed by the car terminal. Specifically, each step in the screen projection method for a vehicle-mounted mobile device can be executed by a processor in the car terminal. The processor can call other components of the car when executing the steps.

[0059] like Figure 1 As shown, the person on the vehicle carries a mobile device such as a mobile phone and rides the vehicle. The person on the vehicle may specifically refer to the driver or passenger on the vehicle. In this embodiment, the driver in the driving process is used as an example of the person on the vehicle for explanation.

[0060] Reference Figure 2 , the method for implementing screen projection of a vehicle-mounted mobile device includes the following steps:

[0061] S1. Obtain a list of applications on mobile devices carried by passengers in the vehicle;

[0062] S2. Obtain real-time driving scene information;

[0063] S3. Sort each visualization information according to the real-time driving scene information;

[0064] S4. Generate a first display screen according to the sorted visualization information;

[0065] S5. Display the first display screen through the vehicle computer system.

[0066] In this embodiment, when executing step S1, that is, obtaining the application list of the mobile device carried by the person in the vehicle, the following steps may be specifically performed:

[0067] S101. Receive a first projection screen sent by a mobile device;

[0068] S102. Identify the first projection screen and determine the application icons included in the first projection screen;

[0069] S103. Use the application icons as visual information to form an application list.

[0070] The principle of steps S101-S103 is as follows Figure 3 shown.

[0071] Reference Figure 3 In step S101, the mobile device may run a desktop page, which includes application icons of multiple applications installed on the mobile device. The application icons include application logos, and the application icons generally include information such as the application name of the application. The mobile device takes a screenshot of the desktop page to obtain Figure 3As shown in the first projection screen, the mobile device sends the first projection screen to the communication module on the vehicle side.

[0072] Reference Figure 3 In step S102, after receiving the first projection screen, the communication module sends the first projection screen to the processor for processing. The processor identifies the first projection screen, thereby identifying multiple application icons contained in the first projection screen, such as game application 1, game application 2, browser application, camera application, beauty application, social application, navigation application, search application and artificial intelligence question and answer application. The order of these application icons in the first projection screen is the order in the desktop page, which is generally set by the passengers according to their own usage habits and other factors.

[0073] Reference Figure 3 In step S102, the processor separates the identified multiple application icons from the first projection screen, so that each application icon can be in the form of a single picture, which becomes the visual information in this embodiment. In step S103, these visual information constitute an application list.

[0074] In this embodiment, in addition to being able to use Figure 3 The application icon including only the application name and logo is used as the visualization information of the application program. The mobile device can also obtain images such as thumbnails of the running page of the application program as the visualization information of the application program.

[0075] In step S2, the processor may call Figure 1 The environmental information sensor, driving status sensor, voice sensor (microphone) and body movement sensor in the vehicle detect the current driving scene related information, that is, the real-time driving scene information. For example, the processor can call the environmental information sensor to detect the real-time time, real-time location, weather, temperature and holiday type (working day, ordinary rest day or holiday), call the voice sensor (microphone) and body movement sensor to detect the Face ID and emotional state of the people in the car. The information detected by these sensors constitutes the real-time driving scene information. The real-time driving scene information describes the current driving scene from the perspectives of the driving state of the vehicle, the state of the environment in which the vehicle is located, and the state of the people in the vehicle.

[0076] In this embodiment, when executing step S3, that is, sorting the various visualization information according to the real-time driving scene information, the following steps may be specifically performed:

[0077] S301. Obtain an artificial intelligence model;

[0078] S302. Input the real-time driving scene information into the artificial intelligence model for processing;

[0079] S303. Obtain the application name sequence output by the artificial intelligence model;

[0080] S304. Sort each visualization information according to the order of each visualization information in the application name sequence.

[0081] The principle of steps S301-S304 is as follows Figure 4 shown.

[0082] In step S301, the processor can run the trained artificial intelligence model locally, or request the cloud server to use the trained artificial intelligence model through the communication module. The artificial intelligence model can be constructed based on a convolutional neural network, etc. The trained artificial intelligence model has the performance of identifying the usage habits of the application programs of the vehicle personnel in the corresponding scene according to the real-time driving scene information, and determining the ranking of each application program in order from high to low according to the degree of habitual use.

[0083] Specifically, the artificial intelligence model may be trained in a first training process. The first training process includes the following steps:

[0084] P1. Acquire the first historical driving scene information and the first actual use application sequence detected when multiple drivers are driving their own cars;

[0085] P2. Use each first historical driving scene information as the input of the artificial intelligence model, use each first actual use application sequence as the expected output of the artificial intelligence model, and adjust the parameters of the artificial intelligence model.

[0086] In step P1, data sampling can be performed on multiple drivers such as driver 1, driver 2, driver 3, etc. to obtain the first historical driving scene information detected when they drive the car, as well as the first actually used application sequence in the same time period as the first historical driving scene information is collected (for example, within half an hour before and after the first historical driving scene information is collected). In this way, the first historical driving scene information 1 and the first actually used application sequence 1 are collected for driver 1, the first historical driving scene information 2 and the first actually used application sequence 2 are collected for driver 2, and the first historical driving scene information 3 and the first actually used application sequence 3 are collected for driver 3...

[0087] Taking driver 1 as an example, the first historical driving scene information 1 collected is a vector in the form of (time, location, weather, temperature, holiday type, face ID of the person in the car, emotional state of the person in the car), which indicates the time when driver 1 drove his car, the location of the car, the weather type, temperature, holiday type (weekday, ordinary rest day or holiday), face ID of the person in the car with the driver, and the emotional state of the person in the car; the specific content of the first actual use application sequence 1 is application program 11 ,app 12 ,app 13 ...where applications 11 The application with the highest usage intensity (specifically, the frequency of use or other parameters) of the mobile terminal carried by the driver 1 through the projection function of the vehicle system during the collection time period of the first historical driving scene information 1 (for example, within half an hour before and after the collection time of the first historical driving scene information 1), the application 12 is the second most used application by driver 1 during the same time period. 13 It was the third most used application by driver 1 during the same time period...

[0088] By executing step P1, the first historical driving scene information 1, the first historical driving scene information 2, the first historical driving scene information 3... and the first actual use application sequence 1, the first actual use application sequence 2, the first actual use application sequence 3... etc. can be obtained as a training set for the artificial intelligence model.

[0089] In step P2, multiple rounds of training cycles are executed. For example, in the first round of training cycles, the first historical driving scene information 1 is used as the input of the artificial intelligence model, the actual output of the artificial intelligence model processing the first historical driving scene information 1 is obtained, the error function value is calculated based on the actual output and the first actual use application sequence 1, the first training process is terminated when the error function value converges, and the parameters of the artificial intelligence model are updated using the back propagation algorithm based on the error function value if the error function value does not converge, and then the second round of training cycles is executed; in the second round of training cycles, the first historical driving scene information 2 is used as the input of the artificial intelligence model, the actual output of the artificial intelligence model processing the first historical driving scene information 2 is obtained, the error function value is calculated based on the actual output and the first actual use application sequence 2, the first training process is terminated when the error function value converges, and the parameters of the artificial intelligence model are updated using the back propagation algorithm based on the error function value if the error function value does not converge, and then the third round of training cycles is executed... until the error function value converges, or the number of training cycles executed reaches the upper limit, the training of the artificial intelligence model is completed.

[0090] The driver in steps P1-P2 can be a tester hired under laboratory calibration conditions, or an ordinary user who voluntarily participates in data sampling. After training in steps P1-P2, the artificial intelligence model has the ability to identify the driving scene information of the person, determine the person's usage habits of the application in the corresponding scene, that is, output multiple applications that are used in such driving scenes, and sort them in order from high to low according to the degree of habitual use.

[0091] Reference Figure 4 In step S302, the processor can represent the real-time driving scene information as a vector in the form of (real-time time, real-time location, weather, temperature, holiday type, Face ID of the person in the car, and emotional state of the person in the car), and input this vector into the artificial intelligence model for processing.

[0092] In step S303, the artificial intelligence model performs feature recognition, extraction and processing on the real-time driving scene information, thereby outputting an application name sequence. Figure 4 , the application name sequence output by the artificial intelligence model is specifically navigation application, charging application, artificial intelligence question and answer application, social application, highway rescue application, video live broadcast application, search application, browser application, camera application, beauty application, game application 1, game application 2... The significance of the application name sequence is that in the driving scene represented by the real-time driving scene information, a large number of drivers are accustomed to using the applications according to their usage intensity (such as frequency of use or frequency of use). That is, in the driving scene represented by the real-time driving scene information, most drivers are accustomed to frequently using navigation applications and charging applications, but the degree of habitual use of charging applications is lower than that of navigation applications...

[0093] After executing step S303 to obtain the application name sequence, in step S304, each visualization information can be sorted according to the order of each visualization information in the application name sequence. Specifically, the application name sequence output by the artificial intelligence model is navigation application, charging application, artificial intelligence question and answer application, social application, highway rescue application, video live broadcast application, search application, browser application, camera application, beauty application, game application 1, game application 2..., among which the charging application, highway rescue application, and video live broadcast application are applications that are not installed on the mobile terminals of the people on the vehicle in this embodiment, while other applications are installed on the mobile terminals of the people on the vehicle. After removing the applications that are not installed from the application name sequence, the navigation application, artificial intelligence question and answer application, social application, search application, browser application, camera application, beauty application, game application 1, game application 2... are obtained as the sorting of each visualization information, that is, the obtained application name sequence is obtained. Figure 3The sorted visualization information is shown.

[0094] Reference Figure 3 After executing step S3 to obtain the sorted visualization information, execute step S4 to generate a first display screen. The first display screen includes the visualization information obtained in step S1, and the order of the visualization information has been rearranged, that is, the order of the visualization information in the first display screen is generally different from the order of the visualization information in the first projection screen.

[0095] Figure 3 In the first display screen, the order of each visualization information is represented from left to right and from top to bottom. The order of each visualization information can also be represented from the center to the periphery or from the corresponding area from large to small.

[0096] In step S5, the processor executes the Figure 3 The first display picture shown is sent to the interaction module (touch screen) for display.

[0097] In this embodiment, by executing steps S1-S5, a list of applications installed in the mobile device carried by the occupants of the vehicle can be obtained, and the visual information corresponding to each application is rearranged to generate a first display screen for display; since each visual information is obtained by sorting according to the real-time driving scene information, the order of each visual information in the first display screen can adapt to the usage habits of the occupants of the vehicle for the application in the driving scene represented by the real-time driving scene information, so that when the mobile device is projected onto the vehicle system, it is convenient for the occupants of the vehicle to quickly find, start, activate and use the desired application, thereby improving the efficiency of the use of the application, reducing the time and energy required by the occupants of the vehicle to use the application, and ensuring traffic safety.

[0098] Moreover, in the present embodiment, the application list of the mobile device can be extracted by the processor in the vehicle system from the first projection screen sent by the mobile device. The mobile device can only obtain image data to generate the first projection screen, that is, the first projection screen may not contain the underlying original data related to the mobile device and the application running on it. Therefore, the data that the vehicle system needs to obtain from the mobile device can be reduced, and the information and privacy security on the mobile device side can be guaranteed. It is especially suitable for unfamiliar usage scenarios such as renting a car, borrowing a car or taking a taxi, where the person carrying the mobile device is not the owner of the car or is someone familiar with the owner of the car, thereby improving the adaptability and coverage of the projection function of the vehicle system.

[0099] In this embodiment, on the basis of executing the first training process, i.e., steps P1-P2, on the artificial intelligence model, the following steps may also be executed:

[0100] P3. Acquire multiple second historical driving scene information detected when the vehicle is driven by a person on board and multiple second actual use application sequences;

[0101] P4. When the cumulative number of all second actual use application sequences is greater than the first number threshold, clustering the second actual use application sequences;

[0102] P5. When the number of clusters obtained by clustering is less than the second number threshold, the artificial intelligence model is trained in a second training process, and the second training process includes:

[0103] P6. Use the second historical driving scene information as the input of the artificial intelligence model, use the second actual use application sequence as the expected output of the artificial intelligence model, and adjust the parameters of the artificial intelligence model.

[0104] In step P3, the t of the vehicle during the process of the vehicle being driven by the person in the vehicle can be 1 The second historical driving scene information 1 is obtained by detecting at time t 1 The time period before and after the moment is detected to obtain the second actual use application sequence 1, at t 2 The second historical driving scene information 2 is obtained by detecting at time t 2 The second actually used application sequence 2 is obtained by detecting the time period before and after the moment... The second historical driving scene information 1, the second historical driving scene information 2, etc., have the same data format as the first historical driving scene information, and all represent driving scene parameters measured at a certain moment such as time, location, weather, and temperature; the second actually used application sequence 1, the second actually used application sequence 2, etc., have the same data format as the first actually used application sequence, and all represent multiple application names actually used by people when driving a car within a certain time period and sorted by actual usage intensity, but the first actually used application sequence is obtained by detecting the test driver, while the second actually used application sequence is obtained by detecting the people in the car who want to use the car system projection function in steps S1-S5.

[0105] In step P4, a first quantity threshold N can be set (for example, N can be set to 1000). When the cumulative number of the second actual use application sequence 1, the second actual use application sequence 2, etc. obtained by executing step P3 is greater than the first quantity threshold N, that is, after more than N second actual use application sequences are obtained, each second actual use application sequence is clustered using a clustering algorithm such as kmeans. Multiple second actual use application sequences with similar features will be clustered into one cluster, and second actual use application sequences with greatly different features will be clustered into different clusters.

[0106] In step P5, a second quantity threshold n can be set (for example, n can be set to 3). When the number of clusters obtained by executing step 4 is less than the second quantity threshold n, that is, after less than n clusters are obtained, it can be considered that the rules between the multiple second actual use application sequences obtained by executing step P1 are relatively obvious, thereby triggering the execution of step P6 to perform a second training process on the artificial intelligence model.

[0107] The principle of the second training process performed in step P6 is the same as the first training process performed in steps P1-P2, with the second historical driving scene information 1, the second historical driving scene information 2, etc. as the input of the artificial intelligence model, and the second actual use application sequence 1, the second actual use application sequence 2, etc. as the expected output of the artificial intelligence model, to update the parameters of the artificial intelligence model.

[0108] In this embodiment, the principle of executing steps P3-P6 is that the artificial intelligence model trained only by steps P1-P2 has not been adapted and optimized to the personalized habits of the people on the bus. By executing steps P3-P6, when the behavior of the people on the bus using the projection application is relatively regular, the artificial intelligence model can be trained using the data detected from the people on the bus, so that the artificial intelligence model can adapt to the personalized habits of the people on the bus, and when steps S1-S5 are executed again in the future, a visual information sorting result that is more adapted to the habits of the people on the bus can be obtained.

[0109] In this embodiment, after executing steps S1-S3 and before executing steps S4-S5, the processor can first send the sorted visual information (for example, the first few sorted visual information) to the interactive module (touch screen) for display, and request confirmation from the people on the vehicle; if the people on the vehicle confirm the operation on the interactive module (touch screen), the processor continues to execute steps S4-S5, otherwise, the processor does not execute steps S4-S5, that is, it does not generate and display the first display screen, but can directly display the first projection screen sent by the mobile device, so as to display each visual information in the order set by the people on the vehicle on the mobile device; after a period of time T, step S1 is triggered again, and the above steps are repeated, that is, after executing steps S1-S3 and before executing steps S4-S5, the processor first sends the sorted visual information to the interactive module (Touch screen) is displayed to request confirmation from the person on the vehicle; if the person on the vehicle confirms the operation on the interactive module (touch screen), the processor continues to execute steps S4-S5, otherwise, the processor does not execute steps S4-S5, that is, it does not generate and display the first display screen, but can directly display the first projection screen sent by the mobile device; then after a time of 2T, step S1 is triggered again, and the above steps are repeated... In other words, steps S1-S5 can be repeatedly executed multiple times, and each time steps S1-S5 are executed, confirmation from the person on the vehicle is requested after completing steps S1-S3 and before executing steps S4-S5. If the person on the vehicle does not confirm, confirmation is requested again after a certain time. In the case that the person on the vehicle does not confirm, the duration of each time is increased (for example, doubled) based on the duration of the previous time, thereby reducing the disturbance to the people on the vehicle.

[0110] In this embodiment, on the basis of executing steps S1-S5, the processor may further execute the following steps:

[0111] S6. determining at least part of the visualization information;

[0112] S7. For the determined visualization information, a first operation instruction is generated; the first operation instruction is used to instruct the mobile device to operate the application corresponding to the visualization information to generate a second projection screen;

[0113] S8. Receive the second projection screen sent by the mobile device, record the second projection screen, and obtain the recorded screen;

[0114] S9. Cache the recorded screen.

[0115] In step S6, Figure 3In the sorted visualization information shown, all or part of the visualization information is selected. If part of the visualization information is selected, specifically, the visualization information that is sorted the most can be selected except for the visualization information that is sorted the most.

[0116] In this embodiment, in step S6, if the step of partially visualizing information is selected, the following steps may be specifically performed:

[0117] S601A obtains the real-time operation of each corresponding application of each visualization information;

[0118] S602A. Get real-time threshold;

[0119] S603A. Filter out the visualization information whose corresponding running real-time is less than the real-time threshold as the determined visualization information.

[0120] In step S601A, the real-time operation refers to the real-time information processing and presentation when the application runs its function. The real-time operation of the application can be obtained according to the attribute information of the application. The real-time operation can be "high", "medium", "low", etc. For example, when the navigation application uses its navigation function, the real-time information processing and presentation is relatively high (for example, the information such as the location indicated by the navigation application needs to be viewed and used at the time, and it will not be effective if it is viewed after a period of time), so the real-time operation of the navigation application can be obtained as "high"; the real-time information processing and presentation of the artificial intelligence question and answer application is relatively low (for example, the question and answer dialogue of the artificial intelligence question and answer application can be viewed at the time and later), so the real-time operation of the navigation application can be obtained as "low".

[0121] In step S602A, a threshold for measuring the real-time performance of the operation may be set as a real-time threshold. In this embodiment, the real-time threshold may be set to "medium".

[0122] In step S603A, visualization information corresponding to the application programs with a running real-time lower than the real-time threshold, that is, visualization information corresponding to the application programs with a running real-time of "low", is screened out as the visualization information determined in step S6.

[0123] For example, the artificial intelligence question and answer application belongs to the visualization information determined by executing steps S601A-S603A.

[0124] By executing steps S601A-S603A, the determined visualization information all belongs to applications running with relatively low real-time performance.

[0125] In step S6, if the step of partially visualizing information is selected, the following steps may be performed:

[0126] S601B. Obtaining future driving scenario information;

[0127] S602B obtains the correlation between each visualization information corresponding to the application and the future driving scene information;

[0128] S603B. Get the relevance threshold;

[0129] S604B. Filter out visualization information whose corresponding correlation degree is greater than the correlation degree threshold as the determined visualization information.

[0130] In step S601B, a prediction algorithm may be executed to predict future driving scene information based on the real-time driving scene information obtained by executing step S2, or the real-time driving scene information obtained by executing step S2 may be directly used as future driving scene information. The future driving scene information has the same data format as the real-time driving scene information, and the future driving scene information can represent the driving scene of the vehicle within a period of time in the future (e.g., 10 minutes).

[0131] In step S602B, the function information of each application can be obtained from the attribute information of each application installed on the mobile terminal, and the matching degree between the function information and the future driving scene information can be obtained as the correlation between the visualization information and the future driving scene information. For example, if the future driving scene information obtained in step S601B indicates that the vehicle will arrive at a scenic spot, then the function information of the search application, social application, and artificial intelligence question and answer application has a high correlation with the future driving scene information, so the correlation corresponding to the visualization information of the search application, social application, artificial intelligence question and answer application, etc. is determined to be "high"; while the function information of the beauty application has a low correlation with the future driving scene information, so the correlation corresponding to the visualization information of the beauty application, etc. is determined to be "low".

[0132] In step S603B, a threshold for measuring the degree of association may be set as a correlation threshold. In this embodiment, the correlation threshold may be set to "medium".

[0133] In step S604B, visualization information corresponding to a correlation greater than a correlation threshold, that is, visualization information corresponding to those applications with "high" real-time performance, is screened out as the visualization information determined in step S6.

[0134] For example, the artificial intelligence question and answer application belongs to the visualization information determined by executing steps S601B-S604B.

[0135] By executing steps S601B-S603B, the determined visualization information all belongs to applications that are highly correlated with future driving scene information.

[0136] For the visualization information determined by executing step S6, a first operation instruction can be generated respectively, thereby instructing the mobile device to operate the application corresponding to the visualization information to generate a second projection screen.

[0137] For example, taking the artificial intelligence question-and-answer application determined in step S6 as an example, referring to Figure 5 , the corresponding first operation instruction is a set of prompt words (prompt), so as to operate the question-answering function of the artificial intelligence question-answering application. In this embodiment, the first operation instruction can be generated according to the real-time driving scene information, for example Figure 5 In the example, according to the real-time driving scene information, a prompt word is generated as the first operation instruction for the artificial intelligence question-and-answer application, which is "Hello, it is Sunday, February 9, 2025. We are about to arrive at the parking lot of scenic spot A. It is sunny outside and the temperature is 12 degrees Celsius. There are A (driver), B, C (elderly), and D (child) in our car. Everyone is very excited. How can I visit scenic spot A better?" The processor sends the first operation instruction to the communication module, thereby sending the first operation instruction to the mobile device.

[0138] After the mobile device receives the first operation instruction, it uses the first operation instruction to operate the artificial intelligence question and answer application, thereby triggering the artificial intelligence question and answer application to answer questions, and the mobile device generates a screen projection image of the process of the artificial intelligence question and answer application executing its function, thereby obtaining Figure 5 The second projection screen is shown. The mobile device sends the second projection screen to the processor.

[0139] The processor executes step S8, receives the second projection screen sent by the mobile device, and records the second projection screen to obtain the recorded screen. Specifically, the processor can execute step S9 to directly store the second projection screen as the recorded screen in the memory.

[0140] When executing steps S6-S9, the processor only caches the second projection screen, and does not control the touch screen to directly display the second projection screen, that is, when executing steps S6-S9, the touch screen can display the first display screen or other screens.

[0141] In this embodiment, on the basis of executing steps S6-S9, the processor may execute the following steps:

[0142] S10. Detecting a second operation instruction pointing to the visual information;

[0143] S11. Read the screen recording corresponding to the visual information pointed to by the second operation instruction;

[0144] S12. Generate a second display screen according to the read screen recording screen;

[0145] S13. Using the vehicle computer system, the first display image is superimposed on or replaced by the second display image.

[0146] In step S10, the processor may call the touch screen or voice sensor (microphone) to detect whether there is a second operation instruction pointing to the visual information. For example, if the person on the vehicle clicks on the visual information corresponding to the artificial intelligence question and answer application in the first display screen displayed on the touch screen, or if the person on the vehicle says "please open the artificial intelligence question and answer application", the processor will be triggered to generate a second operation instruction pointing to the visual information of the artificial intelligence question and answer application.

[0147] In step S11, the processor can read out Figure 5 The screen recording screen is shown, and step S12 is executed to generate a second display screen.

[0148] For the second display screen generated by executing step S12, when the processor executes step S13, the touch screen is controlled to overlay the first display screen with the second display screen, for example, after the first display screen is reduced, the first display screen and the second display screen are displayed at the same time, or the first display screen is set with a certain transparency as the background, and the second display screen is displayed on the first display screen. When the processor executes step S13, the touch screen can also be controlled to replace the first display screen with the second display screen, that is, the first display screen is no longer displayed, and the second display screen is switched to be displayed.

[0149] In this embodiment, the principle of executing steps S6-S13 is that when displaying the first display screen, for those applications with low real-time operation or high correlation with future driving scene information, the processor can automatically operate and record the application running on the mobile device first, and when the personnel on the vehicle actually need to operate these applications, the screen recording screen can be directly called to display the second display screen, thereby realizing the preloading of the application, improving the response speed of the application, and improving the user experience. The preloaded application has low real-time operation, and preloading has little effect on the functional effect of the application; or the preloaded application has a high correlation with future driving scene information, so the functional effect of the application can be obtained in time through preloading, thereby meeting the needs of future driving scene information.

[0150] A computer program that executes the screen projection method for the vehicle-mounted mobile device in this embodiment can be written and written into a computer device or storage medium. When the computer program is read out and executed, the screen projection method for the vehicle-mounted mobile device in this embodiment is executed, thereby achieving the same technical effect as the screen projection method for the vehicle-mounted mobile device in the embodiment.

[0151] It should be noted that, unless otherwise specified, when a feature is referred to as being "fixed" or "connected" to another feature, it may be directly fixed or connected to the other feature, or it may be indirectly fixed or connected to the other feature. In addition, the descriptions of up, down, left, right, etc. used in the present disclosure are only relative to the relative positional relationship of the components of the present disclosure in the accompanying drawings. The singular forms of "a", "" and "the" used in the present disclosure are also intended to include the plural forms, unless the context clearly indicates other meanings. In addition, unless otherwise defined, all technical and scientific terms used in this embodiment have the same meaning as those generally understood by those skilled in the art. The terms used in the specification of this embodiment are only for describing specific embodiments and are not intended to limit the present invention. The term "and / or" used in this embodiment includes any combination of one or more related listed items.

[0152] It should be understood that, although the term first, second, third etc. may be adopted to describe various elements in the present disclosure, these elements should not be limited to these terms. These terms are only used to distinguish the same type of elements from each other. For example, without departing from the scope of the present disclosure, the first element may also be referred to as the second element, and similarly, the second element may also be referred to as the first element. The use of any and all examples or exemplary language ("for example", "such as" etc.) provided by the present embodiment is only intended to better illustrate embodiments of the present invention, and unless otherwise required, the scope of the present invention will not be limited.

[0153] It should be appreciated that embodiments of the present invention may be implemented or enforced by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer-readable memory. The method may be implemented in a computer program using standard programming techniques - including a non-transitory computer-readable storage medium configured with a computer program, wherein the storage medium so configured causes the computer to operate in a specific and predefined manner - according to the methods and drawings described in the specific embodiments. Each program may be implemented in a high-level procedural or object-oriented programming language to communicate with a computer system. However, if desired, the program may be implemented in assembly or machine language. In any case, the language may be a compiled or interpreted language. In addition, the program may be run on a programmed dedicated integrated circuit for this purpose.

[0154] In addition, the operations of the process described in this embodiment may be performed in any suitable order, unless otherwise indicated in this embodiment or otherwise clearly contradicted by the context. The process described in this embodiment (or variations and / or combinations thereof) may be performed under the control of one or more computer systems configured with executable instructions, and may be implemented as a code (e.g., executable instructions, one or more computer programs, or one or more applications) executed on one or more processors in common, by hardware or a combination thereof. A computer program includes a plurality of instructions that may be executed by one or more processors.

[0155] Further, the method can be implemented in any type of computing platform that is operably connected to a suitable computer, including but not limited to a personal computer, a minicomputer, a mainframe, a workstation, a network or distributed computing environment, a separate or integrated computer platform, or in communication with a charged particle tool or other imaging device, etc. Various aspects of the present invention can be implemented in machine-readable code stored on a non-transitory storage medium or device, whether removable or integrated into a computing platform, such as a hard disk, an optical read and / or write storage medium, a RAM, a ROM, etc., so that it can be read by a programmable computer, and when the storage medium or device is read by the computer, it can be used to configure and operate the computer to perform the process described herein. In addition, the machine-readable code, or part thereof, can be transmitted via a wired or wireless network. When such media includes instructions or programs that implement the above steps in conjunction with a microprocessor or other data processor, the invention of this embodiment includes these and other different types of non-transitory computer-readable storage media. When programmed according to the methods and techniques of the present invention, the present invention also includes the computer itself.

[0156] The computer program can be applied to input data to perform the functions of the present embodiment, thereby converting the input data to generate output data stored in a non-volatile memory. The output information can also be applied to one or more output devices such as a display. In a preferred embodiment of the present invention, the converted data represents a physical and tangible object, including a specific visual depiction of the physical and tangible object produced on the display.

[0157] The above are only preferred embodiments of the present invention. The present invention is not limited to the above embodiments. As long as the technical effects of the present invention are achieved by the same means, any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included in the scope of protection of the present invention. Within the scope of protection of the present invention, its technical solutions and / or implementation methods may have various modifications and changes.

Claims

1. A method for implementing screen projection of a vehicle-mounted mobile device, characterized in that: The method for implementing screen projection of the vehicle-mounted mobile device includes: Obtaining an application list of a mobile device carried by a person in the vehicle; the application list includes visual information of each of a plurality of application programs running on the mobile device; Obtain real-time driving scene information; Sorting each of the visualization information according to the real-time driving scene information; Generate a first display screen according to the sorted visualization information; The first display screen is displayed through the vehicle system.

2. The method for implementing screen projection of a vehicle-mounted mobile device according to claim 1, characterized in that: The step of obtaining the application list of the mobile device carried by the person in the vehicle includes: Receiving a first projection screen sent by the mobile device; the first projection screen is generated by the mobile device according to a desktop page running on the mobile device; Identify the first projection screen and determine the application icons included in the first projection screen; The application list is composed by using the application icons as the visual information.

3. The method for implementing screen projection of a vehicle-mounted mobile device according to claim 1, characterized in that: The step of sorting each of the visualization information according to the real-time driving scene information includes: Get the AI ​​model; Inputting the real-time driving scene information into the artificial intelligence model for processing; Obtaining a sequence of application names output by the artificial intelligence model; The visualization information is sorted according to the order of each visualization information in the application name sequence.

4. The method for implementing screen projection of a vehicle-mounted mobile device according to claim 3, characterized in that: The artificial intelligence model is trained through a first training process, wherein the first training process includes: Acquire first historical driving scene information and a first actually used application sequence detected by multiple drivers when each of the drivers drives the car; the first actually used application sequence includes multiple application program names actually used by the driver when driving the car and sorted by actual use intensity; The first historical driving scene information is used as the input of the artificial intelligence model, and the first actual application sequence is used as the expected output of the artificial intelligence model to adjust the parameters of the artificial intelligence model.

5. The method for implementing screen projection of a vehicle-mounted mobile device according to claim 4, characterized in that: The method for implementing screen projection of the vehicle-mounted mobile device also includes: Acquire a plurality of second historical driving scene information detected by the vehicle occupant when driving the vehicle and a plurality of corresponding second actually used application sequences; the second actually used application sequences include a plurality of application program names actually used by the vehicle occupant when driving the vehicle and sorted by actual use intensity; When the cumulative number of all the second actually used application sequences is greater than a first number threshold, clustering each of the second actually used application sequences; When the number of clusters obtained by clustering is less than the second number threshold, the artificial intelligence model is trained in a second training process, and the second training process includes: The parameters of the artificial intelligence model are adjusted by using each of the second historical driving scene information as the input of the artificial intelligence model and each of the second actual use application sequences as the expected output of the artificial intelligence model.

6. The method for implementing screen projection of a vehicle-mounted mobile device according to any one of claims 1 to 5, characterized in that: The method for implementing screen projection of the vehicle-mounted mobile device also includes: determining at least a portion of said visualization information; For the determined visualization information, a first operation instruction is generated; the first operation instruction is used to instruct the mobile device to operate the application corresponding to the visualization information to generate a second projection screen; Receiving a second projection screen sent by the mobile device, and recording the second projection screen to obtain a recorded screen; The screen recording image is cached.

7. The method for implementing screen projection of a vehicle-mounted mobile device according to claim 6, characterized in that: The determining at least part of the visualization information comprises: Obtaining the running real-time performance of the application corresponding to each of the visualization information; Get the real-time threshold; Filtering out the visualization information corresponding to the operation real-time smaller than the real-time threshold as the determined visualization information; or Obtain information about future driving scenarios; Acquire the correlation between the application corresponding to each of the visualization information and the future driving scene information; Get the correlation threshold; The visualization information corresponding to the correlation degree greater than the correlation degree threshold is screened out as the determined visualization information.

8. The method for implementing screen projection of a vehicle-mounted mobile device according to claim 6, characterized in that: The method for implementing screen projection of the vehicle-mounted mobile device also includes: detecting a second operation instruction directed to the visual information; Reading the screen recording image corresponding to the visual information pointed to by the second operation instruction; Generate a second display screen according to the read screen recording screen; The first display picture is superimposed on or replaced by the second display picture through the vehicle system.

9. A computer device, characterized in that: It includes a memory and a processor, the memory is used to store at least one program, and the processor is used to load at least one program to execute the screen projection implementation method of the vehicle-mounted mobile device as described in any one of claims 1-8.

10. A computer-readable storage medium storing a program executable by a processor, characterized in that: The program executable by the processor is used to execute the screen projection implementation method of the vehicle-mounted mobile device described in any one of claims 1-8 when executed by the processor.

Images