In-vehicle application interaction method based on large model, vehicle, medium and product

Abstract

The invention discloses an in-vehicle application interaction method based on a large model, a vehicle, a medium and a product, and relates to the technical field of vehicle-mounted intelligent interaction.The method comprises the steps that when voice information of an in-vehicle user is collected, semantic understanding is conducted on voice text data corresponding to the voice information according to a preset large model, and a first recognition result is obtained; searching first associated data having a data association relationship with the first recognition result in a preset knowledge base by taking the first recognition result as a keyword; transmitting the first recognition result and the first associated data to each piece of first application software, performing data integration on feedback information fed back by at least one piece of first application software to obtain first text data, and outputting and displaying the first text data; and playing a song corresponding to the song information when the displayed first text data comprises the song information and a trigger instruction of the user for the song information is detected. According to the invention, the interaction intelligence of the vehicle-mounted system in the vehicle can be improved.

Description

Technical Field

[0001] This application relates to specific technologies in the field of in-vehicle intelligent interaction, and in particular to in-vehicle application interaction methods based on large models, vehicles, computer-readable storage media, and computer program products. Background Technology

[0002] With the continuous development of artificial intelligence technology, in-vehicle systems (such as in-vehicle information systems) have become an important part of modern automobiles. Among them, the in-vehicle infotainment system can realize multiple application functions, such as music playback and map navigation, and can provide drivers with personalized service experiences, improving driving comfort and safety.

[0003] However, in the complex in-vehicle environment, various application functions (such as music applications) are operated by voice commands. This operation method is simplistic and cannot meet the diverse habits of individual users. For example, if you ask someone to play a song by artist 1 with the lyrics "Give me two more minutes," the traditional method cannot understand this sentence. Furthermore, data from different music applications cannot be acquired simultaneously, which fails to provide users with a more convenient experience and results in a low level of intelligence in the interaction between users and the in-vehicle system.

[0004] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The main objective of this application is to provide a vehicle-based in-vehicle application interaction method, vehicle, computer-readable storage medium, and computer program product based on a large model, aiming to solve the technical problem of how to improve the intelligence of in-vehicle system interaction.

[0006] To achieve the above objectives, this application proposes a vehicle-based in-vehicle application interaction method based on a large model, applicable to vehicles. The vehicle's infotainment system is equipped with multiple application software programs. The vehicle-based in-vehicle application interaction method based on the large model includes: When voice information of users in the vehicle is collected, the speech text data corresponding to the voice information is semantically understood based on a pre-set large model to obtain the first recognition result; Search for first associated data that has a data relationship with the first identification result in the preset knowledge base, using the first identification result as the keyword. The knowledge base includes the data relationship between multiple keywords and multiple associated data. The first recognition result and the first associated data are transmitted to each first application software. The feedback information from at least one first application software is integrated to obtain the first text data. The first text data is then output and displayed. The first application software is the application software in the vehicle system that has a strong correlation with the first recognition result. When the first text data displayed includes song information, and a user trigger command for the song information is detected, the song corresponding to the song information is played.

[0007] Optionally, the step of integrating feedback information from at least one first application software to obtain first text data includes: The earliest feedback message among the feedback messages from at least one first application software is identified as the feedback keyword; Based on the large language model, the feedback keywords are integrated by combining the first recognition result and the first associated data to obtain the first integrated data; Using the first application software that provides feedback keywords as the data source, the first integrated data is labeled with the data source to obtain the first text data.

[0008] Optionally, the step of integrating feedback information from at least one first application software to obtain first text data further includes: The feedback information from at least one first application software is filtered to remove duplicate feedback information from each feedback information, thereby obtaining the first feedback information. If there are at least two different first feedback messages from the first application software, then based on the large language model combined with the first recognition result and the first associated data, the data of each first feedback message is integrated to obtain the second integrated data corresponding to each first feedback message; For each piece of second integrated data, the data is marked with the data source of the first application software corresponding to the second integrated data to obtain the second text data. Data aggregation is performed on all the second text data to obtain the first text data.

[0009] Optionally, the application software includes a navigation application, and after the step of outputting and displaying the first text data, it further includes: When the first text data displayed includes address information, and a user trigger command for the address information is detected, the target address is obtained by reasoning the address information based on the large model, and the target address is transmitted to the navigation application in the vehicle system. Receive route information from the current address to the target address from the navigation application, and display the route information.

[0010] Optionally, the application software includes a music application, and the step of playing the song corresponding to the song information when the first text data displayed includes song information and a user trigger command for the song information is detected includes: The first text data is displayed in a pop-up window. The first text data includes song information. When a user triggers a command for the song information, the pop-up window that displays the first text data plays the song corresponding to the song information. The pop-up window calls the music application to play the song corresponding to the song information and displays text content information related to the song in the pop-up window.

[0011] Optionally, the step of playing the song corresponding to the song information when the first text data displayed includes song information and a user trigger command for the song information is detected further includes: The first text data is displayed in a pop-up window. The first text data includes song information. When a user trigger command for the song information is detected, it is determined whether the triggered song information has copyright information in each music application for each triggered song information. If at least two music applications have version information of the triggered song, then the music application that has both copyright information of the triggered song and user membership account information is identified as the target music application, and the triggered song information is sent to the target application so that the target music application can play the song corresponding to the song information.

[0012] Optionally, the in-vehicle application interaction method based on a large model also includes: The application data of at least one application software in the vehicle system is obtained through a pre-defined large model standardized interface. Identify keywords in application data and related data that are strongly correlated with keywords through large models; Establish a knowledge base that includes each keyword, each associated data, and the correspondence between each keyword and each associated data, wherein each keyword corresponds to at least one associated data.

[0013] In addition, to achieve the above objectives, this application also proposes a vehicle, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the in-vehicle application interaction method based on a large model as described above.

[0014] In addition, to achieve the above objectives, this application also proposes a medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the in-vehicle application interaction method based on a large model as described above.

[0015] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the in-vehicle application interaction method based on a large model as described above.

[0016] In this embodiment, when voice information of a user inside the vehicle is collected, semantic understanding of the corresponding voice-text data is performed based on a large model to obtain a first recognition result. First associated data with a data relationship to the first recognition result is then identified in a knowledge base. This first associated data is transmitted to various first application software. This avoids the problem of inaccurate user intent recognition and inaccurate response content caused by simply searching the first recognition result. By utilizing a large model for accurate semantic understanding, the user intent is quickly determined, and the first recognition result is obtained. Furthermore, to better match the user intent, the first associated data associated with the first recognition result is searched in the knowledge base, and both the first recognition result and the first associated data are transmitted. The data is fed into each primary application software, and the feedback information from at least one primary application software is integrated to obtain and output the primary text data. This allows the data from various application software (i.e., the ecosystem) to be integrated through a large model, breaking down data barriers between ecosystems and making the in-vehicle cabin experience more comprehensive, convenient, and intelligent. For example, when the primary application software is a music application, the feedback data from different music applications can be integrated and output using the large model. Furthermore, the large model can use the first recognition result and the first association data to search multiple application software, so that the final output primary text data covers more dimensions of data information, thereby improving the intelligence of the in-vehicle system interaction. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating the first embodiment of the in-vehicle application interaction method based on a large model in this application. Figure 2 This is a schematic diagram of a pop-up window in the first embodiment of the in-vehicle application interaction method based on a large model in this application. Figure 3 This is a flowchart illustrating the second embodiment of the in-vehicle application interaction method based on a large model in this application. Figure 4 This is a schematic diagram of the overall process of the in-vehicle application interaction method based on a large model according to an embodiment of this application. Figure 5 This is a schematic diagram of the device structure of the hardware operating environment involved in the in-vehicle application interaction method based on a large model in the embodiments of this application.

[0020] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0021] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0022] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0023] Based on this, embodiments of this application provide an in-vehicle application interaction method based on a large model, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the in-vehicle application interaction method based on a large model according to this application.

[0024] In this embodiment, the in-vehicle application interaction method based on a large model is applied to a vehicle, and the in-vehicle application interaction method based on a large model includes steps S10 to S40.

[0025] Step S10: When collecting the voice information of the user in the vehicle, semantic understanding is performed on the voice text data corresponding to the voice information based on the preset large model to obtain the first recognition result; Optionally, the vehicle involved in this application embodiment may be equipped with an in-vehicle infotainment system, which may contain multiple normally functioning applications, such as music applications (e.g., QQ Music), navigation applications (e.g., Amap), and telephone applications. The large-scale model involved may be located in the background terminal of the in-vehicle infotainment system, or it may be an external terminal, server, or cloud connected to the in-vehicle infotainment system.

[0026] Optionally, the large model can be a large language model or other AI models such as "Tongyi Qianwen" or "DeepSeek". A large language model can be a language model, which is a model that can be used to perform natural language processing tasks, such as LLaMa2 13B, LLaMa2 7B, OPT 7B, etc.

[0027] Optionally, the large model can be trained in advance to enable it to achieve the expected functions. Furthermore, a mapping table of the correspondence between keywords and related data can be constructed based on the data in various applications within the vehicle system and stored in the knowledge base.

[0028] Optionally, when the vehicle's in-vehicle system collects the voice information of the user inside the vehicle through the microphone, the voice information can be recognized by a voice assistant, a voice recognition model, or other means to determine the user's intent and obtain the voice text data contained in the voice information.

[0029] Optionally, when collecting voice information from users inside the vehicle, the location of the user sending the voice information can also be identified (e.g., driver's seat, front passenger seat, and rear seat). If the user's location is determined to be the driver's seat, voice recognition is performed on the user's voice information to obtain voice-text data. If the user's location is determined to be the front passenger seat or a rear seat, it can be first determined whether the user has the corresponding control permissions. If the user has the corresponding control permissions, voice recognition is performed on the user's voice information to obtain voice-text data. If the user does not have the corresponding control permissions, a corresponding permission prompt message can be issued, and step S10 can be paused.

[0030] Optionally, the speech and text data can be input into a large model, which can then be used to perform semantic understanding of the speech and text data, such as entity recognition and intent recognition, to obtain the first recognition result.

[0031] For example, if the voice text data is "The mountains and rivers are safe and sound, and the fireworks are commonplace" and the user is asking who sings it, the large model can perform entity recognition when performing semantic understanding on the voice text data: identify the lyric fragment "The mountains and rivers are safe and sound, and the fireworks are commonplace" as the query entity, and perform intent recognition: determine that the user's intent is "query the singer". The large model performs corresponding processing based on the results of entity recognition and intent recognition to obtain the semantic understanding result (i.e., the first recognition result), such as singer 2.

[0032] Optionally, in a feasible embodiment, the in-vehicle application interaction method based on a large model further includes steps a10-a30.

[0033] Step a10: Obtain application data of at least one application software in the vehicle system through the preset large model standardized interface; Optionally, if the large model is set in a terminal included in the vehicle, such as directly in the in-vehicle infotainment system, then the standardized interface of the large model can be an internal interface of the in-vehicle infotainment system, so that the large model in the in-vehicle infotainment system can obtain application data of at least one application software set in the in-vehicle infotainment system through the standardized interface of the large model, such as application data of multiple music applications.

[0034] Optionally, if the large model is located on an external terminal, server, or cloud connected to the vehicle's infotainment system (the following example only uses external devices), then after the vehicle's infotainment system controller obtains the application data of at least one application software installed in the vehicle's infotainment system, it can send the application data of at least one application software to the large model located on the external device through the large model's standardized interface.

[0035] Optionally, the applications installed in the vehicle's infotainment system can include different types of applications, and there can be multiple applications of the same type. For example, if multiple different music applications are installed in the vehicle's infotainment system, such as Kugou Music, QQ Music, and Baidu Music, music data (such as music titles, lyrics, artists, etc.) can be obtained from Kugou Music, QQ Music, and Baidu Music respectively through a standardized interface with user authorization, and used as application data for each music application. If multiple different navigation applications are installed in the vehicle's infotainment system, such as Baidu Maps and Gaode Maps, map data (such as regional road maps and highway route maps of the vehicle's current location) can be obtained from Baidu Maps and Gaode Maps respectively through a standardized interface with user authorization, and used as application data for each navigation application. If a phone application is installed in the vehicle's infotainment system, phone data (such as contacts, phone numbers, call logs, etc.) from the user's mobile phone can be obtained through the phone application with user authorization, and used as application data for the phone application.

[0036] Step a20: Identify keywords in the application data and related data that are strongly correlated with the keywords through a large model; Step a30: Establish a knowledge base including each keyword, each associated data, and the correspondence between each keyword and each associated data, wherein one keyword corresponds to one associated data. Optionally, the application data of each application software can be input into a large model, such as the AI ​​large model (DeepSeek), and the large model can be used to identify and classify the input application data to determine the keywords in the application data of each application software and the related data that are strongly correlated with those keywords.

[0037] Optionally, a large model can be used to identify keywords in the application data of various application software of the same type, as well as to identify related data that are strongly correlated with the keywords. For example, keywords and their strongly correlated data can be identified in the music data of various music applications, in the map data of various navigation applications, and in the telephone data of telephone applications.

[0038] Optionally, after identifying keywords in the application data, a large model can be used to identify related data that are strongly correlated with the keywords. For example, if the keyword is "singer," then the strongly correlated words for that singer could be the singer's introduction and representative works.

[0039] Optionally, when the application software type includes music applications, then the application data can be determined to include music data. A large model can be used to identify the music data of various music applications. Keywords can be extracted from the music data, such as singers, song titles, and albums. Strongly correlated data with singers (e.g., singer's basic information, representative works, etc.), song titles (e.g., singer, lyrics, composer, style, release date, album, target audience, and charts), and albums (e.g., album description, release date, style, target audience, and charts) can also be extracted. Optionally, keywords can also be lyrics, and strongly correlated data with these keywords can include songs containing those lyrics and the corresponding singers.

[0040] Optionally, when the type of application software includes navigation applications, it can be determined that the application data includes map data. The map data of each navigation application can be identified through a large model. For example, supermarket A can be used as a keyword, and the specific locations of supermarket A in this area (such as No. B on A Road) can be used as strongly related data.

[0041] Optionally, when the type of application software includes telephone applications, it can be determined that the application data includes telephone data. The telephone data of each telephone application can be identified through a large model. For example, contact b can be used as a keyword, and the phone number, call records, etc. corresponding to contact b can be used as strongly correlated data.

[0042] Optionally, a template for guiding large models to perform model training operations can be set in advance. This template can be set and adjusted arbitrarily according to the user's needs, and the template can include at least one keyword, related data that is strongly associated with the at least one keyword, and the correspondence between the keyword and the related data.

[0043] Optionally, different types of applications may use different templates. Taking a music application as an example, the template can be as shown in Table 1 below.

[0044]

[0045] Table 1 Optionally, a pre-set template and various application data can be input into a large model. The large model extracts keywords and related data from each application data based on the input template, and then outputs at least one keyword and related data that are strongly correlated with the at least one keyword.

[0046] Optionally, a corresponding mapping table can be established and stored for each keyword output by the large model and the associated data that are strongly related to each keyword, thereby obtaining a knowledge base including each keyword, each associated data, and the correspondence between each keyword and each associated data.

[0047] Alternatively, the knowledge base can be a database or a device or server used to store data, such as a cache.

[0048] Optionally, different mapping tables can be created for different types of application software, and the number of mapping tables can be one or more. For example, for music applications, a first mapping table containing the data relationships between keywords and associated data can be created and stored in a knowledge base.

[0049] Optionally, the first mapping table can be as shown in Table 2 below.

[0050]

[0051] Table 2 Step S20: Search for first associated data that has a data association relationship with the first identification result in the preset knowledge base using the first identification result as the keyword. The knowledge base includes the data association relationships between multiple keywords and multiple associated data. Optionally, the knowledge base can be constructed according to steps a10-a30, or the user can pre-configure and construct a corresponding database as the knowledge base.

[0052] Optionally, when searching in the knowledge base based on the first recognition result, the user intent corresponding to the first recognition result (such as listening to music, making a phone call, or map navigation) can be determined by a large language model, and the corresponding mapping table can be searched in the knowledge base according to the determined user intent. For example, if the user intent is to listen to music, the mapping table corresponding to the music application can be searched in the knowledge base; if the user intent is to make a phone call, the mapping table corresponding to the phone application can be searched in the knowledge base; if the user intent is map navigation, the mapping table corresponding to the navigation application can be searched in the knowledge base.

[0053] Optionally, a precise search can be performed again within the mapping table found in the knowledge base to determine the keyword with the highest similarity to the first identification result and whose similarity to the first identification result is greater than a preset similarity threshold (e.g., 90%). Alternatively, the first identification result can be directly used as the keyword to search in the mapping table and determine the associated data corresponding to the keyword as the first associated data.

[0054] For example, if the first identification result is singer 2, it can be determined that the user's intent is related to music. The mapping table associated with music applications can be searched in the knowledge base, such as the first mapping table. The keyword corresponding to the first identification result and the associated data corresponding to the keyword can be determined in the first mapping table. For example, if the keyword is "singer" and the corresponding associated data is "basic information introduction of singer" and "representative works", then the first associated data can be the basic information introduction of singer 2 and the representative works of singer 2.

[0055] Optionally, in one implementation scenario, if no keyword with high similarity is found in the mapping table found in the knowledge base (i.e., the similarity between the first identification result and all keywords in the mapping table is less than a preset similarity threshold), a query can be performed on each associated data in the mapping table to determine the associated data with the highest similarity. Then, the keyword corresponding to the associated data with the highest similarity is determined as the keyword that matches the first identification result, and all associated data corresponding to the keyword that matches the first identification result in the mapping table are taken as the first associated data.

[0056] Optionally, in one implementation scenario, if no keyword with a high degree of similarity is found in the mapping table searched in the knowledge base, at least one application software can be directly determined based on the user intent corresponding to the first identification result, and the first identification result can be input into the at least one application software to obtain feedback information from the at least one application software.

[0057] Step S30: The first recognition result and the first associated data are transmitted to each first application software. The feedback information from at least one first application software is integrated to obtain the first text data. The first text data is then output and displayed. The first application software is the application software in the vehicle system that has a strong correlation with the first recognition result. Optionally, the application software to be enabled in the vehicle system can be determined based on the user intent corresponding to the first recognition result, and it can be regarded as the application key in the vehicle system that has a strong correlation with the first recognition result, i.e., the first application software.

[0058] Optionally, if the user's intention is to listen to music, then the various music applications in the in-vehicle infotainment system can be identified as the primary application software. The first recognition result and the first associated data can be input into each music application, and feedback information from each music application can be received. For example, when a user searches for a song title and artist name using lyrics, the voice assistant transmits the recognized lyrics to the large model. The large model can simultaneously perform retrieval processing through the lyrics search interfaces of various music applications (such as QQ Music, Baidu Music, and Kugou Music in the in-vehicle infotainment system), inputting the voice data and / or the first associated data into each music application, and receiving feedback information from each music application. Alternatively, the first recognition result can be input into each music application, and relevant data corresponding to the first associated data can be obtained online. For example, the first associated data can be input into a web application (such as Baidu) for a search, obtaining feedback information from Baidu, and the first recognition result can be input into each music application (such as QQ Music and Kugou Music), obtaining feedback information from QQ Music and Kugou Music.

[0059] Optionally, if the user's intention is map navigation, the voice data and / or the first associated data can be input into each navigation application (i.e., the first application software), and feedback information from each music application can be received; alternatively, if the user's intention is to make a phone call, the voice data and / or the first associated data can be input into the phone software (i.e., the first application software), and feedback information from each phone software can be received. For example, if the voice data and / or the first associated data includes a contact w, the feedback information from the phone software can include at least one phone number of the contact w, in which case the user can select a phone number to dial.

[0060] Optionally, feedback information received from the application software within a preset time range (e.g., 5 seconds) can be determined, and the received feedback information can be integrated to obtain a text fragment, which can be used as the first text data. The first text data can be output and displayed on the vehicle system or a terminal connected to the vehicle system, for example, in the form of a pop-up window.

[0061] For example, the large model connects to the data interfaces of Kuwo Music, NetEase Cloud Music, and QQ Music via API. When a user asks what the song title is, after speech recognition is completed, the data is sent to the large model. The large model uses semantic understanding to find that the song title is "Song Title 1" (i.e., the first recognition result). Based on the data correspondence, the large model searches online for "the singer, lyricist, composer, style, release time, album, target audience, and charts of the song title 1" (i.e., the first related data) (or searches in various music applications). At the same time, it searches for the song "Song Title 1" in music applications (such as Kuwo Music, NetEase Cloud Music, and QQ Music) through API interfaces and obtains various feedback information. The large model integrates the online search content and the data obtained from the interfaces of various music apps (i.e., data integration of the feedback information from various first application software) to obtain the first text data, and displays the first text data to the user in a pop-up window, showing the corresponding data source. Based on copyright and membership status, it recommends the user to play the source.

[0062] Optionally, such as Figure 2 As shown, the first text data could be: "The singer of 'Song Title 1' is Singer 1, the lyricist is A1, the composer is B1, the arranger is C1, the music style is pop, incorporating Chinese elements. The song was released on November 2, 2007, and included in Singer 1's album 1. This song is suitable for people who like Chinese-style music, classical poetry, and are interested in traditional culture, and is also very popular among Singer 1's fans. In terms of charts, 'Song Title 1' once ranked 5th on the song recognition chart. In addition, it also won the Best Song of the Year Award, Best Lyricist Award, Best Composer Award, and the champion award, among other honors." This first text data can be displayed via a pop-up window, which can also show the data source corresponding to the feedback data (such as Kuwo Music, NetEase Cloud Music, QQ Music, Kugou Music, etc.). Based on copyright and membership status, the pop-up window can recommend the user's playback source, for example, displaying "Considering copyright and membership status, we recommend you use QQ Music to play:". When displaying the data source, it can show two data sources corresponding to "Song Title 1" and "Singer 1", for example, 1 in the image represents QQ Music, and 2 represents Kuwo Music.

[0063] Step S40: When the first text data displayed includes song information and a user trigger command for the song information is detected, the song corresponding to the song information is played.

[0064] After the first text data is displayed, such as in a pop-up window, the user can perform secondary operations on the displayed first text data or pop-up window. For example, if the displayed first text data contains song information, the user can choose to play the corresponding song for at least one of the displayed song information. For instance, the pop-up window displays a trigger command (such as touching the play button) corresponding to at least one song information. When the user touches the trigger command, or when the user's new voice information contains the playback of at least one song information displayed in the pop-up window, the song corresponding to that at least one song information is played.

[0065] In this embodiment, when voice information from the user inside the vehicle is collected, semantic understanding of the corresponding voice-text data is performed based on a large model to obtain a first recognition result. First associated data with a data relationship to the first recognition result is then identified in the knowledge base, and this first associated data is transmitted to various first application software. This avoids the problem of inaccurate user intent recognition and inaccurate response display caused by simply searching the first recognition result. By utilizing a large model for accurate semantic understanding, the user intent is quickly determined, and the first recognition result is obtained. Furthermore, to better match the user intent, the first associated data associated with the first recognition result is searched in the knowledge base, and both the first recognition result and the first associated data are transmitted to the relevant application software. The data is input into various primary application software, and the feedback information from at least one primary application software is integrated to obtain and output the primary text data. This allows the data from various application software (i.e., the ecosystem) to be integrated through a large model, breaking down data barriers between ecosystems and making the in-vehicle cabin experience more comprehensive, convenient, and intelligent. For example, when the primary application software is a music application, the feedback data from different music applications can be integrated and output using the large model. Furthermore, the large model can use the first recognition result and the first association data to search multiple application software, so that the final output primary text data covers more dimensions of data information, thereby improving the intelligence of the in-vehicle system interaction.

[0066] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in the first embodiment can be referred to the above description, and will not be repeated hereafter. On this basis, refer to Figure 3 Step S30, which involves integrating feedback information from at least one first application software to obtain first text data, includes steps b10 to b30.

[0067] Step b10: Determine the earliest feedback message among the feedback messages from at least one first application software as the feedback keyword; Step b20: Based on the large language model, the feedback keywords are integrated by combining the first recognition result and the first association data to obtain the first integrated data; Step b30: Using the first application software that provides feedback keywords as the data source, mark the first integrated data to obtain the first text data.

[0068] Optionally, when the vehicle or vehicle-mounted system receives feedback information from at least one first application software, the earliest feedback information can be used as the feedback keyword, and the first identification result, the first key data, and the feedback keyword can all be input into the large language model so that the large language model can integrate the data to obtain the first integrated data.

[0069] Optionally, when the vehicle or vehicle-mounted system receives feedback information from at least one first application software, the earliest feedback information can be used as the feedback keyword. Based on the feedback keyword, the system can search the knowledge base for related data (such as third related data) that are associated with the feedback word. The feedback keyword and the third related data are then transmitted to each first application software for querying to obtain new feedback information from each first application software. Finally, the system integrates the data based on the large language model, the first recognition result, the first related data, the feedback keyword, and the new feedback information to obtain the first integrated data.

[0070] Optionally, the large model can integrate data by synthesizing the first associated data and feedback keywords into text to obtain local text data, then synthesizing the first recognition result with the local text data to obtain initial text data, and then using the large language model to semantically modify the initial text data to make it semantically fluent, thus obtaining the first integrated data.

[0071] Optionally, after obtaining the first integrated data, the first integrated data can be marked with a data source to obtain the first text data. That is, the text content in the first text data can include the first integrated data, and can also include data sources marked with the first integrated data, such as Kugou Music, QQ Music, etc.

[0072] In this embodiment, by determining the earliest feedback information in the feedback information as the feedback keyword, and based on the large language model, combining the first recognition result and the first associated data to integrate the feedback keyword, and obtaining the first text data by marking the data source, the first text data can be obtained quickly and accurately, and then the subsequent output display can be performed, thereby improving the interaction efficiency between the vehicle and the user.

[0073] Optionally, step S30, which involves integrating the feedback information from at least one first application software to obtain first text data, further includes steps c10-c40.

[0074] Step c10: Filter the feedback information from at least one first application software to filter out duplicate feedback information and obtain the first feedback information. Step c20: If there are at least two different first feedback messages from the first application software, then based on the large language model, combined with the first recognition result and the first associated data, each first feedback message is integrated to obtain the second integrated data corresponding to each first feedback message. Step c30: For each piece of second integrated data, the data source is marked using the first application software corresponding to the second integrated data as the data source, to obtain the second text data; Step c40: Perform data aggregation processing on all the second text data to obtain the first text data.

[0075] Optionally, when the vehicle system receives feedback information from multiple first application software, the feedback information can be filtered to remove duplicate information and obtain the first feedback information.

[0076] If all the feedback information from the first application software is consistent, then after filtering, the first feedback information obtained will be only one piece of feedback information, such as "The singer of 'Song Title 1' is singer 1, the lyricist of 'Song Title 1' is A1, the composer of 'Song Title 1' is B1, and the arranger of 'Song Title 1' is C1". At this time, the first feedback information can be integrated based on the large language model, the first recognition result, and the first associated data. The data obtained by integrating the first feedback information can be labeled with the data source of each first application software, thus obtaining the first text data.

[0077] If there are inconsistencies in the feedback information from at least two first application software, then after filtering, there will be multiple first feedback information, and each first feedback information corresponds to a different data source. Therefore, based on the large language model, the data of each first feedback information can be integrated and the data source can be marked to obtain the corresponding second text data. Then, the second text data can be aggregated and semantically modified to obtain first text data with fluent text semantics.

[0078] For example, if the first identification result is "song title is song title 1", the first associated data includes the singer, lyricist, composer, style, release date, and album of the song title 1. The feedback information from Kugou Music includes that the singer of "song title 1" is singer 1, the lyricist of "song title 1" is A1, the composer of "song title 1" is B1, and the arranger of "song title 1" is C1. The feedback information from Kuwo Music includes that the singer of "song title 1" is singer 1, the lyricist of "song title 1" is A1, the composer of "song title 1" is B1, and the arranger of "song title 1" is C1. The feedback information from QQ Music includes that the singer of "song title 1" is singer 1, the music style of "song title 1" is pop, "song title 1" was released on November 2, 2007, and "song title 1" is included in album 1. It is evident that the feedback content from Kugou Music and Kuwo Music is identical. Therefore, after filtering the feedback information, two sets of first feedback information are obtained. One set of first feedback information (hereinafter referred to as first feedback information 1) states: The singer of "Song Title 1" is Singer 1, the lyricist of "Song Title 1" is A1, the composer of "Song Title 1" is B1, and the arranger of "Song Title 1" is C1. The other set of first feedback information (hereinafter referred to as first feedback information 2) states: The singer of "Song Title 1" is Singer 1, the music style of "Song Title 1" is POP, "Song Title 1" was released on November 2, 2007, and "Song Title 1" is included in album 1.

[0079] Optionally, the first feedback information 1 is combined with "song title is song title 1" and "singer, lyricist, composer, style, release time, and album of song title 1" to obtain the second integrated data, which includes: the singer of "song title 1" is singer 1, the lyricist is A1, the composer is B1, and the arranger is C1. Kugou Music and Kuwo Music are used as data sources to mark the second integrated data to obtain the second text data.

[0080] Optionally, the first feedback information 2 is combined with "song title is song title 1" and "song title 1's singer, lyricist, composer, style, release date, and album" to obtain the second integrated data, which includes: the singer of "song title 1" is singer 1, the music style is pop, with Chinese elements, the song was released on November 2, 2007, and it is included in singer 1's album 1. QQ Music is used as the data source to mark the second integrated data to obtain the second text data.

[0081] Optionally, the second text data corresponding to the first feedback information 1 and the second text data corresponding to the first feedback information 2 are subjected to data aggregation processing to obtain the first text data, which may be as follows: The singer of "Song Title 1" is Singer 1, the lyricist is A1, the composer is B1, the arranger is C1, the music style is pop, and it incorporates Chinese elements. The song was released on November 2, 2007, and is included in Singer 1's album 1.

[0082] In this embodiment, by filtering multiple feedback information to obtain first feedback information, and when there are at least two different first feedback information from the first application software, the different first feedback information are integrated and data source marked according to the large language model to obtain second text data. Then, the second text data is aggregated to obtain first text data. This can ensure the accuracy of the first text data while also ensuring its simplicity, avoiding the occurrence of excessive redundancy.

[0083] Based on the first or second embodiment of this application, in the third embodiment of this application, the contents that are the same as or similar to the first or second embodiment described above can be referred to the above description, and will not be repeated hereafter.

[0084] Optionally, the application software includes navigation applications such as Baidu Maps and Gaode Maps.

[0085] Optionally, after step S30, which outputs and displays the first text data, steps d10-d20 are also included.

[0086] Step d10: When the first text data displayed includes address information and a user trigger command for the address information is detected, the target address is obtained by reasoning the address information based on the large model, and the target address is transmitted to the navigation application in the vehicle system. Step d20: Receive route information from the current address to the target address from the navigation application and display the route information.

[0087] Optionally, after displaying the first text data, the user can perform secondary operations on the first text data. For example, if the displayed first text data includes address information, the user can choose to perform corresponding navigation processing on at least one of the displayed address information. For example, if the first text data is displayed in a pop-up window, a trigger command (such as a touch selection button) corresponding to at least one address information can be displayed in the pop-up window. When the user touches the trigger command or when the user's new voice information contains the selection of at least one address information displayed in the pop-up window, the large model can infer the specific target address from the selected address information. In the inference process, a knowledge base can be used for inference. For example, a mapping table corresponding to the navigation application can be determined in the knowledge base, and the keyword with the highest matching degree with the address information and the associated information corresponding to the keyword can be determined in the mapping table. The large model combines the keyword with the highest matching degree with the address information and the associated information corresponding to the keyword to perform semantic organization and integration to obtain the target address. The target address is then input into the navigation application in the vehicle system, and the system receives and displays the various route information from the current address to the target address from the navigation application for the user to choose from.

[0088] Optionally, if multiple navigation applications (such as Baidu Maps and Gaode Maps) are installed in the vehicle's infotainment system, it can receive route information from multiple navigation applications and continue to display the route information from each navigation application in the form of pop-ups. The corresponding data source will be marked in the corresponding pop-up window. For example, pop-up window 1 displays the route information from Baidu Maps, and pop-up window 2 displays the route information from Gaode Maps.

[0089] In this embodiment, by including address information in the first text data displayed, and when the user triggers the address information, the target address can be obtained by reasoning based on the large model, and sent to the navigation application. The user can then receive and display the route information fed back by the navigation application, thereby enabling intelligent interaction between the user and the vehicle.

[0090] Based on the first, second, or third embodiment of this application, in the fourth embodiment of this application, the content that is the same as or similar to the first, second, or third embodiment described above can be referred to the above description, and will not be repeated hereafter.

[0091] Optionally, the application software includes music applications such as Kugou Music, Kuwo Music, and QQ Music.

[0092] Optionally, step S40, which involves playing the song corresponding to the song information when the first text data displayed includes song information and a user trigger command for the song information is detected, includes step e10.

[0093] Step e10: The first text data is displayed in a pop-up window. The first text data includes song information. When a user trigger command for the song information is detected, the song corresponding to the song information is played through the pop-up window that displays the first text data. The pop-up window calls the music application to play the song corresponding to the song information and displays text content information related to the song in the pop-up window.

[0094] Optionally, if the first text data is displayed in a pop-up window and the pop-up window content includes the first text data, then the pop-up window content includes song information, and when a user trigger command for the song information is detected, the song corresponding to the song information can be played directly. While the song is playing, the pop-up window interface can be retained, and the corresponding music application can be called through the background of the vehicle system to play the song. At this time, the vehicle system interface does not display the music application interface, but can continue to display the pop-up window interface.

[0095] Optionally, when a song is playing on the in-vehicle infotainment system, a pop-up window can be used to display text information related to the song being played, such as lyrics and song title.

[0096] Optionally, the pop-up can be split into two windows to display the content of the previously displayed pop-up and the text content related to the song in the current display. For example, the pop-up can be divided into left and right windows, with the left window continuing to display the pop-up content and the right window displaying the text content related to the currently playing song.

[0097] In this embodiment, when the display mode is a pop-up window, the user can perform secondary operations on the displayed first text data, such as playing a song. The song can be played directly through the pop-up window without the user having to manually open the music application again, which improves the intelligence of the user's interaction with the vehicle.

[0098] Optionally, step S40, which involves playing the song corresponding to the song information when the first text data displayed includes song information and a user trigger command for the song information is detected, further includes steps f10-f20.

[0099] Step f10: The first text data is displayed in a pop-up window. The first text data includes song information. When a user trigger command for the song information is detected, for each triggered song information, it is determined whether the triggered song information has copyright information in each music application. Optionally, when the displayed pop-up content (such as the first text data) includes song information, and a user trigger command for at least one song information is detected, the same operation can be performed for each triggered song information to play the song corresponding to each triggered song information.

[0100] Optionally, for each triggered song information, it can first be determined whether each music application in the vehicle's infotainment system has the copyright information to play the song corresponding to the triggered song information; if none of them have the corresponding copyright information, a corresponding prompt message will be output to remind the user that the song corresponding to the triggered song information cannot be played.

[0101] Step f20: If at least two music applications have version information of the triggered song, then the music application that has both copyright information of the triggered song and user membership account information is identified as the target music application, and the triggered song information is sent to the target music application so that the target music application can play the song corresponding to the triggered song information.

[0102] Optionally, if a music application has version information of the triggered song, the song corresponding to the triggered song information can be played directly through that music application.

[0103] Optionally, if at least two music applications have version information of the triggered song, then membership account detection can be performed on the at least two music applications to detect whether the user account currently logged into the vehicle system has a membership account in the at least two music applications. If one of the music applications has a membership account, then that music application can be used as the target music application that simultaneously has the copyright information of the triggered song and the user's membership account information, and the song corresponding to the triggered song information can be played through the target music application.

[0104] Optionally, if none of the music applications with the version information of the triggered song have the corresponding member account information, then one music application can be selected from all the music applications with the version information of the triggered song (for example, the music application with the longest playback time can be selected as the target music application, or the music application selected by the user in the last operation can be selected as the target music application, etc.) as the target music application, and the song corresponding to the triggered song information can be played through the target music application.

[0105] In this embodiment, when the display mode is a pop-up window, the user can perform secondary operations on the displayed first text data, such as playing a song. The user can also choose a music application that contains both the copyright information of the song and the user's membership account information to play the song, thereby improving the intelligence of the interaction between the user and the vehicle and meeting the needs of real-time dynamic driving scenarios.

[0106] In addition, to aid in understanding the principles of the in-vehicle application interaction method based on a large model in this embodiment, examples are provided below.

[0107] For example, such as Figure 4 As shown, data is collected through interface integration; data correspondence is established; integrated into large-scale model training; information / query / search / manipulation is triggered; the large-scale model performs semantic understanding; the data correspondence is determined, and if it exists, the large-scale model searches for relevant information online based on the data correspondence, while simultaneously querying song information from multiple ecosystem application interfaces; it is determined whether there are multiple data sources; if not, the large-scale model integrates the information and outputs a pop-up window displaying one data source; if yes, multiple data sources are displayed; it is determined whether a secondary operation should be performed, and if not, the pop-up window displays the end after 5 seconds; if yes, the type of secondary operation is determined; if it is other, navigation / playlist creation (created to the corresponding APP based on the audio source) / saving to the memo is initiated according to the operation content.

[0108] If the secondary operation type is playing music, then it checks which app has the copyright. If not, a pop-up window will display that the song is copyright-invalid and cannot be played. If it does, it checks which app has a membership. If not, the default app will play the song. If it does, the corresponding app will be selected for playback.

[0109] Optionally, an example can be given using a music application.

[0110] For example, when a user asks, "The mountains and rivers are safe, and the fireworks are commonplace," who sings the song? After semantic understanding, the large model obtains the answer "Singer 2." It can then search for related song recommendations for Singer 2 through the song retrieval interfaces of various music applications in the car's infotainment system (such as Kuwo, QQ Music, and NetEase Cloud Music). It obtains the first feedback information from each music application and selects the earliest feedback information as the second feedback information. If the second feedback information is Singer 2's song 1 from QQ Music, the large model can then use Singer 2's song 1 as a keyword to query the knowledge base and obtain the associated data corresponding to that keyword. This data is then used as the second associated data (such as the singer, lyricist, and composer of song 1). The first associated data, the second feedback information, and the second associated data are then integrated to obtain the pop-up content to be displayed. This pop-up content is then displayed on the car's infotainment system, for example, displaying the answer "Singer 2, basic information about Singer 2, and representative works" as the pop-up content.

[0111] Furthermore, this application provides a vehicle, the vehicle including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to execute the in-vehicle application interaction method based on a large model as described in Embodiment 1 above.

[0112] The following is for reference. Figure 5The figure illustrates a structural diagram of a vehicle suitable for implementing embodiments of this application. The vehicle in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. The devices shown in the figure are merely examples and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0113] The vehicle may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for device operation. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. The communication device 1009 allows the vehicle to communicate wirelessly or wiredly with other devices to exchange data. Although vehicles with various systems are shown in the figures, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.

[0114] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0115] The vehicle provided in this application, employing the large-model-based in-vehicle application interaction method described in the above embodiments, enables the vehicle system to meet the needs of real-time dynamic driving scenarios. Compared with the prior art, the beneficial effects of the vehicle provided in this application are the same as those of the large-model-based in-vehicle application interaction method provided in the above embodiments, and other technical features of this vehicle are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0116] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0117] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0118] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the in-vehicle application interaction method based on a large model in the above embodiments.

[0119] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0120] The aforementioned computer-readable storage medium may be included in the vehicle or may exist independently and not installed in the vehicle.

[0121] The aforementioned computer-readable storage medium carries one or more programs that, when executed by a vehicle, enable the vehicle to perform the steps of the aforementioned in-vehicle application interaction method based on a large model.

[0122] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0123] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0124] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0125] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., computer programs) for executing the above-described in-vehicle application interaction method based on a large model, enabling the vehicle system to meet the needs of real-time dynamic driving scenarios. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the in-vehicle application interaction method based on a large model provided in the above embodiments, and will not be repeated here.

[0126] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described in-vehicle application interaction method based on a large model.

[0127] The computer program product provided in this application enables vehicle systems to meet the needs of real-time dynamic driving scenarios. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the in-vehicle application interaction method based on a large model provided in the above embodiments, and will not be repeated here.

[0128] The above are only some embodiments of this application and do not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for in-vehicle application interaction based on a large model, characterized in that, Applied to vehicles, where the vehicle's infotainment system is equipped with multiple application software, the in-vehicle application interaction method based on a large model includes: When voice information of users in the vehicle is collected, semantic understanding of the voice text data corresponding to the voice information is performed based on a preset large model to obtain the first recognition result. In a preset knowledge base, the first identification result is used as a keyword to search for first associated data that has a data association relationship with the first identification result. The knowledge base includes data association relationships between multiple keywords and multiple associated data. The first identification result and the first associated data are transmitted to each first application software. The feedback information from at least one first application software is integrated to obtain first text data. The first text data is then output and displayed. The first application software is an application software in the vehicle system that has a strong correlation with the first identification result. When the first text data displayed includes song information, and a user trigger command for the song information is detected, the song corresponding to the song information is played.

2. The in-vehicle application interaction method based on a large model as described in claim 1, characterized in that, The step of integrating feedback information from at least one of the first application software to obtain first text data includes: The earliest feedback message among the feedback messages from at least one of the first application software is identified as the feedback keyword; Based on the large language model, the feedback keywords are integrated by combining the first recognition result and the first associated data to obtain the first integrated data; Using the first application software that provides the feedback keywords as the data source, the first integrated data is marked with a data source tag to obtain the first text data.

3. The in-vehicle application interaction method based on a large model as described in claim 1, characterized in that, The step of integrating the feedback information from at least one of the first application software to obtain the first text data further includes: The feedback information from at least one of the first application software is filtered to remove duplicate feedback information from each feedback information, thereby obtaining the first feedback information; If there are at least two different first feedback messages from the first application software, then based on the large language model combined with the first recognition result and the first associated data, each first feedback message is integrated to obtain the second integrated data corresponding to each first feedback message. For each piece of the second integrated data, the data is marked with a data source using the first application software corresponding to the second integrated data as the data source, to obtain the second text data. Data aggregation is performed on all the second text data to obtain the first text data.

4. The in-vehicle application interaction method based on a large model as described in claim 1, characterized in that, The application software includes a navigation application, and after the step of displaying the first text data, it further includes: When the first text data displayed includes address information, and a user trigger command for the address information is detected, the target address is obtained by reasoning about the address information based on the large model, and the target address is transmitted to the navigation application in the vehicle system. Receive route information from the current address to the target address from the navigation application, and display the route information.

5. The in-vehicle application interaction method based on a large model as described in claim 1, characterized in that, The application software includes a music application. The step of playing the song corresponding to the song information when the first text data displayed includes song information and a user trigger command for the song information is detected includes: The first text data is displayed in a pop-up window. The first text data includes song information. When a user triggers a command for the song information, the pop-up window displaying the first text data plays the song corresponding to the song information. The pop-up window calls a music application to play the song corresponding to the song information and displays text content information related to the song in the pop-up window.

6. The in-vehicle application interaction method based on a large model as described in claim 5, characterized in that, The step of playing the song corresponding to the song information when the first text data displayed includes song information and a user trigger command for the song information is detected further includes: The first text data is displayed in a pop-up window. The first text data includes song information. When a user trigger command for the song information is detected, it is determined whether the triggered song information has copyright information in each of the music applications for each triggered song information. If at least two of the music applications have version information of the triggered song, then the music application that simultaneously has copyright information of the triggered song and user membership account information is identified as the target music application, and the triggered song information is sent to the target application so that the target music application can play the song corresponding to the song information.

7. The in-vehicle application interaction method based on a large model as described in claim 1, characterized in that, The in-vehicle application interaction method based on a large model also includes: The application data of at least one application software in the vehicle system is obtained through a pre-defined large model standardized interface. The large model identifies keywords in the application data, as well as related data that are strongly correlated with the keywords. A knowledge base is established, comprising each of the aforementioned keywords, each of the aforementioned associated data, and the correspondence between each of the aforementioned keywords and each of the aforementioned associated data, wherein each of the aforementioned keywords corresponds to at least one of the aforementioned associated data.

8. A vehicle, characterized in that, The vehicle includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the in-vehicle application interaction method based on a large model as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the in-vehicle application interaction method based on a large model as described in any one of claims 1 to 7.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the in-vehicle application interaction method based on a large model as described in any one of claims 1 to 7.

Images