Vehicle machine control method and device and storage medium

Abstract

The invention discloses a vehicle machine control method and device and a storage medium. The vehicle machine control method comprises the steps that action information of a person in a vehicle is obtained, the action information and verification information are verified, and when the action information and the verification information are matched, a social application is activated and operated. By executing the vehicle machine control method, the person in the vehicle can activate and wake up the social application for use only through the forms of voice, specially made gestures and other limb actions or limb actions related to steering wheel turning and other driving operations, so that the social application can be conveniently used on the vehicle, especially in the driving process, and the driving experience of the person in the vehicle is improved. A driver does not need to make dangerous behaviors such as two-hand departure from a steering wheel or sight deviation, and traffic safety is guaranteed. The invention is widely applied to the technical field of automobiles.

Description

Technical Field

[0001] The present invention relates to the technical field of automobiles, and in particular to a vehicle machine control method, device, and storage medium. Background Art

[0002] Social applications can maintain close communication between different people, so the use of social applications has become an important part of modern life. Currently, people widely use social applications on mobile devices such as mobile phones. Usually, it is necessary to type on the mobile phone to edit the conversation content in the social application. Even when chatting in a voice manner, it is necessary to operate the mobile phone with hands to record or initiate a call. However, when driving a car, since both hands are needed to control components such as the steering wheel and gear lever, in order to ensure traffic safety, it is impossible to be distracted to operate, which brings inconvenience to the use of social applications. Summary of the Invention

[0003] Aiming at the current technical problems such as the inconvenience of using social applications in cars, the purpose of the present invention is to provide a vehicle machine control method, device, and storage medium.

[0004] On the one hand, an embodiment of the present invention includes a vehicle machine control method, and the vehicle machine control method includes the following steps:

[0005] Obtain the motion information of the people in the vehicle;

[0006] Verify the motion information with verification information;

[0007] When the motion information matches the verification information, activate and run the social application.

[0008] Further, the activation and running of the social application includes:

[0009] Activate and run the social application in the vehicle machine system;

[0010] Or

[0011] Activate and run the social application on the mobile device of the people in the vehicle;

[0012] Project the running result of the social application on the mobile device to the vehicle machine system.

[0013] Further, the vehicle machine control method further includes:

[0014] Obtain the conversation information formed by the people in the vehicle using the social application;

[0015] Generate a control instruction for the vehicle-mounted components according to the conversation information;

[0016] Execute the control instruction to control the vehicle-mounted components.

[0017] Further, generating a control instruction for a vehicle component according to the conversation information includes:

[0018] Obtaining the current conversation record of the other party according to the conversation information; the current conversation record of the other party is the current conversation record formed by other participants who participate in the conversation with the vehicle occupants through the social application;

[0019] Verifying the current conversation record of the other party;

[0020] When the verification passes, generating the control instruction according to the current conversation record of the other party.

[0021] Further, verifying the current conversation record of the other party includes:

[0022] Obtaining the current conversation record of the home party according to the conversation information; the current conversation record of the home party is the current conversation record formed by the vehicle occupants participating in the conversation through the social application;

[0023] Obtaining the first semantic information corresponding to the current conversation record of the home party and the second semantic information corresponding to the current conversation record of the other party;

[0024] Obtaining the matching degree between the first semantic information and the second semantic information;

[0025] When the matching degree is greater than the matching degree threshold, it is determined that the verification passes.

[0026] Further, verifying the current conversation record of the other party includes:

[0027] Obtaining the historical conversation record of the home party and the historical conversation record of the other party according to the conversation information; the historical conversation record of the home party is the historical conversation record formed by the vehicle occupants participating in the conversation through the social application, and the historical conversation record of the other party is the historical conversation record formed by other participants who participate in the conversation with the vehicle occupants through the social application;

[0028] Obtaining the third semantic information corresponding to the historical conversation record of the home party and the fourth semantic information corresponding to the historical conversation record of the other party;

[0029] Obtaining the intimacy between the third semantic information and the fourth semantic information;

[0030] When the intimacy is greater than the intimacy threshold, it is determined that the verification passes.

[0031] Further, the vehicle machine control method further includes:

[0032] Obtaining the control result of controlling the vehicle component by executing the control instruction;

[0033] Generate response information according to the control result;

[0034] Input the response information as the speech of the conversation corresponding to the conversation information participated by the vehicle occupants into the social application.

[0035] Further, the generating response information according to the control result includes:

[0036] During the driving process of the vehicle triggered by the control result, collect information on the driving state of the vehicle and / or the external environment of the vehicle to obtain driving process information;

[0037] Use the driving process information as a prompt word and input it into the large language model;

[0038] Obtain the output result of the large language model;

[0039] Determine the response information according to the output result.

[0040] On the other hand, an embodiment of the present invention further includes a computer device, including a memory and a processor. The memory is used to store at least one program, and the processor is used to load at least one program to execute the vehicle machine control method in the embodiment.

[0041] On the other hand, an embodiment of the present invention further includes a computer-readable storage medium, in which a program executable by a processor is stored. The program executable by the processor is used to execute the vehicle machine control method in the embodiment when executed by the processor.

[0042] The beneficial effect of the present invention is that by executing the vehicle machine control method in the embodiment, vehicle occupants only need to activate and wake up the social application for use in forms such as voice, body movements such as special gestures, or body movements related to driving operations such as turning the steering wheel, etc., so as to facilitate the use of the social application in the car, especially during the driving process, without the driver performing dangerous behaviors such as leaving both hands off the steering wheel or deviating the line of sight, thus ensuring traffic safety. Description of the Drawings

[0043] Figure 1 It is a schematic diagram of an automotive system to which the vehicle machine control method can be applied in the embodiment;

[0044] Figure 2 It is a schematic diagram of the steps of the vehicle machine control method in the embodiment;

[0045] Figure 3 It is a schematic diagram of the principle of steps S4 - S6 in the embodiment. Detailed Embodiments

[0046] In this embodiment, the in-vehicle system control method can be applied to Figure 1 the automotive system shown. Refer to Figure 1 , the automotive system includes a control module (in-vehicle system), a voice sensor (microphone), a body movement sensor, an environmental information sensor, a driving state sensor, a communication module, and vehicle components.

[0047] Among them, the control module is a component with control and data processing functions; the voice sensor (microphone) can be installed at a position inside the vehicle passenger compartment to collect the spoken voices of vehicle occupants such as the driver and passengers; the body movement sensor can be a camera installed inside the vehicle passenger compartment, which detects body movements such as the gestures, head, or facial expressions of vehicle occupants by taking images of the vehicle occupants and analyzing their movements. Alternatively, the body movement sensor can also be a sensor installed on components such as the vehicle's steering wheel, accelerator pedal, brake pedal, gear shift lever, air conditioning button, window button, and ambient light button, used to detect body movements such as turning the steering wheel, stepping on the accelerator pedal, stepping on the brake pedal, operating the gear shift lever, pressing the air conditioning button, pressing the window button, and pressing the ambient light button by vehicle occupants; the environmental information sensor can be a camera installed at an external position of the vehicle to collect external environment images, or a microphone to collect external sounds, or a positioning device to detect the real-time position of the vehicle; the driving state sensor can be an engine speed sensor, a vehicle speed sensor, or an acceleration sensor, so as to collect driving states such as engine speed, vehicle driving speed, and vehicle driving acceleration; the communication module can be connected and communicate with mobile devices such as mobile phones through wireless or wired communication protocols such as Bluetooth, WiFi, and USB, and the communication module can also be connected and communicate with devices such as base stations through wireless communication protocols such as 5G; the vehicle components include components such as audio-visual entertainment devices, windows, ambient lights, and air conditioners used to adjust the vehicle driving and riding environment, and can also include components such as engines, brakes, gearboxes, and autonomous driving systems used to drive the vehicle.

[0048] In this embodiment, the vehicle that has the structure shown in Figure 1 and executes the in-vehicle system control method is called "this vehicle". Specifically, the control module can execute each step in the in-vehicle system control method, and the control module can call other components of this vehicle when executing the steps. Refer to Figure 2 , the in-vehicle system control method includes the following steps:

[0049] S1. Obtain the movement information of vehicle occupants;

[0050] S2. Verify the movement information with verification information;

[0051] S3. When the movement information matches the verification information, activate and run the social application.

[0052] The vehicle occupants involved in Steps S1 - S3 can refer to the driver or passengers in the vehicle. In this embodiment, the driver during the driving process is taken as an example of the vehicle occupant for illustration. The social applications involved in Steps S1 - S3 include instant messaging software (Instant Message, IM, also known as chat software), video live streaming software, information publishing and commenting platforms, etc. These social applications all involve obtaining the information of the local party (the driver of this vehicle) (specific forms include the driver's speech in a certain chat in the instant messaging software, the video stream captured of the driver in the video live streaming software, the post published by the driver on the information publishing and commenting platform), and sending it to the other party (including the private chat or group chat participants in the instant messaging software, the video viewers in the video live streaming software, the post readers on the information publishing and commenting platform), and obtaining the information of the other party (specific forms include the speech of the other party in the same chat in the instant messaging software, the comments and likes on the video stream captured of the driver by the video viewers in the video live streaming software, the comments, likes and forwards on the post published by the driver by the post readers on the information publishing and commenting platform, etc.) and feeding it back to the local data processing process.

[0053] In this embodiment, it is assumed that there is only one vehicle occupant in the vehicle for illustration. Without special indication, the vehicle occupant, the mobile device used by them, and the social application running on the mobile device may not be distinguished, and the instant messaging software is taken as an example of the social application for illustration.

[0054] In Step S1, the action information to be obtained by the control module (in - vehicle system) can be the voice information or the body movement information of the vehicle occupant. For example, when executing Step S1, the control module (in - vehicle system) can call the voice sensor (microphone) to detect the speech of the vehicle occupant and obtain the action information in the form of voice information; the control module (in - vehicle system) can also call the body movement sensor to detect the gestures, steering wheel operation actions or button - pressing actions of the vehicle occupant and obtain the action information in the form of body movement information.

[0055] In Step S2, the control module pre - stores verification information, where the data type of the verification information is the same as that of the action information. For example, for the action information in the form of voice information, the verification information represents the voice or text of a specific semantic content. For example, the content of the verification information can be "Please open the chat software"; for the action information in the form of body movement information, the verification information represents the body movement of a specific content. For example, the content of the verification information can be "Steer the steering wheel continuously for left - turn, straight - ahead, right - turn, straight - ahead, left - turn", etc.

[0056] When executing step S2, for the action information in the form of voice information, the control module can run a speech recognition algorithm to recognize the content of the action information and compare it with the verification information. If the action information contains the same content as the verification information (such as "Please open the chat software"), it is determined that the action information matches the verification information, triggering the execution of step S3. Otherwise, it is determined that the action information does not match the verification information, and step S3 is not executed; for the action information in the form of body movement information, the control module can compare the content of the action information with the verification information. If the action information contains the same content as the verification information (such as "Manipulate the steering wheel to continuously turn left, go straight, turn right, go straight, turn left"), it is determined that the action information matches the verification information, triggering the execution of step S3. Otherwise, it is determined that the action information does not match the verification information, and step S3 is not executed.

[0057] When executing step S2, the recognition of the action information in the form of voice information can be optimized as follows:

[0058] 1. Record the voice data input by the vehicle occupants each time and the recognized classical Chinese instructions, and upload them to the factory server using the vehicle networking platform.

[0059] 2. Manually check whether the voice and the recognized instructions match. If they do not match, mark them to generate a proprietary data set. Add the proprietary data set to the speech recognition model for training to continuously improve the speech recognition accuracy.

[0060] 3. Record the classical Chinese instructions recognized by the vehicle occupants each time to generate a historical vocabulary. When the vehicle occupants speak again, first preliminarily recognize the classical Chinese, and then based on the pinyin similarity matching technology, search for the voice instructions in the historical vocabulary to improve the speech recognition accuracy according to the historical habits of the vehicle occupants.

[0061] In step S3, the control module activates and runs the social application. The control module (in-vehicle system) can respond to the action information, query the SDK interface of the social application from a command library. The query method can be based on various matching technologies such as synonym matching. Based on the queried SDK interface, the content of the current action information is passed in to realize the wake-up and control of the social application.

[0062] When executing step S3, the control module (in-vehicle system) can activate and run the social application within itself, that is, within the control module (in-vehicle system). Specifically, the control module (in-vehicle system) can obtain the authorization of the social application running on the mobile device used by the vehicle occupants, and also run the social application within itself. That is, the control module (in-vehicle system) is authorized to obtain all the processing permissions for the social application and the data to be processed and generated by it, and the social application run by the control module (in-vehicle system) is synchronized with the social application running on the mobile device used by the vehicle occupants. The control module (in-vehicle system) activates the social application it runs to an available state. For example, the social application run by the control module (in-vehicle system) is in a state of waiting for the vehicle occupants to speak to input voice content.

[0063] When executing step S3, the control module (in-vehicle system) can also not run the social application within itself, but instead send an instruction to activate and run the social application to the mobile device of the vehicle occupants through the communication module, thereby triggering the mobile device to activate and run the social application, making the social application in an available state. For example, it is in a state of waiting for the vehicle occupants to speak to input voice content. The mobile device can project the running result of its social application to the in-vehicle system. For example, it can cast the image interface of the social application to the control module (in-vehicle system).

[0064] By executing step S3, if the social application is not started and run by the mobile device or the control module (in-vehicle system), then the mobile device or the control module (in-vehicle system) starts and runs the social application; if the social application is transferred to the background by the mobile device or the control module (in-vehicle system), then the mobile device or the control module (in-vehicle system) activates the social application; if the social application itself has been normally run by the mobile device or the control module (in-vehicle system), then the mobile device or the control module (in-vehicle system) maintains the current running state of the social application.

[0065] After executing step S3, whether the social application is run by the control module (in-vehicle system), run by the mobile device, or the control module (in-vehicle system) and the mobile device run the social application synchronously, the social application is in an available state. The vehicle occupants can use the social application to have a conversation (to ensure driving safety, it can be restricted that the vehicle occupants can only use voice input). When the social application receives messages of types such as text, voice, emoji, and video sent by the other party, the control module (in-vehicle system) can also display or play the messages. In the case of playing the messages, the control module (in-vehicle system) can control the audio-visual entertainment device to suppress the sound of the processes such as the music being played, and control the audio-visual entertainment device to play the messages received by the social application.

[0066] In this embodiment, by performing steps S1 - S3, the occupants in the vehicle can activate and wake up the social application for use only through forms such as voice, specific gestures and other body movements, or body movements related to driving operations such as turning the steering wheel. Thus, it is convenient to use the social application in the vehicle, especially during driving, without the driver performing dangerous behaviors such as taking both hands off the steering wheel or deviating the line of sight, ensuring traffic safety.

[0067] In this embodiment, on the basis of performing steps S1 - S3, the following steps can also be executed:

[0068] S4. Obtain the conversation information formed by the occupants in the vehicle using the social application;

[0069] S5. Generate a control instruction for the vehicle-mounted components according to the conversation information;

[0070] S6. Execute the control instruction to control the vehicle-mounted components.

[0071] The principle of steps S4 - S6 is as Figure 3 shown. In this embodiment, Figure 3 each session participant such as A, B, and C shown is a natural person, and the session participant can also be a robot, such as an artificial intelligence question-and-answer model, a large language model, etc.

[0072] In step S4, the control module (in-vehicle system) can search the data in the social application (instant messaging software) and find the sessions therein. For example, Figure 3 in, it is found that there is a session named "Hometown Chat Group" in the social application (instant messaging software) used by the occupants in the vehicle, which includes the conversation information generated by the occupants in the vehicle (the local party) and participants such as A, B, and C (all belonging to the other party) participating in this session. Figure 3 The session in is a group chat, and step S4 can also be applied to private chat sessions.

[0073] Figure 3 In, the time point is February 4th, that is, the part corresponding to before 09:56 (current time) is the conversation generated by all parties before. The content of A, B, and C such as "@Occupant in the vehicle Have you arrived home, dear?", "@Occupant in the vehicle Buy some mutton by the way, we'll have a barbecue tonight.", "I don't like mutton, so I won't come tonight." etc. belongs to the historical conversation records of the other party, and the content of the occupant in the vehicle such as "Almost there, I'm waiting for the traffic light." belongs to the historical conversation record of the local party.

[0074] Figure 3Among them, the part corresponding to the time point of 09:56 (current time) is generated by the current chat of all parties. The content of A and C such as "@Passengers in the car Honey, have you arrived at the park? It's a bit cold today. You can turn on the heater in the car." and "Putting on some music would also be quite nice." belongs to the current conversation records of the other party, and the content of the passengers in the car such as "I was just thinking the same thing." (voice message) belongs to the current conversation records of this party.

[0075] In step S5, the control module (in-vehicle system) can, according to Figure 3 the shown conversation information, generate control instructions for in-vehicle components. For example, the control module (in-vehicle system) can detect semantic content related to the control intention of in-vehicle components from the conversation information, generate control instructions to execute step S6 to control the corresponding in-vehicle components to perform functions, so as to meet the control intention of in-vehicle components expressed in the conversation information.

[0076] In this embodiment, when performing step S5, that is, generating control instructions for in-vehicle components according to the conversation information, the following steps can be specifically executed:

[0077] S501. According to the conversation information, obtain the current conversation records of the other party;

[0078] S502. Verify the current conversation records of the other party;

[0079] S503. When the verification passes, generate control instructions according to the current conversation records of the other party.

[0080] In step S501, referring to Figure 3 , obtain the current conversation records of the other party, that is, the chat records of A and C in the part corresponding to the time point of 09:56 (current time).

[0081] In step S502, verify the current conversation records of the other party obtained in step S501. For example, to ensure safety, filtering rules can be set. If the current conversation records of the other party involve intention content directly related to the driving actions of this vehicle (such as "You should step on the accelerator to catch up with him.", "There is no surveillance on that road, you can accelerate to your heart's content.", "Stop the car immediately for me.", etc.), then such current conversation records of the other party can be filtered out, that is, the verification of such current conversation records of the other party in step S502 fails; while if the current conversation records of the other party only express intentions related to the adjustment of the vehicle driving environment in this vehicle (such as hoping to use the audio-visual entertainment equipment, atmosphere lights, air conditioner, etc. in this vehicle), and do not involve intentions directly related to the driving actions of this vehicle, then such current conversation records of the other party can be included in the white list, that is, the verification of such current conversation records of the other party in step S502 passes.

[0082] In step S503, for the verified current conversation record of the other party, a control instruction is generated based on the current conversation record of the other party. For example, for Figure 3 the shown current conversation record of the other party, if the current conversation record of the other party where A said "@Dear passengers in the car, have you arrived at the park? It's a bit cold today. You can turn on the heater in the car" is verified, then when executing step S503, the semantic recognition algorithm can be used to perform semantic recognition on this current conversation record of the other party, extract its semantics, and determine that the intention expressed by this current conversation record of the other party is to control the air conditioner in this vehicle to generate heat. Therefore, the control module (in-vehicle system) can generate a control instruction for the air conditioner; if the current conversation record of the other party where B said "It's also quite suitable to play some music" is verified, then when executing step S503, the semantic recognition algorithm can be used to perform semantic recognition on this current conversation record of the other party, extract its semantics, and determine that the intention expressed by this current conversation record of the other party is to control the in-vehicle audio and video entertainment device to play music. Therefore, the control module (in-vehicle system) can generate a control instruction for the audio and video entertainment device. Correspondingly, if the current conversation record of the other party said by A is not verified, then the control module (in-vehicle system) does not generate a control instruction for the air conditioner; if the current conversation record of the other party said by B is not verified, then the control module (in-vehicle system) does not generate a control instruction for the audio and video entertainment device.

[0083] In this embodiment, by executing steps S501 - S503, it is possible to control the in-vehicle components of this vehicle to perform their functions according to the conversation records of the other party in the in-vehicle social application, thereby achieving the effect of remote control by the other party under the supervision of the passengers in this vehicle, enabling other participating parties of the social application to participate in the control of the in-vehicle components of this vehicle through the social application, improving the information communication ability of the social application, playing a role in shortening the distance between the parties in the conversation, expanding the functions of the social application in a new dimension, thereby increasing the usage coverage and flexibility of the social application, and also making the vehicle become a part of the social communication chain and giving full play to the usage value of the vehicle.

[0084] Moreover, by executing steps S501 - S503, when other people except the passengers in the vehicle remotely control the in-vehicle components of this vehicle, they can rely not on sending instructions specifically for controlling in-vehicle components such as "Please turn on the air conditioner for heating", but extract such instructions from the conversations between the passengers in the vehicle and other people using the social application. Therefore, the requirements for the expression ability of other people can be reduced and the user experience can be improved.

[0085] In this embodiment, when executing step S502, that is, the step of verifying the current conversation record of the other party, the following steps can be specifically executed:

[0086] S50201A. Obtain the current conversation record of the local party according to the conversation information;

[0087] S50202A. Obtain the first semantic information corresponding to the current conversation record of the local party and the second semantic information corresponding to the current conversation record of the other party;

[0088] S50203A. Obtain the matching degree between the first semantic information and the second semantic information;

[0089] S50204A. When the matching degree is greater than the matching degree threshold, determine that the verification passes.

[0090] Steps S50201A - S50204A are the first execution manner of step S502.

[0091] Taking Figure 3 the shown conversation information as an example, by executing step S50202A, the second semantic information corresponding to the current conversation record of the other party said by A can be obtained as "control the air conditioner to heat", the second semantic information corresponding to the current conversation record of the other party said by B can be obtained as "control the audio - visual entertainment device to play music", and the first semantic information corresponding to the current conversation record of the local party said by the person in the vehicle can be obtained as "control the air conditioner to heat".

[0092] In step S50203A, a semantic matching algorithm can be executed to calculate the matching degree between the first semantic information and the second semantic information. For example, the semantic distance between the first semantic information and the second semantic information can be calculated to obtain the matching degree. In this way, through S50203A - S50204A, it can be known that there is a high matching degree (greater than the matching degree threshold, which means that the person in the vehicle agrees with A's intention) between the first semantic information corresponding to the current conversation record of the local party said by the person in the vehicle and the second semantic information corresponding to the current conversation record of the other party said by A. Therefore, it is determined that the verification of the current conversation record said by A passes; while the matching degree between the first semantic information corresponding to the current conversation record of the local party said by the person in the vehicle and the second semantic information corresponding to the current conversation record of the other party said by C is low (less than the matching degree threshold, which means that the person in the vehicle does not agree or even opposes C's intention). Therefore, it is determined that the verification of the current conversation record said by C passes.

[0093] In this embodiment, by executing steps S50201A - S50204A to execute step S502, it is possible to decide whether to generate a control instruction according to the current conversation record of the other party based on the conversation information of the person in the vehicle using the social application, so as to reserve the control right of the vehicle's in - vehicle components for the person in the vehicle.

[0094] In this embodiment, when executing step S502, that is, the step of verifying the current conversation record of the other party, the following steps can be specifically executed:

[0095] S50201B. Obtain the historical conversation records of the local party and the historical conversation records of the other party according to the conversation information;

[0096] S50202B. Obtain the third semantic information corresponding to the historical conversation records of the local party and the fourth semantic information corresponding to the historical conversation records of the other party;

[0097] S50203B. Obtain the intimacy corresponding to the third semantic information and the fourth semantic information;

[0098] S50204B. When the intimacy is greater than the intimacy threshold, determine that the verification passes.

[0099] Steps S50201B - S50204B are the second execution method of step S502.

[0100] By executing step S50202B, the keywords indicating the conversation objects and the keywords indicating the emotional colors in the historical conversation records of the local party and the historical conversation records of the other party can be extracted as the third semantic information and the fourth semantic information. For example, taking the Figure 3 shown conversation information as an example, in the other party's historical conversation record said by A, "@person in the car Honey, have you arrived home?" contains the keyword "@person in the car" indicating the conversation object and the keyword "Honey" indicating the emotional color. A keyword list can be established, and scores are assigned to the intimacy levels of social relationships represented by each keyword according to daily language habits. For example, a keyword like "Honey" obviously represents a relatively high intimacy level and can be assigned 10 points, while a keyword like "Hello" represents a relatively low intimacy level and can be assigned 1 point. And the keyword indicating the conversation object can determine to which two conversation participants the score corresponding to the keyword indicating the emotional color belongs. For example, the keyword "@person in the car" in the other party's historical conversation record said by A indicates that the score (10 points) corresponding to its keyword "Honey" belongs to the two conversation participants A and the person in the car.

[0101] Based on the principle of the above step S50202B, when performing step S50203B, one of the counterparts in the conversation can be selected for the vehicle occupants first (for example, A), and then the third semantic information corresponding to the historical conversation record of the local party, as well as the fourth semantic information corresponding to the historical conversation record of the other party said by A, can be obtained. For the part in the third semantic information that points to A, keywords indicating emotional color are searched to obtain the corresponding scores, and the searched scores are marked as belonging to the two conversation participants, namely the vehicle occupants and A. Also, for the part in the fourth semantic information that points to the vehicle occupants, keywords indicating emotional color are searched to obtain the corresponding scores, and the searched scores are marked as belonging to the two conversation participants, namely the vehicle occupants and A. Finally, all the scores belonging to the two conversation participants, namely the vehicle occupants and A, are added up, and the total score obtained can be used as the intimacy between the third semantic information and the fourth semantic information of the conversation participant A. If the intimacy between the third semantic information and the fourth semantic information of the conversation participant A is greater than the intimacy threshold, then when performing step S50204B, it can be determined that the current conversation record of A's counterpart is verified.

[0102] According to the above principle, each conversation participant such as A, B, C, etc. can be traversed, and the intimacy between the third semantic information and the fourth semantic information of each conversation participant can be calculated respectively, and it can be judged whether the current conversation record of the counterpart of each participant is verified.

[0103] For example, taking Figure 3 the shown conversation information as an example, by performing steps S50201B - S50204B, it may be determined that the verification of the current conversation record of A's counterpart "@Vehicle occupants Honey, have you arrived at the park? It's a bit cold today. You can turn on the heater in the car" is passed, while the current conversation record of C's counterpart "Playing some music would also be appropriate" is not passed. Thus, when performing step S503, a control instruction for controlling the air conditioner to generate heat is generated according to the current conversation record of A's counterpart, and no control instruction is generated according to the current conversation record of C's counterpart. When performing step S6, the control instruction is executed to control the vehicle component of the air conditioner to generate heat, but the vehicle component of the audio - visual entertainment device is not controlled to play music.

[0104] In this embodiment, step S502 is executed by performing steps S50201B - S50204B. Whether to generate a control instruction based on the other party's current conversation record can be determined according to the intimacy between the local party's current conversation record and the other party's current conversation record. Since the local party's current conversation record and the other party's current conversation record represent the past chat records between the vehicle occupants and a conversation participant, the intimacy can represent the degree of trust of the conversation participant by the vehicle occupants. When the intimacy is greater than the intimacy threshold, it means that the degree of trust of the conversation participant is quite high. Therefore, partial control permissions for vehicle-mounted components can be opened to the conversation participant, so as to achieve an efficient response to the other party's current conversation record of the trusted conversation participant, and can timely respond to the other party's current conversation record when the vehicle occupants fail to reply to the other party's current conversation record in time using the social application, improving the flexibility of vehicle use.

[0105] In this embodiment, on the basis of executing steps S1 - S6, the following steps can also be executed:

[0106] S7. Obtain the control result of controlling the vehicle-mounted component by executing the control instruction;

[0107] S8. Generate a reply message according to the control result;

[0108] S9. Input the reply message as the speech of the session corresponding to the vehicle occupants' participation in the conversation information into the social application.

[0109] In step S7, the control result to be obtained can be the performance shown by the controlled vehicle-mounted component itself after executing the control instruction. For example, parameters such as the music track, version, and volume started to play by the audio-visual entertainment device under the control of the control instruction, and the real-time temperature reached in the vehicle cabin when the air conditioner performs heating under the control of the control instruction. It can also be the new state reached by the entire vehicle after the controlled vehicle-mounted component executes the control instruction (regardless of whether it is caused by the vehicle-mounted component executing the control instruction). For example, after the audio-visual entertainment device starts to play music under the control of the control instruction, the real-time position and speed of the vehicle (even if such position and speed are not caused by the audio-visual entertainment device playing music).

[0110] In step S8, the control module (in-vehicle system) can directly edit the parameters obtained in step S7 to obtain the reply message. For example, the content of the reply message can be "The currently playing music track is... The version is... The volume is...", or "The current temperature in the vehicle cabin is...", etc.

[0111] In step S9, the control module (in-vehicle system) inputs the reply message edited in step S8 as the speech of the session corresponding to the vehicle occupants' participation in the conversation information into the social application. For example, in Figure 3In the conversation shown, the control module (vehicle system) inputs the reply information into the social application, so that in the "fellow countryman chat group" conversation, the person on the car sends a message with the content "The currently playing music track is...the version is...the volume is...", etc., and the conversation participants A, B and C in the conversation can all receive the message sent by the person on the car.

[0112] By executing steps S7-S9, the control results can be fed back to the conversation participants who control the vehicle-mounted components through the social application. When it is inconvenient for the people in the car to operate the social software while driving the car, automatic replies can be made to the conversation participants, thereby improving the efficiency of information communication and the user experience, especially the user experience of the conversation participants who are having a conversation with the people in the car.

[0113] In this embodiment, when executing step S8, that is, the step of generating reply information according to the control result, the following steps may be specifically performed:

[0114] S801. During the driving process of the vehicle triggered by the control result, information collection is performed on the driving state of the vehicle and / or the external environment of the vehicle to obtain driving process information;

[0115] S802. Using the driving process information as a prompt word, input into the large language model;

[0116] S803. Obtain the output result of the large language model;

[0117] S804. Determine reply information according to the output result.

[0118] In step S801, the control module (vehicle system) can call the driving status sensor to detect and obtain driving status such as engine speed, vehicle driving speed and vehicle driving acceleration, and call the environmental information sensor to detect and obtain driving process information such as external environment images, external sounds and real-time position of the vehicle. The driving status and driving process information can be obtained by themselves or in combination to obtain driving process information.

[0119] In step S802, the control module (vehicle system) runs the large language model locally, or calls the large language model running on the cloud server through the communication module, and inputs the driving process information obtained by executing step S801 as a prompt word into the large language model. The large language model can respond to the prompt word through artificial intelligence technology or generate text, and its output result is content with richer semantic information related to the prompt word.

[0120] For example, when the driving process information obtained by executing step S801 is "driving speed 100 km / h", the output result obtained by executing step S803 may be "Honey, the air conditioner temperature you adjusted for me has made my car so enthusiastic that it's about to fly"; when the driving process information obtained by executing step S801 is a landscape image, the output result obtained by executing step S803 may be "Honey, listening to the song you selected for me has made the world in front of my eyes so beautiful."

[0121] In step S804, the control module (in-vehicle system) edits the output result obtained by executing step S803 into a reply message, and then executes step S9 to input it into the social application and send it to each session participant.

[0122] In this embodiment, by executing steps S801 - S804, the monotonous output result can be converted into a vivid reply message through the large language model, and the content of the reply message is related to the current driving process state of the vehicle occupants, so as to provide an immersive experience for the session participants who are having a conversation with the vehicle occupants, achieving the effect of improving the user experience.

[0123] A computer program for executing the vehicle-mounted control method in this embodiment can be written, and this computer program can be written into a computer device or a storage medium. When the computer program is read and run, the vehicle-mounted control method in this embodiment is executed, thereby achieving the same technical effect as the vehicle-mounted control method in the embodiment.

[0124] It should be noted that, unless otherwise specified, when a certain feature is referred to as "fixed" or "connected" to another feature, it can be directly fixed or connected to the other feature, or indirectly fixed or connected to the other feature. In addition, the up, down, left, right, etc. descriptions used in this disclosure are only relative to the mutual positional relationship of the various components of this disclosure in the drawings. The singular forms of "a", "an", and "the" used in this disclosure are also intended to include the plural forms, unless the context clearly indicates otherwise. In addition, unless otherwise defined, all the technical and scientific terms used in this embodiment have the same meaning as commonly understood by those skilled in the technical field of this application. The terms used in the description of this embodiment are only for describing specific embodiments, rather than for limiting the present invention. The term "and / or" used in this embodiment includes any combination of one or more of the related listed items.

[0125] It should be understood that although the terms first, second, third, etc. may be used in this disclosure to describe various elements, these elements should not be limited to these terms. These terms are only used to distinguish elements of the same type from each other. For example, without departing from the scope of this disclosure, the first element may also be referred to as the second element, and similarly, the second element may also be referred to as the first element. The use of any and all examples or exemplary language ("for example", "such as", etc.) provided in this embodiment is only intended to better illustrate the embodiments of the present invention and will not impose a limitation on the scope of the present invention unless otherwise required.

[0126] It should be recognized that embodiments of the present invention can be implemented or carried out by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer-readable memory. The methods can be implemented in a computer program using standard programming techniques - including a non-transitory computer-readable storage medium configured with a computer program, wherein the storage medium so configured causes the computer to operate in a specific and predefined manner - according to the methods and drawings described in the specific embodiments. Each program can be implemented in a high-level procedural or object-oriented programming language to communicate with a computer system. However, if desired, the program can be implemented in assembly or machine language. In any case, the language can be a compiled or interpreted language. In addition, for this purpose the program is capable of running on a programmed application-specific integrated circuit.

[0127] In addition, the operations of the processes described in this embodiment can be performed in any suitable order, unless this embodiment otherwise indicates or is otherwise clearly contradicted by the context. The processes described in this embodiment (or variations and / or combinations thereof) can be executed under the control of one or more computer systems configured with executable instructions, and can be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) executed commonly on one or more processors, by hardware, or a combination thereof. A computer program includes a plurality of instructions executable by one or more processors.

[0128] Further, the method can be implemented in any type of computing platform operatively connected to a suitable one, including but not limited to personal computers, minicomputers, mainframes, workstations, network or distributed computing environments, separate or integrated computer platforms, or communicating with charged particle tools or other imaging devices, etc. Aspects of the present invention can be implemented in machine-readable code stored on a non-transitory storage medium or device, whether removable or integrated into the computing platform, such as a hard disk, optical read and / or write storage medium, RAM, ROM, etc., such that it can be read by a programmable computer and can be used to configure and operate the computer to execute the processes described herein when the storage medium or device is read by the computer. In addition, the machine-readable code, or portions thereof, can be transmitted via a wired or wireless network. When such media includes instructions or programs that implement the above steps in conjunction with a microprocessor or other data processor, the invention of this embodiment includes these and other different types of non-transitory computer-readable storage media. When programmed according to the methods and techniques of the present invention, the present invention also includes the computer itself.

[0129] A computer program can be applied to input data to perform the functions of this embodiment, thereby transforming the input data to generate output data stored in non-volatile memory. The output information can also be applied to one or more output devices such as a display. In a preferred embodiment of the present invention, the transformed data represents physical and tangible objects, including a specific visual depiction of the physical and tangible objects generated on the display.

[0130] The above are only the preferred embodiments of the present invention. The present invention is not limited to the above embodiments. As long as it achieves the technical effects of the present invention by the same means, any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention. There can be various different modifications and variations to its technical solutions and / or implementation manners within the scope of protection of the present invention.

Claims

1. A vehicle control method, characterized in that: The vehicle control method comprises: Get the action information of the people in the car; Verifying the action information and verification information; When the action information matches the verification information, the social application is activated and executed.

2. The vehicle control method according to claim 1, characterized in that: The activating and running the social application includes: Activate and run the social application in the vehicle system; or Activating and running the social application on the mobile device of the person on the vehicle; Projecting the social application running result of the mobile device to the vehicle system.

3. The vehicle control method according to claim 1, characterized in that: The vehicle control method further includes: Acquire conversation information formed by the passengers on the vehicle using the social application; generating control instructions for vehicle-mounted components according to the dialogue information; The control instruction is executed to control the vehicle-mounted component.

4. The vehicle control method according to claim 3, characterized in that: Generating a control instruction for the vehicle-mounted component according to the dialogue information includes: According to the conversation information, the other party's current conversation record is obtained; the other party's current conversation record is a current conversation record formed with other participants who participate in the conversation through the social application with the person on the vehicle; Verifying the other party's current conversation record; When the verification is passed, the control instruction is generated according to the other party's current conversation record.

5. The vehicle control method according to claim 4, characterized in that: The verifying of the other party's current conversation record includes: According to the conversation information, the current conversation record of the party is obtained; the current conversation record of the party is the current conversation record formed by the person on the vehicle participating in the conversation through the social application; Acquire the first semantic information corresponding to the current conversation record of the party, and the second semantic information corresponding to the current conversation record of the other party; Obtaining a matching degree between the first semantic information and the second semantic information; When the matching degree is greater than the matching degree threshold, it is determined that the verification is passed.

6. The vehicle control method according to claim 5, characterized in that: The verifying of the other party's current conversation record includes: According to the conversation information, the historical conversation records of the party and the historical conversation records of the other party are obtained; the historical conversation records of the party are historical conversation records formed by the person on the bus participating in the conversation through the social application, and the historical conversation records of the other party are historical conversation records formed with other participants who participated in the conversation with the person on the bus through the social application; Acquire third semantic information corresponding to the historical conversation record of the party concerned, and fourth semantic information corresponding to the historical conversation record of the other party concerned; Acquire the intimacy between the third semantic information and the fourth semantic information; When the intimacy is greater than the intimacy threshold, it is determined that the verification is passed.

7. The vehicle control method according to any one of claims 3 to 6, characterized in that: The vehicle control method further includes: obtaining a control result of executing the control instruction to control the vehicle-mounted component; Generate reply information according to the control result; The reply information is input into the social application as the speech of the person on the vehicle participating in the conversation corresponding to the dialogue information.

8. The vehicle control method according to claim 7, characterized in that: Generating reply information according to the control result includes: During the driving process of the vehicle triggered by the control result, information collection is performed on the driving state of the vehicle and / or the external environment of the vehicle to obtain driving process information; Using the driving process information as prompt words and inputting them into a large language model; Obtaining an output result of the large language model; The reply information is determined according to the output result.

9. A computer device, characterized in that: It comprises a memory and a processor, the memory is used to store at least one program, and the processor is used to load at least one program to execute the vehicle control method described in any one of claims 1-8.

10. A computer-readable storage medium storing a program executable by a processor, characterized in that: The program executable by the processor is used to execute the vehicle control method described in any one of claims 1 to 8 when executed by the processor.

Images