Self-driving vehicle escape method, device and system, storage medium and equipment

Abstract

The invention discloses an out-of-trap method, device and system for an autonomous vehicle, a storage medium and equipment, and the method comprises the steps: obtaining a navigation map, recently collected multi-frame road environment images, recent navigation broadcast text information and a problem list when an out-of-trap trigger instruction is received, the question list comprises questions used for inquiring whether the vehicle executes a detrapping related behavior or not; the navigation map, the multiple frames of road environment images, the navigation broadcast text information and the problem list are sent to a cloud end, so that the cloud end processes the navigation map, the multiple frames of road environment images, the navigation broadcast text information and the problem list based on a large language model, and an escape planning track is obtained; and receiving the escape planning track issued by the cloud, and controlling the vehicle to escape according to the escape planning track. According to the scheme, the method can achieve the automatic escape of the vehicle based on the large language model, does not need manual driving, and further improves the reliability of automatic driving.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent driving, in particular to a method, device, system, storage medium and equipment for automatically driving a vehicle out of trouble. BACKGROUND

[0002] Unmanned driving, also known as automatic driving, refers to a technology that enables a vehicle to have environmental perception, path planning and autonomous vehicle control. It can improve driving safety, alleviate traffic congestion, improve driving efficiency, reduce traffic accidents and enhance travel experience, bringing more convenient, safe and efficient travel to humans and promoting innovation and development of human society.

[0003] However, in some special situations, such as continuous parking and waiting, continuous low-speed driving and other vehicle trapped scenarios, the driver often takes over and drives the vehicle out of trouble by manual driving. Therefore, how to enable the vehicle to automatically escape from trouble is one of the long-tail problems faced by automatic driving. SUMMARY

[0004] The present application provides a method, device, system, storage medium and equipment for automatically driving a vehicle out of trouble, which can automatically drive the vehicle out of trouble.

[0005] The specific technical solutions are as follows:

[0006] In a first aspect, the present application provides a method for automatically driving a vehicle out of trouble, which comprises:

[0007] When receiving a trouble-escaping trigger instruction, a navigation map, a plurality of recently collected road environment images, a most recent navigation broadcast text information and a problem list are obtained, wherein the problem list includes questions for asking whether the vehicle performs a trouble-escaping related behavior;

[0008] The navigation map, the plurality of road environment images, the navigation broadcast text information and the problem list are sent to a cloud end, so that the cloud end processes the navigation map, the plurality of road environment images, the navigation broadcast text information and the problem list based on a large language model to obtain a trouble-escaping planning trajectory;

[0009] The trouble-escaping planning trajectory issued by the cloud end is received, and the vehicle is controlled to escape from trouble according to the trouble-escaping planning trajectory.

[0010] It can be known from the above scheme that when the escape trigger instruction is received, the navigation map, the most recently collected multiple frames of road environment images, the most recent navigation broadcast text information, and the problem list can be automatically acquired, which are reported to the cloud, and the escape planning track automatically output by the cloud based on the large language model is received, so that automatic escape of the vehicle can be realized according to the escape planning track, without manual driving, thereby further improving the reliability of automatic driving.

[0011] In a possible implementation, the training method of the large language model comprises:

[0012] A training sample set is acquired, wherein the training sample set comprises training samples for multiple trapped scenarios, each of the training samples comprises a navigation map sample, a most recently collected multiple frames of road environment image sample, a most recent navigation broadcast text sample, a problem list sample, and an escape planning track true value;

[0013] The training sample set is trained by using a pre-trained initial large language model to obtain an escape planning track prediction value, wherein the initial large language model is a general large language model trained according to text data;

[0014] A current loss is determined according to a difference between the escape planning track true value and the escape planning track prediction value;

[0015] In a case where the current loss is greater than a loss threshold, the model parameters of the initial large language model are adjusted, and the training of the training sample set is continued until the current loss is less than or equal to the loss threshold, and a final large language model is obtained.

[0016] It can be known from the above scheme that the embodiments of the present application can further train the general large language model according to the training sample set comprising the navigation map sample, the most recently collected multiple frames of road environment image sample, the most recent navigation broadcast text sample, the problem list sample, and the escape planning track true value, so that the large language model can realize the automatic escape planning function.

[0017] In a possible implementation, the method further comprises:

[0018] In a case where the voice information is received, if the preset keyword is identified in the voice information, the escape trigger instruction is generated; or,

[0019] In a case where the key operation for requesting escape is detected, the escape trigger instruction is generated; or,

[0020] When it is detected that the vehicle is in a trapped scenario, the escape trigger instruction is generated at a preset frequency until the vehicle escapes from the predicament.

[0021] In a possible implementation, the question list further includes a question for inquiring whether the vehicle needs to be re-navigated, and the method further includes:

[0022] obtaining answers to the question list output by the large language model;

[0023] in a case where the answers to the question list include a need for re-navigation, re-navigating the vehicle and controlling the vehicle to travel along a re-navigated trajectory after the vehicle completes the escape;

[0024] in a case where the answers to the question list include no need for re-navigation, not re-navigating the vehicle after the vehicle completes the escape;

[0025] in a case where the answers to the question list represent no escape operation and the large language model does not output the escape planning trajectory, the vehicle maintains an original state.

[0026] It can be learned from the above solution that, by combining answers to a question list output by a large language model and an escape planning trajectory, the embodiments of the present application divide automatic escape into three scenarios, i.e., “escape + re-navigation”, “escape”, and “maintain an original state”, so as to not only improve the comprehensiveness and safety of automatic escape, but also immediately perform re-navigation after completing escape, thereby realizing seamless connection between escape and re-navigation and improving smoothness of automatic driving.

[0027] In a possible implementation, the method further includes:

[0028] performing message reminding of different contents for different output results of the large language model.

[0029] It can be learned from the above solution that, by giving different message reminders for different output results of a large language model, the embodiments of the present application can inform a driver or passenger in advance of a decision of an automatic driving system, thereby further improving a driving experience.

[0030] In a second aspect, the embodiments of the present application provide an escape method of an automatic driving vehicle, and the method includes:

[0031] obtaining a navigation map, a plurality of recently collected road environment images, a most recent navigation broadcast text information, and a question list sent by the vehicle when receiving an escape trigger instruction, wherein the question list includes a question for inquiring whether the vehicle needs to perform an escape-related behavior;

[0032] processing the navigation map, the plurality of road environment images, the navigation broadcast text information, and the question list based on a large language model to obtain the escape planning trajectory.

[0033] issuing the escape planning trajectory to the vehicle, so that the vehicle escapes according to the escape planning trajectory.

[0034] Through the above scheme, it can be known that the embodiment of the present application can first acquire the navigation map, the recently collected multiple frames of road environment images, the recently navigation broadcast text information, and the problem list sent by the vehicle when receiving the escape trigger instruction, and process these information based on the large language model to automatically output the escape planning trajectory, and then issue the escape planning trajectory to the vehicle, so that the vehicle can realize automatic escape according to the escape planning trajectory without manual driving, thereby further improving the reliability of automatic driving.

[0035] In a possible implementation, before processing the navigation map, the multiple frames of road environment images, the navigation broadcast text information, and the problem list based on the large language model to obtain the escape planning trajectory, the method further includes:

[0036] acquiring a training sample set, wherein the training sample set includes training samples for multiple trapped scenarios, each of the training samples includes a navigation map sample, a recently collected multiple frames of road environment image samples, a recently navigation broadcast text sample, a problem list sample, and an escape planning trajectory true value;

[0037] training the training sample set by using a pre-trained initial large language model to obtain an escape planning trajectory prediction value, wherein the initial large language model is a general large language model trained according to text data;

[0038] determining a current loss according to the difference between the escape planning trajectory true value and the escape planning trajectory prediction value;

[0039] in the case that the current loss is greater than a loss threshold, adjusting the model parameters of the initial large language model, and continuing to train the training sample set until the current loss is less than or equal to the loss threshold, and obtaining a final large language model.

[0040] Through the above scheme, it can be known that the embodiment of the present application can further train the general large language model based on the training sample set including the navigation map sample, the recently collected multiple frames of road environment image sample, the recently navigation broadcast text sample, the problem list sample, and the escape planning trajectory true value, so that the large language model can realize the automatic escape planning function.

[0041] In a possible implementation, the problem list further includes a question for inquiring whether the vehicle re-navigates;

[0042] When the large language model processes the navigation map, the multiple frames of road environment images, the navigation broadcast text information, and the question list, the large language model further outputs answers to the question list, and the answers to the question list are sent to the vehicle, so that the vehicle determines whether to re-navigate according to the answers to the question list after completing the escape.

[0043] According to the above scheme, the embodiments of the present application can not only realize automatic escape of the vehicle according to the escape planning track output by the cloud, but also output answers to the question list by the large language model, so that the vehicle determines whether to re-navigate, thereby realizing seamless connection of escape and re-navigation, and improving the smoothness and safety of automatic driving.

[0044] In a third aspect, the embodiments of the present application provide an escape device of an automatic driving vehicle, and the device comprises:

[0045] An acquisition unit is configured to acquire a navigation map, multiple frames of recently collected road environment images, and recent navigation broadcast text information, and a question list when receiving an escape trigger instruction, wherein the question list comprises questions for inquiring whether the vehicle performs an escape-related behavior;

[0046] A sending unit is configured to send the navigation map, the multiple frames of road environment images, the navigation broadcast text information, and the question list to a cloud, so that the cloud processes the navigation map, the multiple frames of road environment images, the navigation broadcast text information, and the question list based on a large language model, and obtains the escape planning track;

[0047] A receiving unit is configured to receive the escape planning track sent by the cloud;

[0048] An escape unit is configured to control the vehicle to escape according to the escape planning track.

[0049] In a possible implementation, a training method of the large language model comprises:

[0050] A training sample set is acquired, wherein the training sample set comprises training samples for multiple trapped scenarios, each of the training samples comprises a navigation map sample, multiple frames of recently collected road environment image samples, recent navigation broadcast text samples, question list samples, and an escape planning track true value;

[0051] The training sample set is trained by using a pre-trained initial large language model to obtain an escape planning track prediction value, wherein the initial large language model is a general large language model trained according to text data;

[0052] determine a current loss according to a difference between the escape planning trajectory ground truth and the escape planning trajectory prediction value;

[0053] In a case where the current loss is greater than a loss threshold, adjust model parameters of the initial large language model, and continue training on the training sample set until the current loss is less than or equal to the loss threshold, and obtain a final large language model.

[0054] In a possible implementation, the apparatus further includes:

[0055] The generation unit is configured to:

[0056] In a case where voice information is received, if a preset keyword is identified in the voice information, generate the escape trigger instruction; or

[0057] In a case where a key operation for requesting escape is detected, generate the escape trigger instruction; or

[0058] When it is detected that the vehicle is in a trapped scenario, generate the escape trigger instruction at a preset frequency until the vehicle escapes from the trapped scenario.

[0059] In a possible implementation, the question list further includes a question for inquiring whether the vehicle needs to be re-navigated;

[0060] The acquisition unit is further configured to acquire answers to the question list output by the large language model;

[0061] The apparatus further includes:

[0062] The navigation unit is configured to, in a case where the answers to the question list indicate that re-navigation is needed, re-navigate the vehicle after the vehicle escapes from the trapped scenario, and control the vehicle to travel according to a re-navigated trajectory; and in a case where the answers to the question list indicate that re-navigation is not needed, not re-navigate the vehicle after the vehicle escapes from the trapped scenario.

[0063] The maintenance unit is configured to, in a case where the answers to the question list indicate that no escape operation is needed, and the large language model does not output the escape planning trajectory, maintain the original state of the vehicle.

[0064] In a possible implementation, the apparatus further includes:

[0065] The output unit is configured to perform message reminding of different content for different output results of the large language model.

[0066] According to the above scheme, when the escape trigger instruction is received, the navigation map, the multiple frames of road environment images collected recently, the latest navigation broadcast text information, and the problem list can be automatically obtained, which are reported to the cloud, and the escape planning track automatically output by the cloud based on the large language model is received, so that the automatic escape of the vehicle can be realized according to the escape planning track, without manual driving, thereby further improving the reliability of automatic driving.

[0067] In a fourth aspect, the embodiments of the present application provide an escape device of an autonomous vehicle, the device comprising:

[0068] an obtaining unit, configured to obtain a navigation map, multiple frames of road environment images collected recently, latest navigation broadcast text information, and a problem list sent by the vehicle when receiving an escape trigger instruction, wherein the problem list comprises a question for inquiring whether the vehicle performs an escape-related behavior;

[0069] a processing unit, configured to process the navigation map, the multiple frames of road environment images, the navigation broadcast text information, and the problem list based on a large language model, to obtain the escape planning track;

[0070] a sending unit, configured to send the escape planning track to the vehicle, so that the vehicle escapes according to the escape planning track.

[0071] In a possible implementation, the obtaining unit is further configured to obtain a training sample set before processing the navigation map, the multiple frames of road environment images, the navigation broadcast text information, and the problem list based on the large language model to obtain the escape planning track, wherein the training sample set comprises training samples for multiple trapped scenarios, each of the training samples comprises a navigation map sample, multiple frames of road environment image samples collected recently, a latest navigation broadcast text sample, a problem list sample, and an escape planning track true value.

[0072] The device further comprises:

[0073] a training unit, configured to train the training sample set by using a pre-trained initial large language model to obtain an escape planning track prediction value, wherein the initial large language model is a general large language model trained according to text data;

[0074] a determining unit, configured to determine a current loss according to a difference between the escape planning track true value and the escape planning track prediction value.

[0075] An adjusting unit is configured to adjust model parameters of the initial large language model to continue training on the set of training samples until a current loss is less than or equal to a loss threshold, and obtain a final large language model.

[0076] In a possible implementation, the question list further includes a question for inquiring whether the vehicle re-navigates;

[0077] The processing unit is further configured to, when processing the navigation map, the multiple frames of road environment images, the navigation broadcast text information, and the question list based on the large language model, output answers to the question list by the large language model;

[0078] The sending unit is further configured to send the answers to the question list to the vehicle, so that the vehicle determines whether to re-navigate according to the answers to the question list after completing the escape.

[0079] According to the above scheme, the navigation map, the multiple frames of road environment images, the latest navigation broadcast text information, and the question list sent by the vehicle when receiving the escape trigger instruction can be acquired, and these information can be processed based on the large language model to automatically output the escape planning trajectory, and the escape planning trajectory is sent to the vehicle, so that the vehicle can automatically escape according to the escape planning trajectory without manual driving, thereby further improving the reliability of automatic driving.

[0080] In a fifth aspect, the embodiments of the present application provide an escape system of an automatic driving vehicle, which includes a vehicle and a cloud side.

[0081] The vehicle is configured to, when receiving an escape trigger instruction, acquire a navigation map, multiple frames of road environment images collected recently, latest navigation broadcast text information, and a question list, wherein the question list includes a question for inquiring whether the vehicle performs an escape-related behavior; and send the navigation map, the multiple frames of road environment images, the navigation broadcast text information, and the question list to the cloud side.

[0082] The cloud side is configured to receive the navigation map, the multiple frames of road environment images, the navigation broadcast text information, and the question list sent by the vehicle, and process the navigation map, the multiple frames of road environment images, the navigation broadcast text information, and the question list based on a large language model to obtain an escape planning trajectory, and send the escape planning trajectory to the vehicle.

[0083] The vehicle is configured to receive the escape planning trajectory sent by the cloud side, and escape according to the escape planning trajectory.

[0084] In a possible implementation, the cloud is further configured to obtain a training sample set before processing the navigation map, the plurality of frames of road environment images, the navigation broadcast text information, and the problem list based on the large language model to obtain the escape planning trajectory, wherein the training sample set includes training samples for a plurality of trapped scenarios, each of the training samples includes a navigation map sample, a plurality of frames of recently collected road environment image samples, a recently broadcast navigation text sample, a problem list sample, and an escape planning trajectory true value;

[0085] training the training sample set by using a pre-trained initial large language model to obtain an escape planning trajectory prediction value, wherein the initial large language model is a general large language model trained according to text data;

[0086] determining a current loss according to a difference between the escape planning trajectory true value and the escape planning trajectory prediction value;

[0087] In a case where the current loss is greater than a loss threshold, adjusting model parameters of the initial large language model, and continuing to train the training sample set until the current loss is less than or equal to the loss threshold, and obtaining a final large language model.

[0088] In a possible implementation, the vehicle is further configured to, in a case where the voice information is received, generate the escape trigger instruction if a preset keyword is identified in the voice information; or

[0089] generate the escape trigger instruction when a key operation for requesting escape is detected; or

[0090] generate the escape trigger instruction at a preset frequency when it is detected that the vehicle is in a trapped scenario, until the vehicle escapes from the trapped scenario.

[0091] In a possible implementation, the problem list further includes a question for inquiring whether the vehicle is to be re-navigated;

[0092] The cloud is further configured to output an answer to the problem list based on the large language model, and distribute the answer to the problem list to the vehicle.

[0093] The vehicle is further configured to receive answers to the question list issued by the cloud, and in a case where the answers to the question list include a need for re-navigation, re-navigate the vehicle after the vehicle completes the escape, and control the vehicle to travel along the re-navigated trajectory; in a case where the answers to the question list do not include a need for re-navigation, do not re-navigate the vehicle after the vehicle completes the escape; and in a case where the answers to the question list represent no escape operation and the large language model does not output the escape planning trajectory, maintain the original state of the vehicle.

[0094] In a possible implementation, the vehicle is further configured to perform message reminders of different contents for different output results of the large language model.

[0095] According to the above scheme, when the vehicle receives an escape triggering instruction, the vehicle obtains a navigation map, a plurality of recently collected road environment images, recently broadcasted text information, and a question list, and reports the information to the cloud. The cloud can process the information according to a large language model, automatically output an escape planning trajectory, and feed back to the vehicle, so that the vehicle can realize automatic escape according to the escape planning trajectory without manual driving, thereby further improving the reliability of automatic driving.

[0096] In a sixth aspect, an embodiment of the present application provides a computer readable storage medium having a computer program stored thereon, the program being executed by a processor to implement the method according to any possible implementation manner of the first aspect or to implement the method according to any possible implementation manner of the second aspect.

[0097] In a seventh aspect, an embodiment of the present application provides an electronic device, which includes:

[0098] one or more processors;

[0099] The processor is coupled with a storage device configured to store one or more programs;

[0100] When the one or more programs are executed by the one or more processors, the electronic device implements the method according to any possible implementation manner of the first aspect or implements the method according to any possible implementation manner of the second aspect.

[0101] In an eighth aspect, an embodiment of the present application provides a computer program product, which includes instructions, when the instructions are run on a computer or a processor, the computer or the processor executes the method according to any possible implementation manner of the first aspect or implements the method according to any possible implementation manner of the second aspect. BRIEF DESCRIPTION OF DRAWINGS

[0102] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the accompanying drawings in the following description only represent some of the embodiments of the present application. For those skilled in the art, other drawings can also be obtained without creative effort based on these drawings.

[0103] Figure 1 A flowchart of an automatic driving vehicle escape method provided by an embodiment of the present application is shown in the figure.

[0104] Figure 2 A flowchart of another automatic driving vehicle escape method provided by an embodiment of the present application is shown in the figure.

[0105] Figure 3 A composition block diagram of an automatic driving vehicle escape device provided by an embodiment of the present application is shown in the figure.

[0106] Figure 4 A composition block diagram of another automatic driving vehicle escape device provided by an embodiment of the present application is shown in the figure.

[0107] Figure 5 A structural diagram of an electronic device or computer device provided by an embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0108] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments only represent some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative effort fall within the scope of protection of the present application.

[0109] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict. The terms “include” and “have” and any variations thereof in the embodiments of the present application and the accompanying drawings are intended to cover non-exclusive inclusion. For example, the processes, methods, systems, products or devices including a series of steps or units are not limited to the listed steps or units, but can optionally include steps or units not listed or can optionally include other steps or units inherent to these processes, methods, products or devices.

[0110] Figure 1 A flowchart of an automatic driving vehicle escape method is shown in the figure. The method can be applied to an electronic device or computer device, and specifically can be applied to a terminal such as a vehicle. The method includes the following steps.

[0111] S110: When receiving the escape trigger instruction, acquire the navigation map, the recently collected multiple frames of road environment images, the latest navigation broadcast text information, and the problem list.

[0112] The escape trigger instruction can be generated in various ways, such as based on manual triggering, such as voice triggering, key triggering, etc., or automatically triggered by the vehicle.

[0113] The following three methods are introduced as follows:

[0114] (1) Voice triggering

[0115] In the case of receiving voice information, if the voice information includes a preset keyword, an escape trigger instruction is generated.

[0116] The preset keyword includes escape or words with similar meanings to escape. For example, the driver can issue a voice message "Xiao Yousheng, please escape" through the human-computer interaction device on the vehicle. After receiving the voice message, the vehicle can use voice recognition technology to obtain the text information in the voice and match the text information with the preset keyword. If the matching is successful, an escape trigger instruction is generated. If the matching is not successful, no escape trigger instruction is generated.

[0117] (2) Key triggering

[0118] In one possible implementation, a key for requesting escape can be set on the human-computer interaction interface. When detecting a key operation for requesting escape, an escape trigger instruction can be generated.

[0119] (3) Automatic triggering

[0120] The vehicle can detect whether it is in a trapped scene in real time or periodically. When detecting that the vehicle is in a trapped scene, an escape trigger instruction is generated at a preset frequency (such as 1s) until the vehicle escapes from the predicament.

[0121] When the duration of detecting that the vehicle is in a parked waiting state reaches a preset duration threshold, or the duration of detecting that the vehicle speed is less than or equal to a preset speed threshold reaches a preset duration threshold, it can be determined that the vehicle is in a trapped scene.

[0122] During vehicle driving, multiple cameras installed on the vehicle can collect road environment images around the vehicle in real time. During navigation based on the navigation map, the vehicle will also output navigation broadcast voice. When receiving the escape trigger instruction, the vehicle can acquire the navigation map, the recently collected multiple frames of road environment images, the latest navigation broadcast text information, and the problem list, so as to report these information to the cloud for escape planning.

[0123] The navigation map is an image format map generated according to a set of navigation track points. The plurality of frames of road environment images recently collected include a plurality of frames of road environment images collected at the current time and within a predetermined time range before the current time. The plurality of frames of road environment images can include a plurality of continuous frames of road environment images within the predetermined time range, or the total number of frames of road environment images collected within the predetermined time range exceeds a preset value. Each frame of road environment image can be a bird's eye view (BEV) obtained by fusing original images collected by a plurality of cameras based on a transformer model. The recent navigation broadcast text information includes text information corresponding to the last navigation voice broadcast, or text information corresponding to the last navigation voice broadcast before the current time. The question list includes questions for inquiring whether the vehicle performs a related behavior for escaping from a predicament, and can also include questions for inquiring whether the vehicle re-navigates. For example, the questions in the question list include "whether to continue to go forward", "whether to change lanes", "whether to reverse", "whether to re-navigate", "if navigation, please draw an escape planning route on the BEV map", and the like.

[0124] S120: sending the navigation map, the plurality of frames of road environment images, the navigation broadcast text information and the question list to the cloud end, so that the cloud end processes the navigation map, the plurality of frames of road environment images, the navigation broadcast text information and the question list based on a large language model to obtain an escape planning track.

[0125] After obtaining the navigation map, the plurality of frames of road environment images, the navigation broadcast text information and the question list, these information can be reported to the cloud end through a mobile communication module in the vehicle, such as a 5G communication module, so that the cloud end processes the navigation map, the plurality of frames of road environment images, the navigation broadcast text information and the question list based on a large language model to obtain an escape planning track.

[0126] The cloud end can pre-train a large language model (LLM), and the training method of the large language model can include:

[0127] obtain a final large language model. The loss threshold value can be set according to actual experience.

[0128] The initial large language model is a general large language model trained according to text data, which can generate natural language text or understand the meaning of language text, and complete text classification, question answering, dialogue and other functions. The embodiment of the present application further trains the general large language model by giving the vehicle escape scene required training samples on the basis of the general large language model, so that the final large language model can realize the function of escape planning.

[0129] In addition, the questions in the question list sample can be questions set according to all driving choices faced by the vehicle during the escape process, such as when it is necessary to choose to change lanes or not to change lanes, the question of whether to change lanes can be set.

[0130] The escape planning trajectory true value can be a trajectory generated when an expert driver manually drives the vehicle to escape. Both the escape planning trajectory true value and the escape planning prediction value can be drawn in the last frame of the multi-frame road environment image sample, so that the large language model can generate the escape planning trajectory, and the escape planning trajectory can be displayed in the form of an image to the driver for viewing.

[0131] It should be noted that the difference between the recently collected multi-frame road environment image sample and the multi-frame road environment image collected in step S110 is only that the former is used for training of the large language model, and the latter is used for application of the large language model, and other concepts are the same. Similarly, the difference between the recent navigation broadcast text sample and the recent navigation broadcast text in step S110 is also only that the former is used for training of the large language model, and the latter is used for application of the large language model, and other concepts are the same.

[0132] S130: receiving the escape planning trajectory issued by the cloud, and controlling the vehicle to escape according to the escape planning trajectory.

[0133] The escape planning track can be drawn in the last frame of the multiple frames of road environment images, and the vehicle can display the road environment image including the escape planning track for the driver or passenger to view.

[0134] In actual applications, the escape level can be divided into three types, namely, escape + re-navigation, escape + no re-navigation, and no escape + no re-navigation. The specific implementation modes of the three cases are described as follows:

[0135] The vehicle can obtain the answers to the question list output by the large language model. In the case that the answers to the question list include the need for re-navigation, the vehicle is re-navigated after completing the escape, and the vehicle is controlled to travel along the re-navigation track. In the case that the answers to the question list do not include the need for re-navigation, the vehicle is not re-navigated after completing the escape. In the case that the answers to the question list represent no escape operation, and the large language model does not output the escape planning track, the vehicle maintains the original state.

[0136] The embodiment of the present application divides the automatic escape into three scenarios, namely, “escape + re-navigation”, “escape”, and “maintain the original state”, by combining the answers to the question list output by the large language model and the escape planning track, thereby not only improving the comprehensiveness and safety of automatic escape, but also realizing seamless connection between escape and re-navigation, and improving the smoothness of automatic driving, when re-navigation is performed immediately after completing the escape.

[0137] In a possible implementation, in order to further improve the driving experience, the vehicle can also give a message reminder of different content for different output results of the large language model, in order to inform the decision of the automatic driving system in advance. For example, the message reminder content includes “please note that the vehicle will start to perform automatic escape operation, and the specific escape operation includes operation 1, operation 2, and the like”.

[0138] The escape method of the automatic driving vehicle provided by the embodiment of the present application can automatically obtain the navigation map, the multiple frames of recently collected road environment images, the latest navigation broadcast text information, and the question list when receiving the escape trigger instruction, report these contents to the cloud, and receive the escape planning track automatically output by the cloud based on the large language model, so as to realize automatic escape of the vehicle according to the escape planning track, without manual driving, thereby further improving the reliability of automatic driving.

[0139] Based on the above method embodiment, another embodiment of the present application provides an escape method of an automatic driving vehicle. The method can be applied to an electronic device or a computer device, and specifically can be applied to a cloud end, as shown in Figure 2 The method includes the following steps.

[0140] S210: Obtain the navigation map, the plurality of recently collected road environment images, the most recent navigation broadcast text information, and the question list sent by the vehicle when receiving the escape trigger instruction.

[0141] The navigation map is an image format map generated according to a set of navigation track points. The plurality of recently collected road environment images include a plurality of road environment images collected within a predetermined time range before the current time. The plurality of road environment images can include a plurality of continuous road environment images within the predetermined time range, or the total number of road environment images collected within the predetermined time range exceeds a preset value. Each road environment image can be a BEV image obtained by fusing a plurality of original images collected by a plurality of cameras based on a transformer model. The most recent navigation broadcast text information includes text information corresponding to the most recently broadcast navigation voice, or text information corresponding to the last broadcast navigation voice before the current time. The question list includes questions for inquiring whether the vehicle performs an escape-related behavior, and can also include questions for inquiring whether the vehicle re-navigates.

[0142] S220: Process the navigation map, the plurality of road environment images, the navigation broadcast text information, and the question list based on the large language model to obtain an escape planning track.

[0143] After obtaining the navigation map, the plurality of road environment images, the navigation broadcast text information, and the question list, the navigation map, the plurality of road environment images, the navigation broadcast text information, and the question list can be input into a pre-trained large language model for processing, and an escape planning track is output. The large language model can draw the escape planning track on the last road environment image of the plurality of road environment images.

[0144] The specific implementation method of training the large language model in the cloud can include:

[0145] Obtain a training sample set, wherein the training sample set includes training samples for a plurality of trapped scenarios, each training sample includes a navigation map sample, a plurality of recently collected road environment image samples, a most recent navigation broadcast text sample, a question list sample, and an escape planning track true value; train the training sample set using a pre-trained initial large language model to obtain an escape planning track prediction value, wherein the initial large language model is a general large language model trained from text data; determine a current loss according to the difference between the escape planning track true value and the escape planning track prediction value; adjust the model parameters of the initial large language model in the case that the current loss is greater than a loss threshold, continue to train the training sample set, and obtain a final large language model in the case that the current loss is less than or equal to the loss threshold.

[0146] The initial large language model is a general large language model trained according to text data, and the initial large language model can generate natural language text or understand the meaning of language text, and complete text classification, question answering, dialogue and other functions. Through further training of the general large language model by giving the general large language model the training samples required by the vehicle escape scene, the final large language model can realize the function of escape planning.

[0147] The question in the question list sample can be a question set according to all driving options faced by the vehicle during the escape process, such as a question of whether to change lanes when needing to choose to change lanes or not. The escape planning trajectory true value can be a trajectory generated by an expert driver manually driving the vehicle to escape. And the escape planning trajectory true value and the escape planning prediction value can be drawn in the last frame of the multi-frame road environment image sample, so that the large language model can generate the escape planning trajectory, and the escape planning trajectory can be displayed in the form of an image to the driver and passenger for viewing.

[0148] S230: issuing the escape planning trajectory to the vehicle to make the vehicle escape according to the escape planning trajectory.

[0149] When the large language model outputs the escape planning trajectory, the escape planning trajectory can be sent to the vehicle to make the vehicle escape according to the escape planning trajectory, and when the question list further includes a question for inquiring whether the vehicle re-navigates, the large language model can further output an answer to the question list to make the vehicle determine whether to re-navigate according to the answer to the question list after completing the escape.

[0150] The escape method of the autonomous vehicle provided in the embodiments of the present application can first acquire, by the cloud, the navigation map, the recently collected multi-frame road environment image, the recently broadcasted text information of the navigation, and the question list sent by the vehicle when receiving the escape trigger instruction, and process these information based on the large language model to automatically output the escape planning trajectory, and then issue the escape planning trajectory to the vehicle, so that the vehicle can realize automatic escape according to the escape planning trajectory without manual driving, thereby further improving the reliability of autonomous driving.

[0151] Based on the above method embodiments, another embodiment of the present application provides an escape device of an autonomous vehicle. The device can be applied to electronic equipment or computer equipment, and specifically can be applied to a terminal such as a vehicle, as shown in the accompanying drawings. Figure 3 As shown in the accompanying drawings, the device includes:

[0152] The acquisition unit 310 is configured to acquire a navigation map, a plurality of recently collected road environment images, a most recent navigation broadcast text information, and a question list when the escape trigger instruction is received, wherein the question list includes a question for inquiring whether the vehicle performs an escape-related behavior;

[0153] The sending unit 320 is configured to send the navigation map, the plurality of road environment images, the navigation broadcast text information, and the question list to the cloud end, so that the cloud end processes the navigation map, the plurality of road environment images, the navigation broadcast text information, and the question list based on a large language model to obtain the escape planning trajectory;

[0154] The receiving unit 330 is configured to receive the escape planning trajectory issued by the cloud end.

[0155] The escape unit 340 is configured to control the vehicle to escape according to the escape planning trajectory.

[0156] In a possible implementation, the training method of the large language model includes:

[0157] Acquiring a training sample set, wherein the training sample set includes training samples for a plurality of trapped scenarios, each of the training samples includes a navigation map sample, a plurality of recently collected road environment image samples, a most recent navigation broadcast text sample, a question list sample, and an escape planning trajectory true value;

[0158] Training the training sample set by using a pre-trained initial large language model to obtain an escape planning trajectory prediction value, wherein the initial large language model is a general large language model trained according to text data.

[0159] Determining a current loss according to a difference between the escape planning trajectory true value and the escape planning trajectory prediction value.

[0160] In a case where the current loss is greater than a loss threshold, adjusting model parameters of the initial large language model, and continuing to train the training sample set until the current loss is less than or equal to the loss threshold, and obtaining a final large language model.

[0161] In a possible implementation, the device further includes:

[0162] The generation unit is configured to:

[0163] In a case where the voice information is received, if a preset keyword is identified in the voice information, the escape trigger instruction is generated; or

[0164] generate the getting-out-of-trouble trigger instruction when detecting a key operation for requesting getting out of trouble; or

[0165] generate the getting-out-of-trouble trigger instruction at a preset frequency when detecting that the vehicle is in the trouble scene until the vehicle gets out of the trouble.

[0166] In a possible implementation, the question list further includes a question for inquiring whether the vehicle needs to be re-navigated;

[0167] The obtaining unit 310 is further configured to obtain answers to the question list output by the large language model;

[0168] The device further includes:

[0169] The navigation unit is configured to, in a case where the answers to the question list indicate that the vehicle needs to be re-navigated, re-navigate the vehicle after the vehicle gets out of the trouble and control the vehicle to travel along a re-navigated trajectory; and in a case where the answers to the question list indicate that the vehicle does not need to be re-navigated, not re-navigate the vehicle after the vehicle gets out of the trouble.

[0170] The maintaining unit is configured to, in a case where the answers to the question list indicate that the vehicle does not need to be re-navigated and the large language model does not output the getting-out-of-trouble trajectory, maintain the original state of the vehicle.

[0171] In a possible implementation, the device further includes:

[0172] The output unit is configured to output different content of message reminders for different output results of the large language model.

[0173] The getting-out-of-trouble device of the autonomous vehicle provided in the embodiments of the present application can automatically obtain a navigation map, a plurality of recently collected road environment images, and recently broadcasted navigation text information and a question list when receiving a getting-out-of-trouble trigger instruction, and report these contents to a cloud end and receive a getting-out-of-trouble trajectory automatically output by the cloud end based on a large language model, so that the vehicle can be automatically got out of trouble according to the getting-out-of-trouble trajectory without manual driving, thereby further improving the reliability of autonomous driving.

[0174] Based on the above method embodiments, another embodiment of the present application provides a getting-out-of-trouble device of an autonomous vehicle, which can be applied to an electronic device or a computer device, and specifically can be applied to a cloud end, as shown in Figure 4 The device includes:

[0175] The acquisition unit 410 is configured to acquire a navigation map, a plurality of recently collected road environment images, a most recent navigation broadcast text information, and a question list sent by the vehicle when receiving the escape trigger instruction, wherein the question list comprises a question for inquiring whether the vehicle performs an escape-related behavior;

[0176] The processing unit 420 is configured to process the navigation map, the plurality of road environment images, the navigation broadcast text information, and the question list based on a large language model to obtain the escape planning trajectory.

[0177] The sending unit 430 is configured to send the escape planning trajectory to the vehicle to control the vehicle to escape according to the escape planning trajectory.

[0178] In a possible implementation, the acquisition unit 410 is further configured to acquire a training sample set before processing the navigation map, the plurality of road environment images, the navigation broadcast text information, and the question list based on the large language model to obtain the escape planning trajectory, wherein the training sample set comprises training samples for a plurality of trapped scenarios, each of the training samples comprises a navigation map sample, a plurality of recently collected road environment image samples, a most recent navigation broadcast text sample, a question list sample, and an escape planning trajectory true value.

[0179] The device further comprises:

[0180] The training unit is configured to train the training sample set by using a pre-trained initial large language model to obtain an escape planning trajectory prediction value, wherein the initial large language model is a general large language model trained according to text data.

[0181] The determination unit is configured to determine a current loss according to a difference between the escape planning trajectory true value and the escape planning trajectory prediction value.

[0182] The adjustment unit is configured to adjust model parameters of the initial large language model in a case where the current loss is greater than a loss threshold, continue to train the training sample set, and obtain a final large language model in a case where the current loss is less than or equal to the loss threshold.

[0183] In a possible implementation, the question list further comprises a question for inquiring whether the vehicle re-navigates.

[0184] The processing unit 420 is further configured to output an answer to the question list based on the large language model when processing the navigation map, the plurality of road environment images, the navigation broadcast text information, and the question list.

[0185] The sending unit 430 is further configured to send the answers of the question list to the vehicle, so that the vehicle determines whether to be re-navigated according to the answers of the question list after completing the escape.

[0186] The escape device of the autonomous vehicle provided by the embodiments of the present application can obtain the navigation map, the recently collected multiple frames of road environment images, the recently broadcasted text information, and the question list sent by the vehicle when receiving the escape trigger instruction, and process these information based on a large language model to automatically output an escape planning trajectory, and then send the escape planning trajectory to the vehicle, so that the vehicle can realize automatic escape according to the escape planning trajectory without manual driving, thereby further improving the reliability of autonomous driving.

[0187] Based on the above embodiments, another embodiment of the present application provides an escape system of an autonomous vehicle, which comprises a vehicle and a cloud server.

[0188] The vehicle is configured to obtain a navigation map, recently collected multiple frames of road environment images, recently broadcasted text information, and a question list when receiving an escape trigger instruction, wherein the question list comprises questions for inquiring whether the vehicle performs escape-related behaviors; and send the navigation map, the multiple frames of road environment images, the broadcasted text information, and the question list to the cloud server.

[0189] The cloud server is configured to receive the navigation map, the multiple frames of road environment images, the broadcasted text information, and the question list sent by the vehicle, and process the navigation map, the multiple frames of road environment images, the broadcasted text information, and the question list based on a large language model to obtain an escape planning trajectory, and send the escape planning trajectory to the vehicle.

[0190] The vehicle is configured to receive the escape planning trajectory sent by the cloud server, and perform escape according to the escape planning trajectory.

[0191] In a possible implementation, the cloud server is further configured to obtain a training sample set before processing the navigation map, the multiple frames of road environment images, the broadcasted text information, and the question list based on a large language model to obtain the escape planning trajectory, wherein the training sample set comprises training samples for multiple escape scenarios, and each training sample comprises a navigation map sample, recently collected multiple frames of road environment image samples, recently broadcasted text samples, question list samples, and an escape planning trajectory true value.

[0192] training the initial large language model on the training sample set to obtain a stuck-escape planning trajectory prediction value, wherein the initial large language model is a general large language model trained according to text data;

[0193] determining a current loss according to a difference between the stuck-escape planning trajectory true value and the stuck-escape planning trajectory prediction value;

[0194] adjusting a model parameter of the initial large language model to continue training the training sample set until the current loss is less than or equal to the loss threshold, and obtaining a final large language model.

[0195] In a possible implementation, the vehicle is further configured to, in a case where the voice information is received, generate the stuck-escape trigger instruction if a preset keyword is identified in the voice information; or,

[0196] generate the stuck-escape trigger instruction when a key operation for requesting stuck escape is detected; or,

[0197] generate the stuck-escape trigger instruction at a preset frequency when it is detected that the vehicle is in a stuck scene, until the vehicle is out of the stuck scene.

[0198] In a possible implementation, the question list further includes a question for inquiring whether the vehicle needs to be re-navigated.

[0199] The cloud server is further configured to output answers to the questions in the question list based on the large language model, and send the answers to the questions in the question list to the vehicle.

[0200] The vehicle is further configured to receive the answers to the questions in the question list sent by the cloud server, re-navigate the vehicle and control the vehicle to travel according to a re-navigated trajectory in a case where the answers to the questions in the question list indicate that the vehicle needs to be re-navigated, and not re-navigate the vehicle in a case where the answers to the questions in the question list indicate that the vehicle does not need to be re-navigated; and in a case where the answers to the questions in the question list indicate that no stuck-escape operation is to be performed and the large language model does not output the stuck-escape planning trajectory, the vehicle maintains an original state.

[0201] In a possible implementation, the vehicle is further configured to perform message reminding of different contents for different output results of the large language model.

[0202] The escape system of the autonomous vehicle provided in the embodiments of the present application can obtain a navigation map, a plurality of recently collected road environment images, the most recent navigation broadcast text information, and a problem list when the vehicle receives an escape trigger instruction, and report the information to the cloud. The cloud can process the information according to a large language model, automatically output an escape planning track, and feed back to the vehicle, so that the vehicle can realize automatic escape according to the escape planning track without manual driving, thereby further improving the reliability of autonomous driving.

[0203] Based on the above method embodiments, another embodiment of the present application provides a computer readable storage medium having a computer program stored thereon, the program being executed by a processor to implement the method according to any one of the above embodiments.

[0204] Based on the above method embodiments, another embodiment of the present application provides an electronic device or a computer device, as shown in the accompanying drawings, comprising: Figure 5

[0205] one or more processors 510;

[0206] The processor 510 is coupled to a storage device 520 for storing one or more programs.

[0207] When the one or more programs are executed by the one or more processors 510, the electronic device or computer device implements the method according to any one of the above embodiments.

[0208] Based on the above embodiments, another embodiment of the present application provides a computer program product, which contains instructions. When the instructions are run on a computer or a processor, the computer or the processor executes the method according to any one of the above embodiments.

[0209] The above device embodiments and system embodiments correspond to the method embodiments and have the same technical effects. For specific descriptions, refer to the method embodiments. The device embodiments are based on the method embodiments, and specific descriptions can be referred to the method embodiments, which will not be repeated here. Those skilled in the art can understand that the accompanying drawings are only schematic diagrams of an embodiment, and the modules or processes in the accompanying drawings are not necessarily necessary for implementing the present application.

[0210] Those skilled in the art can understand that the modules in the device in the embodiments can be distributed in the device in the embodiments according to the embodiment description, or can be changed and located in one or more devices different from the present embodiment. The modules in the above embodiments can be combined into one module, or can be further split into a plurality of sub-modules.

[0211] ​It should be noted that the above examples are only used to illustrate the technical solutions of the present application, and are not intended to limit the same. Although the present application has been described in detail with reference to the foregoing examples, it will be understood by those of ordinary skill in the art that modifications can be made to the technical solutions described in the foregoing examples, or some of the technical features thereof can be replaced by equivalent technical features. Such modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for escaping from a trouble of an autonomous vehicle, characterized by, The method comprises: When receiving a getting-out-of-trouble trigger instruction, acquiring a navigation map, a plurality of recently collected road environment images, a most recent navigation broadcast text information, and a question list, wherein the question list comprises questions for inquiring whether the vehicle performs a getting-out-of-trouble related behavior; sending the navigation map, the plurality of road environment images, the navigation broadcast text information, and the question list to the cloud end, so that the cloud end processes the navigation map, the plurality of road environment images, the navigation broadcast text information, and the question list based on a large language model to obtain the getting-out-of-trouble planning trajectory; receiving the getting-out-of-trouble planning trajectory issued by the cloud end, and controlling the vehicle to get out of trouble according to the getting-out-of-trouble planning trajectory.

2. The method of claim 1, wherein, The training method of the large language model comprises: acquiring a training sample set, wherein the training sample set comprises training samples for a plurality of trapped scenarios, each of the training samples comprises a navigation map sample, a plurality of recently collected road environment image samples, a most recent navigation broadcast text sample, a question list sample, and a getting-out-of-trouble planning trajectory true value; training the training sample set by using a pre-trained initial large language model to obtain a getting-out-of-trouble planning trajectory prediction value, wherein the initial large language model is a general large language model trained according to text data; determining a current loss according to the difference between the getting-out-of-trouble planning trajectory true value and the getting-out-of-trouble planning trajectory prediction value; in the case that the current loss is greater than a loss threshold, adjusting the model parameters of the initial large language model, and continuing to train the training sample set until the current loss is less than or equal to the loss threshold, thereby obtaining a final large language model.

3. The method of claim 1, wherein, The method further comprises: in the case of receiving voice information, if a preset keyword is identified in the voice information, generating the getting-out-of-trouble trigger instruction; or in the case of detecting a key operation for requesting getting out of trouble, generating the getting-out-of-trouble trigger instruction; or when detecting that the vehicle is in a trapped scenario, generating the getting-out-of-trouble trigger instruction at a preset frequency until the vehicle gets out of trouble.

4. The method according to any one of claims 1 to 3, characterized in that, The question list further comprises a question for inquiring whether the vehicle re-navigates, and the method further comprises: acquiring an answer of the question list output by the large language model; in the case that the answer of the question list comprises a need for re-navigation, re-navigating the vehicle after the vehicle completes getting out of trouble, and controlling the vehicle to travel according to a re-navigation trajectory; in the case that the answer of the question list comprises no need for re-navigation, not re-navigating the vehicle after the vehicle completes getting out of trouble; in the case that the answer of the question list represents no getting-out-of-trouble operation, and the large language model does not output the getting-out-of-trouble planning trajectory, the vehicle maintains the original state.

5. The method of claim 4, wherein, The method further comprises: performing message reminders of different contents for different output results of the large language model.

6. A method for escaping from a trouble of an autonomous vehicle, characterized by, The method comprises: The navigation map, the multiple frames of road environment images collected recently, the latest navigation broadcast text information, and a question list sent by the vehicle when receiving the escape trigger instruction are acquired, wherein the question list includes questions for inquiring whether the vehicle performs an escape-related behavior; The navigation map, the multiple frames of road environment images, the latest navigation broadcast text information, and the question list are processed based on a large language model to obtain the escape planning track; The escape planning track is sent to the vehicle to enable the vehicle to escape according to the escape planning track.

7. The method of claim 6, wherein, Before the navigation map, the multiple frames of road environment images, the latest navigation broadcast text information, and the question list are processed based on the large language model to obtain the escape planning track, the method further includes: A training sample set is acquired, wherein the training sample set includes training samples for multiple trapped scenarios, each training sample includes a navigation map sample, multiple frames of road environment image samples collected recently, a latest navigation broadcast text sample, a question list sample, and an escape planning track true value; An initial large language model is trained based on the training sample set to obtain an escape planning track prediction value, wherein the initial large language model is a general large language model trained according to text data; A current loss is determined according to a difference between the escape planning track true value and the escape planning track prediction value; In a case where the current loss is greater than a loss threshold, the model parameters of the initial large language model are adjusted, and the training of the training sample set is continued until the current loss is less than or equal to the loss threshold, and a final large language model is obtained.

8. A self-driving vehicle's escape device, characterized by, The device includes: An acquisition unit is configured to acquire a navigation map, multiple frames of road environment images collected recently, latest navigation broadcast text information, and a question list when receiving an escape trigger instruction, wherein the question list includes questions for inquiring whether a vehicle performs an escape-related behavior; A sending unit is configured to send the navigation map, the multiple frames of road environment images, the latest navigation broadcast text information, and the question list to a cloud end to enable the cloud end to process the navigation map, the multiple frames of road environment images, the latest navigation broadcast text information, and the question list based on a large language model to obtain the escape planning track; A receiving unit is configured to receive the escape planning track sent by the cloud end; An escape unit is configured to control the vehicle to escape according to the escape planning track.

9. A self-driving vehicle's escape device, characterized by, The device includes: An acquisition unit is configured to acquire a navigation map, multiple frames of road environment images collected recently, latest navigation broadcast text information, and a question list sent by a vehicle when receiving an escape trigger instruction, wherein the question list includes questions for inquiring whether the vehicle performs an escape-related behavior; A processing unit is configured to process the navigation map, the multiple frames of road environment images, the latest navigation broadcast text information, and the question list based on a large language model to obtain the escape planning track; A sending unit is configured to send the escape planning trajectory to the vehicle, so that the vehicle performs escape according to the escape planning trajectory.

10. A self-driving vehicle's escape system, characterized by, The system comprises a vehicle and a cloud; The vehicle is configured to, when receiving an escape trigger instruction, acquire a navigation map, a plurality of recently-acquired road environment images, recently-acquired navigation broadcast text information, and a question list, wherein the question list comprises questions for inquiring whether the vehicle performs escape-related behaviors; and send the navigation map, the plurality of road environment images, the navigation broadcast text information, and the question list to the cloud; The cloud is configured to receive the navigation map, the plurality of road environment images, the navigation broadcast text information, and the question list sent by the vehicle, process the navigation map, the plurality of road environment images, the navigation broadcast text information, and the question list based on a large language model, obtain the escape planning trajectory, and send the escape planning trajectory to the vehicle; The vehicle is configured to receive the escape planning trajectory sent by the cloud, and perform escape according to the escape planning trajectory.

11. A computer readable storage medium having stored thereon a computer program, characterized in that, The program is executed by a processor to implement the method of any one of claims 1-5, or to implement the method of any one of claims 6-7.

12. An electronic device, comprising: The electronic device comprises: one or more processors; The processor is coupled with a storage device, and the storage device is configured to store one or more programs; When the one or more programs are executed by the one or more processors, the electronic device implements the method of any one of claims 1-5, or implements the method of any one of claims 6-7.

Images