Traffic signal identification method, storage medium, product, equipment and vehicle

Abstract

The invention provides a traffic signal identification method, a storage medium, a product, equipment and a vehicle. The method comprises the following steps: based on a first image, outputting first traffic information; wherein the first traffic information is obtained by processing the first image by using a first model and a second model; the first model is used for detecting a traffic signal in the first image to obtain a detection result; the second model is used for generating first traffic information based on the first image; the second model is determined based on the detection result. And performing corresponding processing on the first image by combining the characteristics of different types of models so as to generate first traffic information. Thus, the performance and efficiency of traffic signal recognition can be improved, traffic knowledge of a user can be efficiently and accurately guided, and the driving experience of the user is improved.

Description

Technical Field

[0001] This application relates to the field of vehicle driving, and in particular to traffic signal recognition methods, storage media, products, equipment and vehicles. Background Technology

[0002] Intelligent cockpits represent a significant development direction in modern automotive technology. Through advanced interactive systems, information display technologies, and driver assistance functions, they can greatly enhance the driving experience and road safety. Currently, vehicles can display various navigation information on screens to guide users. However, there is a lack of solutions for guiding users with traffic knowledge, resulting in a poor user experience. Summary of the Invention

[0003] This application provides a traffic signal recognition method, storage medium, product, device, and vehicle, which can enhance guidance on traffic knowledge for users and improve user experience, thereby at least partially solving the aforementioned technical problems.

[0004] To achieve the above objectives, according to a first aspect of this application, a traffic signal recognition method is provided, the method being applied to a vehicle, the method comprising: outputting first traffic information based on a first image; wherein the first traffic information is obtained by processing the first image using a first model and a second model; the first model is used to detect traffic signals in the first image to obtain a detection result; the second model is used to generate the first traffic information based on the first image; the second model is determined based on the detection result.

[0005] Optionally, the first image may be captured by an onboard camera in the vehicle.

[0006] Optionally, the method further includes: receiving a first traffic question input by a user. Outputting the first traffic information includes: responding to the first traffic question by outputting the first traffic information.

[0007] Optionally, the first traffic information includes a response to the first traffic question.

[0008] Optionally, the answer to the first traffic question includes: the answer to the traffic signal-related question, and / or driving advice based on the traffic signal.

[0009] Optionally, the method further includes:

[0010] Output second traffic information, which includes at least one of the following: traffic knowledge related to the first traffic information, a second traffic question related to the first traffic question, or an answer to the second traffic question.

[0011] In other words, in addition to providing detailed interactive Q&A and driving suggestions, the in-vehicle cockpit system can also support Q&A recommendation functions to expand / explore the user's depth of traffic knowledge.

[0012] In some scenarios, user questions trigger the vehicle to recommend relevant traffic-related questions and answers. For example, if a user asks what signs are ahead, the first and second models work together to determine if there are speed limit signs ahead. The recommended relevant traffic question could be: What is the speed limit on a typical road?

[0013] In other scenarios, vehicles automatically recommend relevant traffic information. For example, instead of relying on user queries, vehicles can automatically capture images from their cameras, including road signs ahead. Through vehicle-to-cloud collaboration, the cloud determines that the road ahead has speed limit signs. Here, the recommended traffic information could be: What is the speed limit on a typical road?

[0014] Optionally, the second traffic question is displayed to the user, and after the customer clicks on the recommended question, the vehicle can display the answer to the second traffic question through the central control screen.

[0015] Optionally, the method further includes:

[0016] Send the first image to the server;

[0017] Receive first traffic information obtained from the server by processing the first image.

[0018] Optionally, the method further includes:

[0019] Receive the second traffic information from the server.

[0020] Optionally, the first model includes an image classification model or an object detection model, and the second model includes a visual language model.

[0021] Optionally, the first traffic information may be output, including by voice output or output via a display screen.

[0022] Secondly, a traffic signal recognition method is provided, the method being applied to a server, the method comprising: receiving a first image from a vehicle; sending first traffic information to the vehicle, the first traffic information being obtained by processing the first image using a first model and a second model, the first model being used to detect traffic signals in the first image to obtain a detection result, the second model being used to generate the first traffic information based on the first image; the second model being determined based on the detection result.

[0023] In this way, the vehicle is responsible for the initial data collection and processing, while complex calculations rely on the powerful computing power of the cloud to carry out model reasoning, ensuring high accuracy of recognition, and returning the reasoning results to the vehicle to be displayed to the user.

[0024] Optionally, the first traffic information includes an answer to a first traffic question posed to the user;

[0025] The answer to the first traffic question includes: the answer to the traffic signal-related question, and / or driving advice based on the traffic signal.

[0026] Optionally, it also includes:

[0027] Generate second traffic information;

[0028] The second traffic information includes at least one of the following: traffic knowledge related to the first traffic information, a second traffic problem related to the first traffic problem, or an answer to the second traffic problem;

[0029] Send the second traffic information to the vehicle.

[0030] Optionally, there may be multiple first models, each of which is used to detect one or more traffic signals.

[0031] Optionally, the first model includes a first sub-model and a second sub-model; wherein the first sub-model is a base model, and the second sub-model is a fine-tuned model based on the first sub-model; the first sub-model is used to detect a first type of traffic signal; and the second sub-model is used to detect a second type of traffic signal.

[0032] Optionally, the second model is determined from multiple sub-models based on the detection results; each sub-model is used to generate traffic information based on the first image and the first traffic problem.

[0033] Optionally, the plurality of sub-models includes a third sub-model and a fourth sub-model; wherein the third sub-model is the base model, and the fourth sub-model is a fine-tuned model based on the third sub-model.

[0034] Optionally, it also includes:

[0035] When a traffic signal is detected, the traffic information generated by the fourth sub-model is used as the first traffic information.

[0036] Optionally, it also includes:

[0037] In the absence of detected traffic signals, the traffic information generated by the third sub-model is used as the first traffic information.

[0038] Optionally, the second traffic information is processed and output by a third model that generates a Retrieval Enhancement Group (RAG). The input of the third model includes a first retrieval result, which is a retrieval result obtained by retrieving a first traffic question from a first corpus.

[0039] Optionally, the retrieval algorithm used for searching in the first corpus may differ depending on the user scenario; the user scenario is related to the response time and / or response frequency required by the user for the retrieval.

[0040] Optionally, a sorting algorithm can be used to sort the search results after a fast response.

[0041] Thirdly, a traffic signal recognition method is provided, which can be applied to a vehicle or a component of a vehicle (such as a chip system). The method may include:

[0042] The first traffic question is received from the user.

[0043] Output second traffic information; the second traffic information includes at least one of the following: traffic knowledge related to the first traffic information, a second traffic problem related to the first traffic problem, or an answer to the second traffic problem; the first traffic information includes an answer to the first traffic problem.

[0044] According to a fourth aspect of this application, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements any design of the traffic signal recognition method described above.

[0045] According to a fifth aspect of this application, a computer program product is provided, comprising a computer program that, when executed by a processor, implements any design of the traffic signal recognition method described above.

[0046] According to a fifth aspect of this application, an electronic device is provided, comprising: a memory having a computer program stored thereon; and a processor for executing the computer program in the memory to implement any design of the traffic signal recognition method of any aspect described above.

[0047] According to a sixth aspect of this application, a server is provided, comprising: a memory storing a computer program thereon; and a processor for executing the computer program in the memory to implement any design of the traffic signal recognition method of any aspect described above.

[0048] According to a seventh aspect of this application, a vehicle is provided that includes the traffic signal recognition device described in any of the above embodiments, or includes the above-described electronic device.

[0049] In summary, the traffic signal recognition method, storage medium, product, device, and vehicle of this application embodiment can combine the characteristics of different types of models to process the first image accordingly, thereby generating first traffic information. Specifically, the first model can quickly detect traffic signals in the first image, and the second model can generate the first traffic information based on these traffic signals, improving the performance and efficiency of traffic signal recognition. This, in turn, enables efficient and accurate guidance of users' traffic knowledge, enhancing their driving experience.

[0050] Other features and advantages of this application will be described in detail in the following detailed description section. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] To gain a more complete understanding of this application and its beneficial effects, the following description will be provided in conjunction with the accompanying drawings, wherein the same reference numerals in the following description denote the same parts.

[0053] Figure 1 This is a schematic diagram of the structure of an electronic device provided in an exemplary embodiment of this disclosure. Figure 1 ;

[0054] Figure 2 This is a schematic diagram of the structure of an electronic device provided in an exemplary embodiment of this disclosure. Figure 2 ;

[0055] Figure 3 This is a schematic diagram of the system architecture involved in a traffic signal recognition method provided in an exemplary embodiment of this disclosure;

[0056] Figure 4 This is a scenario illustration of a traffic signal recognition method provided in an exemplary embodiment of this disclosure. Figure 1 ;

[0057] Figure 5 This is a schematic flowchart of a traffic signal recognition method provided in an exemplary embodiment of this disclosure;

[0058] Figure 6 This is a scenario illustration of a traffic signal recognition method provided in an exemplary embodiment of this disclosure. Figure 2 ;

[0059] Figure 7This is a scenario illustration of a traffic signal recognition method provided in an exemplary embodiment of this disclosure. Figure 3 ;

[0060] Figure 8 This is a scenario illustration of a traffic signal recognition method provided in an exemplary embodiment of this disclosure. Figure 4 ;

[0061] Figure 9 This is a scenario illustration of a traffic signal recognition method provided in an exemplary embodiment of this disclosure. Figure 5 ;

[0062] Figure 10 This is a schematic diagram of the architecture of an electronic device provided in an exemplary embodiment of this disclosure. Figure 3 ;

[0063] Figure 11 This is a schematic diagram of the chip system architecture provided in an exemplary embodiment of this disclosure. Detailed Implementation

[0064] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the protection scope of this application.

[0065] This application provides a traffic signal recognition method applicable to a vehicle, which includes an onboard camera. The method includes: acquiring a first image through the onboard camera; outputting first traffic information; the first traffic information being obtained by processing the first image using a first model and a second model; the first model being used to detect traffic signals in the first image; and the second model being used to generate the first traffic information based on the detected traffic signals. For example, the first traffic information may be a driving suggestion based on the traffic signal. For instance, if the vehicle detects a straight-ahead traffic signal, it can provide the user with the following driving suggestion: "Straight-ahead traffic signal ahead, please proceed straight."

[0066] This method combines the characteristics of different types of models to process the first image accordingly, thereby generating first traffic information. Specifically, the first model can quickly detect traffic signals in the first image, and the second model can generate the first traffic information based on these traffic signals, improving the performance and efficiency of traffic signal recognition.

[0067] In this embodiment, the first model can also be referred to as the small model, and the second model can also be referred to as the large model. By synergistically fusing the large and small models to comprehensively identify traffic signals, the efficiency and performance of the identification process can be improved.

[0068] The large model has more parameters than the small model. For example, a large model may have millions of parameters, while a small model may have several trillion. The specific parameter size of the model is not limited in the embodiments of this application.

[0069] For example, a small model can quickly identify a road sign ahead and pass the information of the identified road sign to a large model. Relying on the processing performance of the large model, the specific content of the road sign can be accurately identified, such as identifying the road sign as a straight-ahead sign.

[0070] This method can also be applied to other types of electronic devices, systems containing electronic devices, or chips within electronic devices. For example, it can be applied to chips in vehicles. This application does not limit the scope of the application.

[0071] For example, Figure 1 A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown. For example, the electronic device may be included in a vehicle.

[0072] like Figure 1 As shown, the electronic device 100 may include a processor 110. Optionally, the electronic device 100 may also include a memory 120 and a display screen 130, etc.

[0073] Processor 110 may include one or more processing units, such as application processors (APs), modem processors, graphics processing units (GPUs), image signal processors (ISPs), controllers, video codecs, digital signal processors (DSPs), baseband processors, and / or neural network processing units (NPUs). These different processing units may be independent devices or integrated into one or more processors.

[0074] The controller can generate operation control signals based on the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0075] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from this memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0076] In some embodiments, processor 110 may include one or more interfaces. These one or more interfaces can be used to connect processor 110 to memory 120, display 130, etc.

[0077] In some embodiments of this application, the processor 110 can be used to run a model, thereby generating first traffic information. Please refer to the following text for details.

[0078] The memory 120 can be used to store computer executable program code, which includes instructions. The memory 120 may include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as image playback function), etc. The data storage area may store data created during the use of the electronic device 100. The processor 110 executes various functional applications and data processing of the electronic device 100 by running instructions stored in the memory 120 and / or instructions stored in memory located in the processor.

[0079] Electronic device 100 implements display functions through a GPU, display screen 130, and application processor. The GPU is a microprocessor for image processing, connected to the display screen 130 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0080] The display screen 130 is used to display images, videos, etc. The display screen 130 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a miniature LED, a microLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include one or N display screens 130, where N is a positive integer greater than 1.

[0081] In some embodiments of this application, the display screen 130 can be used to display various interfaces. For example, it can display driving suggestions generated collaboratively by the first model and the second model.

[0082] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device. In other embodiments of this application, the electronic device may include... Figure 1 The diagram shows more or fewer components, or combinations of components, or separate components, or different arrangements of components. The components shown can be implemented in hardware, software, or a combination of both.

[0083] like Figure 2 This application illustrates one possible structure of a vehicle according to an embodiment of the present application, which includes the device / electronic equipment described in any of the embodiments herein. The vehicle possesses all the beneficial effects of the end-side electronic equipment, etc., as described in this embodiment. In other embodiments, the vehicle may also have some or all of the cloud-based functions / effects; for example, models may also be deployed within the vehicle. Further details are omitted here.

[0084] For example, the vehicle may be a plug-in hybrid electric vehicle or a new energy vehicle, etc., and this application does not specifically limit it.

[0085] This method can also be applied in edge-cloud collaborative scenarios. The first and second models can be deployed on the vehicle and the cloud, respectively. Alternatively, both the first and second models can be deployed on the cloud. The vehicle is responsible for data collection and transmitting the collected data to the cloud, where the cloud performs calculations using the models to generate the first traffic information.

[0086] Figure 3This illustrates a system architecture for edge-cloud collaboration. The edge device (e.g., a vehicle) and the cloud device (e.g., a server) can connect and communicate via a network. The server architecture can be found in [reference needed]. Figure 1 The structure shown will not be described in detail again.

[0087] Figure 4 An exemplary embodiment of the vehicle-cloud collaborative architecture of this application is illustrated. The inference service adopts a vehicle-cloud collaborative deployment mode. The user on the vehicle initiates a service request via voice inquiry (e.g., "Is it safe to go straight ahead?"). This service request is captured by the vehicle's built-in voice software development kit (SDK). Subsequently, the automatic speech recognition (ASR) system converts the user's voice command into text (query) format and transmits the text (query) to the central control service. The central control service performs domain-specific processing based on the intent corresponding to the text (query). For example, "Is it safe to go straight ahead?" represents the user's intent: to determine whether it is safe to go straight ahead. Based on this intent, the central control service determines that traffic signals need to be recognized, and based on the recognition result of the traffic signals, provides feedback to the user to fulfill the user's intent.

[0088] Based on this, the central control service acquires images recorded by the digital video recorder (DVR). The vehicle-mounted camera can also be other types of cameras besides the DVR; there are no restrictions.

[0089] The central control service can transmit user intent and image data to the cloud, which then initiates a traffic signal recognition service to identify traffic information in the environment. Optionally, the central control service can also transmit the query corresponding to the user intent to the cloud. For example, the cloud uses a first model and a second model to recognize traffic signals in the image data.

[0090] After the recognition process is complete, the cloud generates initial traffic information (such as the answer to "Is it safe to go straight ahead?") and transmits this information to the vehicle. Upon receiving the initial traffic information, the vehicle can display it on the central control screen. Alternatively, the vehicle can use text-to-speech (TTS) technology to convert the text-based initial traffic information into speech. The vehicle can also combine display screen output with speech output. In this way, by providing the user with initial traffic information, intuitive and convenient feedback can be offered. Furthermore, this process efficiently combines the advantages of in-vehicle equipment and cloud computing, providing users with accurate traffic signal recognition services.

[0091] like Figure 5An example of the inference service architecture involved in this application is shown. Exemplarily, this framework is a multi-threaded parallel processing framework to improve the overall performance of the target detection and response system in a traffic signal recognition scenario.

[0092] In some embodiments, the cloud can deploy multiple sub-models for the aforementioned large and small models respectively. Thus, after the cloud receives data input from the vehicle (such as images and text (i.e., traffic questions input by the user)), the multiple sub-models integrated in the cloud can perform synchronous inference to accelerate the processing flow and reduce response time. For example, as... Figure 5 For small models, two sub-models, YOLO1 and YOLO2, are deployed in the cloud. For large models, two sub-models, VLM-1 and VLM-2, are deployed in the cloud.

[0093] In some embodiments, different sub-models of the small model can be used to identify different types of traffic signals. As one possible implementation, some sub-models are the base model, while others are fine-tuned models based on the base model. For example, such as... Figure 5 YOLO-1 is the base model, and the cloud can leverage its basic capabilities to detect traffic lights (an example of the first type of traffic signal). YOLO-2 is a finely tuned model using a traffic signal image dataset. The cloud can use YOLO-2 to quickly detect other types of traffic signals besides traffic lights (an example of the second type of traffic signal), such as traffic signs, markings, or directional signs.

[0094] In some embodiments, different sub-models of the larger model can also have different uses. As one possible implementation, some sub-models are the base model, while others are fine-tuned models based on the base model. For example, such as... Figure 5 VLM-1 is the base model, and the cloud can use its basic capabilities for rejection responses. VLM-2 is a model obtained by fine-tuning the model using a multimodal traffic signal dataset. The cloud can use VLM-2 to provide detailed traffic signal response capabilities.

[0095] like Figure 5 Based on the recognition results of the YOLO sub-model, the cloud can intelligently determine whether to use a rejection link or a traffic signal recognition link. As one possible implementation, the cloud can select to use the response output by VLM-1 or VLM-2 based on the recognition results of YOLO1 / YOLO2 (the first model).

[0096] In some examples, the YOLO-1 model (an example of the first sub-model) fails to detect traffic light signs in the image data received from the vehicle, and the YOLO-2 model (an example of the second sub-model) also fails to detect any other traffic signal signs. In this case, the cloud determines to process the issue through a rejection link. For example, the cloud invokes VLM-1 (an example of the third sub-model) for inference and generates a response to the rejection scenario (e.g., no traffic signal detected) based on a preset prompt. The cloud can then transmit this response to the vehicle. For example, the cloud can transmit the content of this response in a streaming manner.

[0097] For example, if the image does not contain traffic signals, the VLM-1 can identify the surrounding environment of the vehicle in the image and output an environmental description, or output driving suggestions based on the environmental description. Environmental descriptions may include, for example, traffic congestion or being on a bridge.

[0098] Because large models can sometimes produce illusions—for example, in a scene without any signs ahead, the model might mistakenly believe there is a speed limit sign—this is due to the imbalance of signage information in the training set. In this embodiment, a smaller model can be used to make some preliminary judgments. When no traffic signal is detected, the inference result of VLM-1 can be directly selected as the answer. When a traffic signal is detected, the inference result of VLM-2 can be directly selected as the answer. This avoids false detections of traffic signals that can occur when directly using the large model for inference, thus preventing inaccurate answers. Furthermore, since the smaller model has already determined whether a traffic signal exists, the larger model (VLM-1 or VLM-2) does not need to make the same determination, improving the efficiency of the large model's traffic information output to some extent. Additionally, by embedding information about "no traffic signal in the image" into the VLM-1 prompt, VLM-1's perception can be adjusted, mitigating its illusions.

[0099] It is understandable that whether an image contains traffic signals can refer to whether it contains traffic signals relevant to the user's input traffic problem. For example, the user's input traffic problem might be whether there is a speed limit sign ahead. If the image does not contain a speed limit sign, the small model can assume that the image does not contain traffic signals. If the image does contain a speed limit sign, the small model can assume that the image contains traffic signals. Alternatively, whether the image contains traffic signals can refer to whether it contains any traffic signals. This application does not limit this.

[0100] Optionally, the cloud can concatenate the prompt (also known as a system prompt) and the user-inputted traffic question, and use the concatenated text and image as input parameters for a large model (such as VLM-1 mentioned above). The system prompt indicates the large model's role in the current dialogue and its intended function. The system prompt can be adjusted as needed and is generally concise. For example, the system prompt could be, "As an intelligent driving assistant, I need to help the user analyze traffic information ahead and provide driving assistance." The system prompt can be preset and fixed as described above, or it can be determined based on the user's input traffic question and other requirements. Simply put, the system prompt can correspond to the traffic question.

[0101] In some examples, if YOLO-1 detects a traffic light sign or YOLO-2 detects other traffic signal signs in the image data received from the vehicle, then inference is performed through the traffic signal recognition link. For example, the cloud enables the VLM-2 model (an example of the fourth sub-model) for inference and outputs a targeted traffic signal response (an example of the first traffic information) based on a preset prompt. For example: "You can go straight ahead, please stay in your lane." The cloud can transmit this response to the vehicle. Alternatively, the cloud can support efficient streaming of this response to ensure that users can obtain accurate traffic signal responses in a timely manner.

[0102] The reasoning framework described above can be applied to traffic signal question-and-answer scenarios. For example, ... Figure 6 VLM-2 can include a large language model (LLM) backbone, a vision encoder, and an input projector. Users can input text, such as asking via voice: What is the traffic sign ahead? (Q1) Or, "Can I go straight ahead?" (Q2) Or, "Can I park ahead?" (Q3) The vehicle can transmit the user's input traffic questions, along with images captured by its onboard cameras, to the cloud. Figure 6 The image contains road signs, traffic signs, traffic lights, and road markings. The cloud receives the user's traffic question, along with the image containing the traffic signals. Based on the above scheme, it calls VLM-2 for inference to generate an answer to the traffic question (example of first traffic information).

[0103] The input to VLM-2 includes multimodal data, which comprises text and image data. The text data may include system prompts and user input questions received from the vehicle. The image data may include images received from the vehicle.

[0104] like Figure 6 The cloud can vectorize the aforementioned text data to obtain text emdeddings, and then pass these text emdeddings to the large language model backbone. The cloud can also use the VLM-2 visual encoder and input projector to process the aforementioned image data, and then input the processed image data into the large language model backbone. The large language model backbone can then perform inference based on the text emdeddings and the processed image data, and output accurate answers to the user's traffic questions.

[0105] Still with Figure 6 For example, for the three traffic questions Q1, Q2, and Q3 input by the user, VLM-2 generates answers A1, A2, and A3, respectively. If the traffic question involves comprehensive driving strategies, such as "Is it permissible to go straight ahead?" or "Is it permissible to stop ahead?", VLM-2's answer will comprehensively utilize information such as traffic signs, road signs, traffic lights, and road markings identified from image data to generate a complete answer and driving suggestions.

[0106] This application also provides a function / method for recommending traffic issues. This method can be executed by a cloud server or a component within the server (such as a chip or a traffic knowledge recommendation module). This document uses server execution as an example. The method can include two stages: constructing a traffic knowledge recommendation question-and-answer database and recommending traffic knowledge, which are described below.

[0107] First, the process of building a traffic knowledge recommendation question-and-answer database is introduced. The server can select information sources for traffic knowledge. For example, it can select traffic knowledge from some official public databases. Then, the server can pull the text from the information sources, process the traffic knowledge through a large language model, and output traffic knowledge in question-and-answer format. This improves the simplicity and logic of the questions and answers, making it suitable for in-cabin voice response scenarios.

[0108] In some embodiments, the aforementioned traffic knowledge can also be manually verified to obtain a traffic knowledge recommendation question-and-answer database. Alternatively, the server can use other methods to verify the aforementioned traffic knowledge. For example, other models can be used to verify the aforementioned traffic knowledge.

[0109] In some embodiments, the server may also process the above-mentioned traffic knowledge recommendation question and answer database (hereinafter referred to as the question and answer database) into a knowledge base in vector format, so as to facilitate the rapid retrieval of the corresponding traffic knowledge from the knowledge base in the future.

[0110] After building the question-and-answer database, the server can subsequently retrieve relevant traffic knowledge from this database and, based on the retrieved traffic knowledge, generate recommended questions and corresponding answers related to the traffic knowledge (an example of second-level traffic information). Reference Figure 7 The method may include the following steps:

[0111] S101. The server analyzes the user scenario and obtains the user scenario vector.

[0112] In this embodiment, the server can obtain recommended dependency text. This recommended dependency text can serve as the basis for the server to recommend traffic knowledge. For example, the recommended dependency text can be a question-and-answer text related to traffic signs, and the server can recommend relevant traffic knowledge based on this text. For instance, if a user asks about the prohibitory signs "No Left Turn" and "No Honking," the server can recommend other questions and answers related to prohibitory signs based on that question.

[0113] As one possible implementation, the server can determine the user scenario vector v based on the network latency of the network currently used by the user and the user's tolerance for response delay.

[0114] S102. The server selects a retrieval algorithm based on the user scenario vector.

[0115] The server can select an appropriate retrieval algorithm based on the user scenario vector v to suit user needs. For example, for user scenarios requiring low response time and high frequency of response, a similarity matching algorithm with fast response can be selected. For user scenarios with high tolerance and low frequency of response, a more complex retrieval algorithm can be selected.

[0116] In some embodiments, the server may also use a sorting algorithm to finely sort the search results that require high-frequency response, so as to distinguish the priority of the search results and meet the user's requirements for search efficiency.

[0117] S103. The server inputs the recommended dependency text, executes the retrieval algorithm, and obtains the retrieval results in the question-and-answer database.

[0118] For example, if a user asks about a "No Left Turn" sign, the server can retrieve the corresponding answer from the question-and-answer database (example of the first corpus) (example of the first search result).

[0119] S104. The server inputs the search result into model A to obtain the recommendation question and the answer to the recommendation question.

[0120] As one possible implementation, the server can select a large language model as model A (an example of the third model). The server can construct a question-answering prompt and input the prompt and the retrieval result (an example of the first retrieval result) into model A.

[0121] Among them, the question-and-answer generation prompts can generate different numbers of recommended questions and answers.

[0122] For example, question-and-answer generation prompts may include the following: generate 3 recommended questions and answers. Model A can then generate 3 related recommended questions, and an answer for each recommended question.

[0123] For example, the question-and-answer generation prompt could be set to include the following: As an expert in the field of transportation, please rewrite and expand the input content…

[0124] For example, the server can input traffic knowledge about "No Left Turn" signs from user inquiries into model A, which then generates relevant recommended questions and answers. For instance, model A might generate recommended questions and answers related to "No Right Turn." It could also recommend driving techniques corresponding to "No Left Turn" signs. In this way, the server can personalize its responses to user traffic questions and provide relevant question-and-answer recommendations.

[0125] In some embodiments, the server may also input the question-and-answer generation prompt, the search results, and the traffic question asked by the user into model A, and generate relevant recommended questions and answers through model A.

[0126] The server can then send the recommended question and its answer to the vehicle. The vehicle can then display the recommended question and its answer on the central control screen. In some examples, the recommended question is displayed to the user, and after the user clicks on the recommended question, the vehicle displays the answer to the recommended question on the central control screen.

[0127] The traffic knowledge recommendations mentioned above are based on retrieval-augmented generation (RAG) technology, which can expand the depth of traffic Q&A and improve the interaction performance between users and vehicles.

[0128] Figure 8 This illustration shows another example of a method flow for the technical solution of this application. The method is applied to a vehicle, which includes an onboard camera. Figure 8 The method may include:

[0129] S201. Based on the first image, output the first traffic information.

[0130] Alternatively, the first image may be captured by an onboard camera in the vehicle.

[0131] The first traffic information is obtained by processing the first image using a first model and a second model; the first model is used to detect traffic signals in the first image and obtain detection results; the second model is used to generate the first traffic information based on the first image; the second model is determined based on the detection results.

[0132] The first model can be understood / replaced with the small model mentioned above, and the second model can be understood / replaced with the large model mentioned above.

[0133] Optionally, the above detection result can indicate whether a traffic signal exists in the first image, that is, whether the first image contains a traffic signal, in other words, whether a traffic signal has been detected. Whether a traffic signal exists in the first image can be referred to the previous description regarding whether an image contains a traffic signal.

[0134] In some embodiments, the method may further include receiving a first traffic question input by a user. Accordingly, the output of first traffic information may be implemented as follows: in response to the first traffic question, outputting the first traffic information, the first traffic information including an answer to the first traffic question.

[0135] The answer to the first traffic question includes: answers to traffic signal-related questions, and / or driving advice based on the traffic signal. For example, answers to traffic signal-related questions might include: [answers to questions related to traffic signals]. Figure 6 The answer to Q1 is shown below. Driving advice is, for example, for: Figure 6 Driving recommendations for Q3 are shown below.

[0136] In this way, the vehicle can provide traffic Q&A functionality, intelligently guiding users with traffic knowledge.

[0137] In some embodiments, the method further includes: outputting second traffic information, the second traffic information including at least one of the following: traffic knowledge related to the first traffic information, a second traffic question related to the first traffic question, and an answer to the second traffic question. For example, the first traffic information is the question: "No left turn". The second traffic information includes: a question related to "No left turn" about "No right turn", and an answer to the question about "No right turn".

[0138] In this way, vehicles can recommend relevant traffic Q&A to users to broaden their traffic-related knowledge.

[0139] In some embodiments, the method further includes: sending the first image to a server and receiving first traffic information obtained by the server processing the first image.

[0140] In some embodiments, the method further includes receiving the second traffic information from a server.

[0141] In some embodiments, the first model includes an image classification model or an object detection model, and the second model includes a visual language model.

[0142] For example, the first model can be at least one of the following: YOLO, quantum convolutional neural network (QCNN), convolutional neural network (CNN), residual network (ResNet), mobile network (MobileNet), or single shot multibox detector (SSD). No limitation is imposed.

[0143] For example, the second model can be a VLM. The second model can also be other types of models. For example, the second model is another model that can recognize image / point cloud data.

[0144] This application also provides a traffic signal recognition method, which can be applied to a server or a component of a server, such as... Figure 9 The method includes:

[0145] S301, Receive the first image from the vehicle.

[0146] S302, Send the first traffic information to the vehicle.

[0147] The first traffic information is obtained by processing the first image using a first model and a second model. The first model is used to detect traffic signals in the first image and obtain detection results. The second model is used to generate the first traffic information based on the first image. The second model is determined based on the detection results.

[0148] In some embodiments, the first traffic information and the answer to the first traffic question can refer to the above embodiments, and will not be repeated here.

[0149] In some embodiments, the method further includes: generating second traffic information and sending the second traffic information to the vehicle. The second traffic information includes at least one of the following: traffic knowledge related to the first traffic information, a second traffic question related to the first traffic question, or an answer to the second traffic question.

[0150] In some embodiments, there are multiple first models, each used to detect one or more traffic signals.

[0151] Optionally, the first model includes a first sub-model and a second sub-model, wherein the first sub-model is a base model and the second sub-model is a fine-tuned model based on the first sub-model. The first sub-model is used to detect a first type of traffic signal; the second sub-model is used to detect a second type of traffic signal.

[0152] Optionally, the second model is determined from multiple sub-models based on the detection results; each sub-model is used to generate traffic information based on the first image and the first traffic problem. Optionally, the multiple sub-models include a third sub-model and a fourth sub-model, the third sub-model being the base model and the fourth sub-model being a fine-tuned model based on the third sub-model.

[0153] In this way, the model can be split into different sub-models, each responsible for a different function, thereby improving the model's output performance.

[0154] For example, the first sub-model can be used to detect the first type of traffic signal, and the second sub-model can be used to detect the second type of traffic signal. In this way, different sub-models have different capabilities, and different sub-models can detect different types of traffic signals. This better matches the capabilities of the sub-models and makes the detection results more accurate.

[0155] In some embodiments, the method further includes: if no traffic signal is detected, that is, if the detection result indicates that there is no traffic signal in the first image, using the traffic information generated by the third sub-model as the first traffic information.

[0156] Still referencing Figure 5 Images captured by the vehicle's camera are simultaneously input into four models: YOLO1, YOLO2, VLM-1, and VLM-2, which run concurrently. Similarly, images captured by the vehicle's camera, along with user queries, are input into these four models.

[0157] If neither YOLO1 nor YOLO2 detects a traffic signal, the traffic information output by VLM-1 will be used as the first traffic information.

[0158] If YOLO1 or YOLO2 detects a traffic signal, the traffic information output by VLM-2 will be used as the first traffic information.

[0159] In this embodiment, by running the size models concurrently (such as the four models mentioned above), the user's waiting time for a response can be shortened, improving the user experience. For example, YOLO1 and YOLO2 take 30ms to infer the presence of a traffic signal, while VLM-1 and VLM-2 take 100ms to infer traffic information based on images. By running the size models in parallel, the user only needs to wait 100ms, which is less than the waiting time (30ms + 100ms) required when running the size models serially.

[0160] Optionally, if no traffic signal is detected, the fourth sub-model can be stopped, and only the third sub-model can be run, thus avoiding unnecessary waste of resources. Alternatively, the fourth sub-model can continue running, but the traffic information generated by the fourth sub-model will not be output.

[0161] In some embodiments, the method further includes: if a traffic signal is detected, that is, if the detection result indicates that a traffic signal exists in the first image, then using the traffic information generated by the fourth sub-model as the first traffic information. Optionally, if a traffic signal is detected, the third sub-model may be stopped or may continue to run.

[0162] In this embodiment, when the small model (such as YOLO1 and YOLO2) does not detect traffic signals, the cloud can select the VLM-1 rejection link to use the traffic information generated by VLM-1 as the first traffic information corresponding to the first traffic question input by the user. Optionally, the cloud can input the detection result indicating the absence of traffic signals to VLM-1, so that VLM-1 can directly determine that there are no traffic signals in the first image based on the detection result, thereby enabling rapid rejection inference to obtain the corresponding traffic information. For example, after receiving the detection result, VLM-1 can directly use the template indicating the absence of traffic signals (e.g., the image does not contain relevant information about traffic signals, or the image does not contain obvious relevant information about traffic signals) to generate the corresponding answer, that is, to generate the corresponding traffic information, without needing to continue detecting whether there are traffic signals in the first image. Of course, the cloud can also not input the detection result to VLM-1, and VLM-1 directly performs rejection inference, such as generating the corresponding answer using the template. In short, VLM-1 performs inference based on the assumption that there are no traffic signals in the first image.

[0163] Similarly, when a small model (such as YOLO1 or YOLO2) detects a traffic signal, the cloud can select VLM-2 as a traffic signal recognition link to provide the traffic information generated by VLM-2 as the first traffic information corresponding to the user's input traffic question. Optionally, the cloud can input the detection result indicating the presence of a traffic signal into VLM-2 to assist VLM-2 in detecting traffic signals in the first image and quickly generating the corresponding traffic signal. For example, after receiving the detection result, VLM-2 can directly use a template indicating the presence of a traffic signal (e.g., the image contains relevant information about traffic signals) to generate the corresponding answer, that is, to generate the corresponding traffic information. Of course, the cloud can also choose not to input the detection result into VLM-2, in which case VLM-2 will directly perform traffic signal recognition inference, such as using a template to generate the corresponding answer.

[0164] It should be noted that when it is necessary to provide driving suggestions to the user, the large model will also detect traffic signals present in the image of the vehicle's surrounding environment (such as the first image mentioned above).

[0165] In some embodiments, the second traffic information is processed and output by a third model that generates a Retrieval Enhancement Group (RAG), the input of which includes a first retrieval result obtained by retrieving a first traffic question from a first corpus.

[0166] In some embodiments, the retrieval algorithm used for searching in the first corpus differs depending on the user scenario; the user scenario is related to the response time and / or response frequency required by the user's search.

[0167] According to a second aspect of this application, embodiments of this application also provide a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the traffic signal recognition method described above. This non-transitory computer-readable storage medium possesses all the beneficial effects of the traffic signal recognition method described above, which will not be elaborated further here. This computer-readable storage medium can be used in vehicles or servers.

[0168] According to a third aspect of this application, embodiments of this application also provide a computer program product, including a computer program that, when executed by a processor, implements the traffic signal recognition method described above. This computer program product possesses all the beneficial effects of the traffic signal recognition method described above, which will not be elaborated further here. This computer program product can be used in vehicles or servers.

[0169] According to a fourth aspect of this application, embodiments of this application also provide an electronic device, including: a memory and a processor, wherein a computer program is stored in the memory; the processor is configured to execute the computer program in the memory to implement the steps of the traffic signal recognition method described above. This electronic device possesses all the beneficial effects of the traffic signal recognition method described above, which will not be elaborated upon further herein.

[0170] According to a fifth aspect of this application, embodiments of this application also provide a server, including: a memory and a processor, wherein a computer program is stored in the memory; the processor is used to execute the computer program in the memory to implement the steps of the traffic signal recognition method described above. This server possesses all the beneficial effects of the traffic signal recognition method described above, which will not be elaborated further here.

[0171] According to a sixth aspect of this application, embodiments of this application also provide a vehicle that includes the electronic device described in any of the above embodiments. This vehicle possesses all the beneficial effects of the aforementioned electronic device, etc., which will not be elaborated upon further herein.

[0172] The aforementioned computer-readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof, and this application does not specifically limit it. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0173] In some embodiments of this application, a computer-readable storage medium may be any tangible medium that contains or stores a program that may be used or combined with an instruction execution system, apparatus, or device.

[0174] The aforementioned computer-readable storage medium may be included in the aforementioned electronic device, or it may exist independently without being assembled into the electronic device. The aforementioned computer-readable storage medium carries one or more programs that, when executed by the electronic device, cause the control chip in the electronic device to communicate with the intelligent network system via Ethernet through a physical layer device.

[0175] Computer program code for performing operations of some embodiments of this application can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network (including a local area network (LAN) or a wide area network (WAN)), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0176] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function.

[0177] It should also be noted that in some alternative implementations, the functions marked in the box may occur in a different order than those marked in the attached figures.

[0178] For example, two consecutively represented blocks can actually be executed in substantially parallel order, and sometimes they can be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, as well as combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified functions or operations, or using a combination of dedicated hardware and computer instructions.

[0179] The units described in some embodiments of this application can be implemented in software or in hardware.

[0180] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0181] like Figure 10 The diagram shown is a structural schematic of another electronic device provided in an embodiment of this application. This electronic device 2200 can be used to implement the methods described in the above method embodiments. For example, the electronic device 2200 may specifically include a processing unit 2201. Optionally, the electronic device 2200 may further include a display unit 2202.

[0182] The processing unit 2201 is used to support the electronic device 2200 in performing operations. Figures 1 to 9 The processing function described in any one of the following.

[0183] The display unit 2202 is used to support the electronic device 2200 in performing its functions. Figures 1 to 9 The display function described in any one of the following statements.

[0184] Optional, Figure 10 The illustrated electronic device 2200 may also include a communication unit ( Figure 10 (Not shown in the image), this communication unit is used to support electronic device 2200 in performing the steps of communication between electronic device and other electronic devices in the embodiments of this application.

[0185] Optional, Figure 10 The illustrated electronic device 2200 may further include a storage unit 2203 that stores programs or instructions. When the processing unit 2201 executes the program or instructions, it causes... Figure 10The electronic device 2200 shown can perform the method described in the above-described method embodiments.

[0186] Figure 10 The technical effects of the electronic device 2200 shown can be referred to the technical effects of the method shown in the above method embodiments, and will not be repeated here. Figure 10 The processing unit 2201 involved in the illustrated electronic device 2200 can be implemented by a processor or processor-related circuit components, and can be a processor or processing module. The communication unit can be implemented by a transceiver or transceiver-related circuit components, and can be a transceiver or transceiver module. The display unit 2202 can be implemented by display screen-related components.

[0187] This application also provides a chip system, such as... Figure 11 As shown, the chip system includes at least one processor 2301 and at least one interface circuit 2302. The processor 2301 and the interface circuit 2302 are interconnected via lines. For example, the interface circuit 2302 can be used to receive signals from other devices. As another example, the interface circuit 2302 can be used to send signals to other devices (e.g., the processor 2301). Exemplarily, the interface circuit 2302 can read instructions stored in memory and send those instructions to the processor 2301. When the instructions are executed by the processor 2301, the electronic device can perform the various steps performed by the electronic device in the above embodiments. Of course, the chip system may also include other discrete devices, and this application embodiment does not specifically limit this.

[0188] Optionally, the chip system may contain one or more processors. These processors can be implemented in hardware or software. When implemented in hardware, the processor can be a logic circuit, an integrated circuit, etc. When implemented in software, the processor can be a general-purpose processor, implemented by reading software code stored in memory.

[0189] Optionally, the chip system may contain one or more memories. The memory may be integrated with the processor or disposed separately from it; this application does not limit this. For example, the memory may be a non-transient processor, such as a read-only memory (ROM), which may be integrated with the processor on the same chip or disposed separately on different chips. This application does not specifically limit the type of memory or the arrangement of the memory and processor.

[0190] For example, the chip system may be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a micro controller unit (MCU), a programmable logic device (PLD), or other integrated chips.

[0191] It should be understood that each step in the above method embodiments can be completed by integrated logic circuits in the processor hardware or by instructions in software form. The method steps disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or being executed by a combination of hardware and software modules in the processor.

[0192] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more features. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0193] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0194] The embodiments, implementation methods, and related technical features of this application can be combined and substituted for each other without conflict.

[0195] The above are merely preferred embodiments of this application and are not intended to limit this application in any way. Any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of this application without departing from the scope of the technical solution of this application shall still fall within the scope of the technical solution of this application.

Claims

1. A traffic signal recognition method characterized by, The method comprises: outputting first traffic information based on a first image; wherein the first traffic information is obtained by processing the first image using a first model and a second model; the first model is used to detect a traffic signal in the first image to obtain a detection result; the second model is used to generate the first traffic information based on the first image; and the second model is determined based on the detection result.

2. The method of claim 1, wherein, The method further comprises: receiving a first traffic question input by a user; The outputting of the first traffic information comprises: outputting the first traffic information in response to the first traffic question.

3. The method of claim 2, wherein, The first traffic information comprises an answer to the first traffic question. The answer to the first traffic question comprises an answer to a traffic signal related question and / or a driving suggestion based on the traffic signal.

4. The method of claim 2, wherein, The method further comprises: outputting second traffic information; wherein the second traffic information comprises at least one of the following: traffic knowledge related to the first traffic information, a second traffic question related to the first traffic question, and an answer to the second traffic question.

5. The method according to any one of claims 1 to 4, characterized in that, The method further comprises: sending the first image to a server; receiving first traffic information processed by the server based on the first image.

6. The method of claim 4, wherein, The method further comprises: receiving the second traffic information from the server.

7. The method according to any one of claims 1 to 4, wherein The first model comprises an image classification model or an object detection model, and the second model comprises a visual language model.

8. A traffic signal recognition method characterized by, The method comprises: receiving a first image from a vehicle; sending first traffic information to the vehicle; wherein the first traffic information is obtained by processing the first image using a first model and a second model; the first model is used to detect a traffic signal in the first image to obtain a detection result; the second model is used to generate the first traffic information based on the first image; and the second model is determined based on the detection result. The first traffic information comprises an answer to a first traffic question of a user.

9. The method of claim 8, wherein, The answer to the first traffic question comprises an answer to a traffic signal related question and / or a driving suggestion based on the traffic signal. The method further comprises:

10. The method of claim 9, wherein, generating second traffic information; wherein the second traffic information comprises at least one of the following: traffic knowledge related to the first traffic information, a second traffic question related to the first traffic question, and an answer to the second traffic question; sending the second traffic information to the vehicle. The first model has multiple, and each of the first models is used to detect one or more types of traffic signals.

11. The method according to any one of claims 8 to 10, characterized in that, The first model comprises a first sub-model and a second sub-model.

12. The method of claim 11, wherein, wherein the first sub-model is a base model, and the second sub-model is a fine-tuned model based on the first sub-model; the first sub-model is used to detect a first type of traffic signal; and the second sub-model is used to detect a second type of traffic signal. The second model is determined from multiple sub-models based on the detection result; and each of the sub-models is used to generate traffic information based on the first image and the first traffic question.

13. The method of any one of claims 8 to 10, wherein, ​ 14. The method of claim 13, wherein, The plurality of sub-models comprises a third sub-model and a fourth sub-model; wherein the third sub-model is a base model, and the fourth sub-model is a fine-tuning model based on the third sub-model.

15. The method of claim 14, wherein, Further comprising: In the case where no traffic signal is detected, the traffic information generated by the third sub-model is taken as the first traffic information.

16. The method of claim 14, wherein, Further comprising: In the case where a traffic signal is detected, the traffic information generated by the fourth sub-model is taken as the first traffic information.

17. The method of claim 10, wherein, The second traffic information is output by processing via a third model of Retrieval Augmentation Generation (RAG), and the input of the third model comprises a first search result obtained by searching a first corpus for a first traffic question.

18. The method of claim 17, wherein, Different search algorithms are used for searching in the first corpus under different user scenarios; the user scenarios are related to the response time and / or response frequency required by the user for the search.

19. An electronic device, comprising: Comprising: a memory having stored thereon a computer program; a processor configured to execute the computer program in the memory to implement the method of any one of claims 1-7.

20. A vehicle characterized by The electronic device of claim 19 is included.

21. A server, comprising: Comprising: a memory having stored thereon a computer program; a processor configured to execute the computer program in the memory to implement the method of any one of claims 8-18.

22. A computer readable storage medium having stored thereon a computer program or instructions, characterized in that, The computer program or instructions, when executed by a processor, implement the method of any one of claims 1-18.

23. A computer program product comprising computer programs or instructions, characterized in that, The computer program or instructions, when executed by a processor, implement the method of any one of claims 1-18. The computer program or instructions, when executed by a processor, implement the method of any one of claims 1-18.

Images