Navigation fault reporting method, navigation terminal, vehicle and server

Abstract

This invention discloses a method for reporting navigation faults, a navigation terminal, a vehicle, and a server, relating to the field of automotive technology. One specific embodiment of the method includes: responding to a trigger operation for voice acquisition, activating a sound sensor to acquire user voice to obtain voice information for the current navigation operation; responding to the trigger operation for voice acquisition, acquiring multimodal data for the navigation operation displayed on the screen to obtain context information for the current navigation operation, the context information including one or more of the following: a screenshot of the navigation interface, a screen recording of the navigation operation, and a navigation log; constructing a fault data packet based on the voice information and the context information, and uploading the fault data packet so that a cloud server generates a fault work order based on the fault data packet. This embodiment can accelerate the processing speed of navigation faults.

Description

Technical Field

[0001] This invention relates to the field of automotive technology, and more particularly to a method for reporting navigation faults, a navigation terminal, a vehicle, and a server. Background Technology

[0002] During the research and development and testing of automobiles, various problems that may occur in in-vehicle navigation during road testing or actual use, such as route planning errors, delayed guidance instructions, failed point of interest search, inaccurate traffic information, system lag or crashes, often rely on human feedback.

[0003] In the process of realizing this invention, the inventors discovered that the prior art has at least the following problems: manual feedback is repetitive and time-consuming, which seriously restricts the processing speed of navigation faults. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a method for reporting navigation faults, a navigation terminal, a vehicle, a server, a device, and a computer-readable medium, which can accelerate the processing speed of navigation faults.

[0005] A method for reporting navigation faults, applied to the vehicle side, includes: In response to the trigger operation of voice acquisition, the sound sensor is activated to acquire the user's voice to obtain voice information for the current navigation operation; In response to the triggering operation of the voice acquisition, multimodal data is acquired for the navigation operation on the display to obtain the context information of the current navigation operation. The context information includes one or more of the following: screenshot of the navigation interface, screen recording of the navigation operation, and navigation log. A fault data packet is constructed based on the voice information and the context information, and the fault data packet is uploaded so that the cloud server can generate a fault work order based on the fault data packet.

[0006] After the sound sensor is activated to collect user voice data, the method further includes: The sound sensor stops collecting user voice after a preset collection time. The preset collection time includes a preset time after the user's voice ends or a preset time after the user's voice begins. And / or, Receive the successful transmission message of the fault work order sent by the cloud server, and display or broadcast the successful transmission message of the fault work order.

[0007] The method further includes: In response to the trigger operation of the voice acquisition, the vehicle-mounted sensors are activated to acquire vehicle parameters, and the vehicle parameters are added to the context information. The vehicle parameters include one or more of the following: current vehicle location, planned route, vehicle speed, traffic conditions, navigation device identifier, navigation application version identifier, and network connection status. And / or, Before generating the fault data packet based on the voice information and the context information, the method further includes: The voice information and the context information are compressed and / or encrypted.

[0008] According to a second aspect of the present invention, a method for reporting navigation faults is provided, applied to a cloud server, comprising: Receive a fault data packet, the fault data packet including voice information of the current navigation operation collected in response to the trigger operation, and context information of the current navigation operation collected synchronously with the voice information; User fault information is obtained by recognizing the voice information of the current navigation operation, and device fault information is obtained by parsing the context information of the current navigation operation. The device fault information includes one or more of the following: navigation interface screenshot, navigation operation screen recording, and navigation log. Based on the user fault information and equipment fault information, a fault work order is constructed and sent to the defect tracking system to report the navigation fault.

[0009] The method further includes: Semantic analysis is performed on the user fault information to obtain the classification labels and / or priorities of navigation faults; The process of constructing a fault work order based on the user fault information and equipment fault information includes: The type of the fault work order is determined based on the classification label and / or priority of the navigation fault, and the user fault information and equipment fault information are filled in according to the type of the fault work order to generate the fault work order.

[0010] The method further includes: Semantic analysis is performed on the user fault information to obtain classification labels for navigation faults; Sending the fault work order to the defect tracking system includes: The defect tracking system [s4.1] that determines the classification label of the navigation fault includes the ticket system, and the fault work order is sent to the defect tracking system in batches / in real time.

[0011] The method further includes: After generating the fault work order and / or sending the fault work order to the defect tracking system, feedback information indicating successful sending of the fault work order is provided to indicate that the fault work order was successfully sent.

[0012] According to a third aspect of the present invention, a navigation terminal is provided, applied to a vehicle, the navigation terminal comprising: The control module, in response to the trigger operation of voice acquisition, activates the sound sensor to acquire the user's voice in order to obtain voice information for the current navigation operation; In response to the triggering operation of the voice acquisition, multimodal data is acquired for the navigation operation on the display to obtain the context information of the current navigation operation. The context information includes one or more of the following: screenshot of the navigation interface, screen recording of the navigation operation, and navigation log. The upload module is used to generate a fault data packet based on the voice information and the context information, and upload the fault data packet so that the cloud server can generate a fault work order based on the fault data packet.

[0013] According to a fourth aspect of the present invention, a vehicle is provided, including the navigation terminal as described above.

[0014] According to a fifth aspect of the present invention, a server is provided, comprising: A receiving module is used to receive fault data packets, the fault data packets including voice information of the current navigation operation collected in response to a trigger operation, and context information of the current navigation operation collected synchronously with the voice information; The parsing module is used to identify the voice information of the current navigation operation to obtain user fault information, and to parse the context information of the current navigation operation to obtain device fault information. The device fault information includes one or more of the following: navigation interface screenshots, navigation operation screen recordings, and navigation logs. The work order module is used to construct a fault work order based on the user fault information and equipment fault information, and send the fault work order to the defect tracking system to report the navigation fault.

[0015] According to a sixth aspect of the present invention, an electronic device for reporting navigation faults is provided, comprising: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors perform the methods described above.

[0016] According to a seventh aspect of the present invention, a computer-readable medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method as described above.

[0017] One embodiment of the above invention has the following advantages or beneficial effects: In response to the trigger operation of voice acquisition, the context information of the current navigation operation is obtained while acquiring the user's voice, so as to construct a fault data packet, upload it to the cloud server, and generate a fault work order. This allows users to report navigation faults in a timely manner during navigation operations, which helps to speed up the handling of navigation faults.

[0018] The further effects of the aforementioned unconventional alternative methods will be explained below in conjunction with specific implementation methods. Attached Figure Description

[0019] The accompanying drawings are provided to better understand the invention and are not intended to unduly limit the scope of the invention. Wherein: Figure 1 This is a flowchart illustrating the navigation fault reporting process; Figure 2 This is a schematic diagram of the main process of the navigation fault reporting method according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the main flow of another navigation fault reporting method according to an embodiment of the present invention; Figure 4 This is a hierarchical diagram of navigation fault handling according to an embodiment of the present invention; Figure 5 This is an application diagram of navigation fault handling according to an embodiment of the present invention; Figure 6 This is an interactive schematic diagram of navigation fault handling according to an embodiment of the present invention; Figure 7 This is a schematic diagram of the structure of a navigation terminal according to an embodiment of the present invention; Figure 8 This is a schematic diagram of the server structure according to an embodiment of the present invention; Figure 9 This is an exemplary system architecture diagram in which embodiments of the present invention can be applied; Figure 10 This is a schematic diagram of the structure of a computer system suitable for implementing terminal devices or servers of the present invention. Detailed Implementation

[0020] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of the present invention, including various details to aid understanding. These details should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0021] See Figure 1, Figure 1 This is a flowchart illustrating the navigation fault reporting process. It adopts... Figure 1 The solution enables manual creation of work orders.

[0022] S101, Problem discovered.

[0023] During road tests or actual use, testers discovered abnormalities in the navigation system, such as: route planning errors, inaccurate guidance instructions, no results for point of interest searches, abnormal interface display, or system crashes.

[0024] S102, Manually record the problem.

[0025] Testers need to manually record detailed information about the fault after it occurs.

[0026] S103, take a screenshot or record the screen manually.

[0027] Testers need to manually operate the in-vehicle equipment to capture images of the navigation interface when the malfunction occurs via screenshots or external cameras. They also need to manually activate the screen recording function to record the operation process and system response.

[0028] S104. Manually export log files.

[0029] Testers manually searched for, located, and exported fault-related log files using the vehicle system's file management function.

[0030] S105. Fill in the problem description.

[0031] Testers compile information such as the fault symptoms, the time and location of the fault, and form a problem description in text form.

[0032] S106. Upload data.

[0033] Testers will manually package the collected screenshots, screen recordings, log files, and problem descriptions and upload them to the designated server or problem management system.

[0034] S107. Manually create work orders.

[0035] After uploading the data, the testers manually create a fault ticket and attach the uploaded data as an attachment.

[0036] S108, Awaiting processing and feedback.

[0037] Once a fault ticket is created, it is handled by the tester. If additional information is needed during the handling process, the tester will repeat the above steps to collect and upload the additional data.

[0038] The aforementioned manual feedback process is repetitive and time-consuming, severely limiting the speed of handling navigation faults.

[0039] To expedite the handling of navigation malfunctions, the following technical solutions from the embodiments of the present invention can be adopted.

[0040] See Figure 2 , Figure 2 This is a schematic diagram of the main flow of the navigation fault reporting method according to an embodiment of the present invention. Figure 2 The navigation terminal is the main entity responsible for executing each step. The navigation terminal can be installed in the vehicle. Specifically, it includes the following steps: S201. In response to the trigger operation of voice acquisition, the sound sensor is activated to acquire the user's voice to obtain voice information for the current navigation operation.

[0041] In embodiments of the present invention, the triggering operation for voice acquisition includes acquiring prompt words and / or gesture commands. For example, the sound sensor is activated to acquire user voice via prompt words and / or gesture commands.

[0042] Additionally, trigger operations can be achieved through buttons located inside the vehicle. For example, physical buttons can be installed on the center console, dashboard, or armrest, with unique shapes or tactile feedback for easy driver operation while driving. Alternatively, existing multifunction buttons on the steering wheel can be used to set preset operation methods (such as a 3-second long press or double-click) to trigger voice input.

[0043] When a user encounters navigation issues such as route deviation, incorrect guidance, or no search results, they simply need to press the aforementioned button. The vehicle detects the trigger button and recognizes it as a voice recording operation. In embodiments of this invention, the user can be either the driver or a passenger. The vehicle utilizes its existing sound sensors, specifically microphones for voice interaction, such as a driver's side microphone or a roof microphone, to record the user's voice while minimizing interference from wind noise, road noise, and other ambient noise within the vehicle.

[0044] After the sound sensor is successfully activated, it acquires voice information related to the current navigation operation. For example, the user's voice description of the current navigation operation: "The route planned by the navigation seems to be wrong. It's telling me to leave the main road, but based on experience, I should continue straight."

[0045] S202. In response to the trigger operation of voice acquisition, multimodal data is acquired for navigation operations on the display to obtain context information of the current navigation operation. The context information includes one or more of the following: screenshot of the navigation interface, screen recording of navigation operation, and navigation log.

[0046] In embodiments of the present invention, the acquisition of multimodal data and the acquisition of user speech are performed synchronously. That is, the triggering operation not only activates the sound sensor, but also triggers the acquisition of multimodal data in parallel, thereby ensuring that the speech information corresponds to the context information in the time dimension.

[0047] Navigation interface screenshots can capture the visual state of the current navigation operation. If the navigation interface undergoes dynamic changes, such as continuous map scrolling or flashing guide arrows, multiple navigation interface screenshots can be taken consecutively to capture key moments of change.

[0048] The navigation operation screen recording function records the navigation operation process for a period of time before and after the current navigation operation. The resolution and frame rate of the navigation operation screen recording can be dynamically adjusted according to storage space and network transmission conditions. The resolution can be set to 720p or 1080p to balance image quality and data volume.

[0049] Navigation logs record the internal state, events, and error information during the operation of the navigation application, serving as the core basis for fault location. After a triggered operation occurs, navigation logs for a period of time before and after the trigger moment are immediately extracted from the navigation application's memory buffer or log file.

[0050] S203. Construct a fault data packet based on voice information and context information, and upload the fault data packet so that the cloud server can generate a fault work order based on the fault data packet.

[0051] After collecting the voice and context information for the current navigation operation, the voice and context information are associated and encapsulated to construct a structured fault data packet. This fault data packet integrates voice and context data that can fully reconstruct the scenario in which the navigation failure occurred.

[0052] The fault data packet is uploaded to the cloud server. After receiving the fault data packet, the cloud server parses the voice information and context information within it, and generates a fault work order that conforms to a preset format, thus realizing the process of navigation fault data collection from the vehicle end to work order creation on the cloud server.

[0053] exist Figure 2 In this embodiment, a fault data packet is constructed by collecting user voice and context information and sent to the cloud server to quickly generate a fault work order, thereby speeding up the processing of navigation faults.

[0054] In one embodiment of the present invention, in order to improve the effectiveness of controlling the amount of user voice data, a preset acquisition duration is used as the execution condition for stopping the acquisition of user voice. After the preset acquisition duration, the sound sensor stops acquiring user voice.

[0055] The preset acquisition duration includes a preset time after the user's voice ends. If no user voice is received within this preset acquisition duration, it indicates the user has stopped input, and the sound sensor stops acquiring the user's voice. Recording stops when the environment is silent for the preset duration after the user's voice has ended. The preset acquisition duration allows for dynamic adjustment of the acquisition time based on the length of the user's voice.

[0056] The preset acquisition duration includes the preset duration after the user begins speaking; that is, the duration for acquiring the user's voice is the preset acquisition duration. Using the preset acquisition duration helps prevent the amount of user voice data from becoming too large.

[0057] In one embodiment of the present invention, after the cloud server completes speech recognition, semantic parsing, multimodal data integration, and successfully generates a fault work order, it executes the fault work order issuance operation. Upon successful issuance of the fault work order, the cloud server provides feedback with a fault work order issuance success message to confirm the successful issuance of the navigation fault report. To promptly remind the user, the fault work order issuance success message can be displayed or broadcast to confirm the successful issuance of the navigation fault report.

[0058] In one embodiment of the present invention, in order to improve the accuracy of identifying navigation faults, the vehicle-mounted sensors are activated to collect vehicle parameters and the vehicle parameters are added to the context information.

[0059] Upon detecting a trigger action to collect voice data, the vehicle's sensors are activated to collect vehicle parameters and add these parameters to the context information. The vehicle sensors include GPS, an onboard camera, and a communication module. GPS collects the vehicle's current location and speed, the onboard camera collects traffic conditions, and the communication module collects network connectivity status. The planned route, navigation device identifier, and navigation application version identifier are obtained from the onboard system.

[0060] To ensure the synchronization of data collection, vehicle parameters are identified with timestamps to ensure accurate alignment with voice information, navigation interface screenshots, navigation operation recordings, and navigation logs.

[0061] In one embodiment of the present invention, in order to optimize the data volume of faulty data packets, improve transmission efficiency and ensure data security, compression and / or encryption processing is performed on voice information and context information.

[0062] Compression processing includes lossy compression and lossless compression. Lossy compression can be used for navigation interface screenshots and navigation operation screen recordings to significantly reduce the data size of data packets. Lossless compression is used for voice information and navigation logs to ensure that no information is lost while reducing the data size.

[0063] Encrypt faulty data packets. For example, use a symmetric encryption algorithm to encrypt faulty data packets, ensuring that even if the transmission channel is eavesdropped on, the content of the faulty data packets cannot be decrypted.

[0064] See Figure 3 , Figure 3 This is a schematic diagram of the main flow of another navigation fault reporting method according to an embodiment of the present invention. Figure 3 The execution of each step is carried out by a cloud server. Specifically, it includes the following steps: S301. Receive fault data packet, which includes voice information of the current navigation operation collected in response to the trigger operation, and context information of the current navigation operation collected synchronously with the voice information.

[0065] In an embodiment of the invention, a cloud server receives and parses fault data packets. The cloud server may employ a distributed architecture to simultaneously process concurrent navigation fault uploads from multiple vehicles.

[0066] The cloud server employs a streaming reception method, writing data to the cloud storage area as it is received, thus preventing memory overflow due to excessively large faulty data packets. Furthermore, the cloud server supports resumeable transmission. If the network is interrupted during the upload process, the navigation terminal can initiate a resume request with the number of bytes already uploaded after the connection is restored. The cloud receiving service then continues receiving the remaining data based on the already received portion.

[0067] If the faulty data packet is compressed and encrypted, the cloud server first decrypts it using a preset decryption key, and then performs the decompression operation.

[0068] S302. Recognize the voice information of the current navigation operation to obtain user fault information, and parse the context information of the current navigation operation to obtain device fault information. The device fault information includes one or more of the following: screenshot of the navigation interface, screen recording of the navigation operation, and navigation log.

[0069] Voice information is identified from the fault data packet. An in-vehicle navigation language model is used to recognize the voice information, resulting in recognized text. Structured user fault information is then extracted from the recognized text. For example, location entities, behavior entities, phenomenon entities, and time entities are extracted from the recognized text. User fault information is constructed based on these entities.

[0070] The device fault information is obtained by parsing the context information of the current navigation operation. The device fault information includes one or more of the following: screenshot of the navigation interface, screen recording of the navigation operation, and navigation log.

[0071] Key information is extracted from navigation interface screenshots using image recognition technology. User operation sequences and navigation behavior trajectories are extracted from navigation operation screen recordings using video analysis technology. Fault clues are extracted from navigation logs using log analysis technology.

[0072] S303. Based on user fault information and equipment fault information, construct a fault work order and send the fault work order to the defect tracking system to report the navigation fault.

[0073] In one embodiment of the present invention, fault work orders are constructed using fault work order templates. The fault work order templates can be configured according to the system settings for receiving fault work orders. For example, fault work order templates may include JIRA templates and internal platform fault work order templates. Fault work order templates can also be configured according to fault types, such as route planning templates, guidance error templates, and search failure templates.

[0074] After filling in the user's fault information and the equipment's fault information in the fault work order template, a fault work order is generated and sent to report the navigation fault. For example, a fault work order can be sent through an application programming interface (API).

[0075] In the above embodiments, fault work orders can be constructed based on voice information and context information without manual intervention, which significantly improves the efficiency of navigation fault reporting and processing.

[0076] In one embodiment of the present invention, the cloud server can perform semantic analysis on user fault information to obtain classification tags and / or priorities for navigation faults. For example, classification tags may include: route planning problems, navigation guidance problems, point-of-interest search problems, real-time traffic problems, interface display problems, speed limit reminder problems, and system stability problems. Priorities may include: highest, high, medium, low, and recommended.

[0077] The type of fault work order is determined based on the classification tags and / or priorities of navigation faults. Specifically, the template, required fields, and field mapping rules for creating fault work orders are determined according to the classification tags and / or priorities. Then, user fault information and equipment fault information are filled in the above template to generate the corresponding fault work order.

[0078] Using the above embodiments, fault work orders are generated based on the classification tags and / or priorities of navigation faults, thereby improving the targeting of fault work orders.

[0079] In one embodiment of the present invention, the cloud server can perform semantic analysis on user fault information to obtain classification tags for navigation faults. The classification tags for navigation faults characterize the type of navigation fault. Each type of navigation fault is handled by a corresponding defect tracking system. For example, the defect tracking system includes a tickets system, which includes a first tickets system and a second tickets system.

[0080] The first ticketing system handles issues related to route planning, navigation guidance, point-of-interest (POI) search, real-time traffic, and speed limit reminders. The second ticketing system handles issues related to user interface display and system stability.

[0081] Based on the ticket system corresponding to the navigation fault's classification tag, fault tickets are sent in batches or in real-time to the defect tracking system. Batch sending is suitable for issues processed by the first ticket system. Real-time sending is suitable for issues processed by the second ticket system.

[0082] Using the above embodiments, navigation faults can be classified and processed, enabling fault work orders to be handled in a targeted manner.

[0083] In one embodiment of the present invention, after a fault work order is generated on the cloud server and / or sent to the defect tracking system, a fault work order sending success message is sent back to the vehicle. The fault work order sending success message is used to inform the user that the fault information has begun processing.

[0084] By using the success message sent by the fault work order, interaction with the user who reported the fault can be achieved, thereby improving the user experience.

[0085] See Figure 4 , Figure 4 This is a hierarchical diagram of navigation fault handling according to an embodiment of the present invention.

[0086] The user interaction layer 401 is located inside the vehicle and includes: physical buttons located on the steering wheel or center console, etc., to receive user trigger operations; and a vehicle infotainment interface / voice assistant used to display or broadcast information to the user.

[0087] The vehicle system layer 402 is located at the vehicle terminal and is responsible for data collection and processing. As an example, it includes a voice module, a data acquisition module, and a data packaging module.

[0088] The voice module responds to physical button presses and collects user voice input.

[0089] The data acquisition module starts synchronously with the voice module to collect context information of the current navigation operation.

[0090] The data packaging module constructs fault data packets based on voice information and context information.

[0091] The communication layer 403 includes an uploading module and a feedback receiving module.

[0092] The upload module is used to upload faulty data packets.

[0093] The feedback receiving module is used to receive the successful sending information of the fault work order returned by the cloud and transmit it to the user interaction layer.

[0094] The cloud processing layer 404 is located on the cloud server and includes: a receiving module, a speech recognition and semantic parsing module, a multimodal data integration module, and a work order generation module.

[0095] The receiving module is used to receive fault data packets.

[0096] The speech recognition and semantic parsing module is used to recognize speech information.

[0097] The multimodal data integration module is used to parse context information.

[0098] The work order generation module is used to create fault work orders based on user fault information and equipment fault information.

[0099] The external system layer 405 includes the OEM Ticket system.

[0100] The OEM Ticket system is used to receive and process fault tickets.

[0101] See Figure 5 , Figure 5 This is a schematic diagram illustrating the application of navigation fault handling according to an embodiment of the present invention. Figure 5 S501 to S505 are executed on the navigation terminal. S506 to S510 are executed on the cloud server.

[0102] S501, Trigger Operation.

[0103] After discovering a navigation malfunction while operating the vehicle, the tester triggered a navigation incident report by pressing a physical button.

[0104] S502, Collect user voice.

[0105] The sound sensor is activated to collect user voice data to obtain voice information related to the current navigation operation.

[0106] S503, Collect context information.

[0107] Simultaneously with user voice input, multimodal data is collected to obtain contextual information for the current navigation operation.

[0108] S504, Data Packaging Fault data packets are constructed based on voice information and context information.

[0109] S505, Data Upload.

[0110] Upload the fault data package to the cloud server.

[0111] S506, cloud server processing.

[0112] Receive fault data packets and identify fault data packets to obtain user fault information and equipment fault information.

[0113] S507, Create a fault work order.

[0114] Fault work orders are generated based on user fault information and equipment fault information.

[0115] S508, Send fault work order.

[0116] Send a fault work order.

[0117] S509. Receive confirmation information.

[0118] The cloud server receives work order confirmation information returned by the defect tracking system.

[0119] S510, User Feedback.

[0120] The cloud server sends a success message to the vehicle terminal regarding the successful transmission of the fault work order.

[0121] See Figure 6 , Figure 6 This is an interactive schematic diagram of navigation fault handling according to an embodiment of the present invention. Figure 6 The system comprises various interactive modules, including: a navigation terminal 61, a voice module 611, a data acquisition module 612, a data module 613, a cloud server 62, a semantic parsing module 621, a data integration module 622, a work order generation module 623, and a ticket system 63. The navigation terminal 61 includes the voice module 611, the data acquisition module 612, and the data module 613. The cloud server 62 includes the semantic parsing module 621, the data integration module 622, and the work order generation module 623.

[0122] Figure 6 The horizontal arrows indicate message passing and data flow between modules, arranged chronologically from top to bottom.

[0123] After a user in the vehicle discovers a navigation malfunction, they send a trigger operation. Upon receiving the trigger operation, the navigation terminal 61 sends a command to the voice module 611 to collect the user's voice and a command to the collection module 612 to collect context information. The data module 613 stores the user's voice collected by the voice module 611 and the context information collected by the collection module 612. The data module 613 encapsulates the voice data and context information to construct a structured fault data packet. The data module 613 then uploads the constructed fault data packet to the cloud server 62.

[0124] After receiving the fault data packet, the cloud server 62 sends the voice data to the semantic parsing module 621 for speech recognition and semantic analysis, converting the user's voice into user fault information. The cloud server 62 then sends the context information to the data integration module 622. The data integration module 622 parses the context information to obtain the equipment fault information. The work order generation module 623 constructs a fault work order based on the user fault information and the equipment fault information.

[0125] The work order generation module 623 sends the fault work order to the ticket system 63. The ticket system 63 is the defect tracking system. After receiving the fault work order and creating the corresponding task entry, the ticket system 63 returns work order creation confirmation information to the work order generation module 623. The work order generation module 623 then sends the confirmation information to the cloud server 62.

[0126] After receiving confirmation of work order creation, cloud server 62 sends a successful work order notification to navigation terminal 61. Upon receiving this notification, navigation terminal 61 alerts the user via the vehicle's screen or voice assistant.

[0127] See Figure 7 , Figure 7 This is a schematic diagram of the structure of a navigation terminal according to an embodiment of the present invention. The navigation terminal includes: The control module 701, in response to the trigger operation of voice acquisition, activates the sound sensor to acquire the user's voice in order to obtain voice information for the current navigation operation; In response to the triggering operation of the voice acquisition, multimodal data is acquired for the navigation operation on the display to obtain the context information of the current navigation operation. The context information includes one or more of the following: screenshot of the navigation interface, screen recording of the navigation operation, and navigation log. The upload module 702 is used to generate a fault data packet based on the voice information and the context information, and upload the fault data packet so that the cloud server can generate a fault work order based on the fault data packet.

[0128] In one embodiment of the present invention, the control module 701 is used to stop the sound sensor from collecting user voice after a preset collection duration, wherein the preset collection duration includes a preset duration after the user voice ends or a preset duration after the user voice begins. Receive the successful transmission message of the fault work order sent by the cloud server, and display or broadcast the successful transmission message of the fault work order.

[0129] In one embodiment of the present invention, the control module 701 is used to respond to the trigger operation of the voice acquisition, start the vehicle sensor to acquire vehicle parameters, and add the vehicle parameters to the context information. The vehicle parameters include one or more of the following: current vehicle location, planned route, vehicle speed, traffic conditions, navigation device identifier, navigation application version identifier, and network connection status. The upload module 702 is used to compress and / or encrypt the voice information and the context information.

[0130] The navigation terminal in this embodiment of the invention can be applied to vehicles.

[0131] See Figure 8 , Figure 8 This is a schematic diagram of the structure of a server according to an embodiment of the present invention. The server includes: The receiving module 801 is used to receive a fault data packet, the fault data packet including voice information of the current navigation operation collected in response to the trigger operation, and context information of the current navigation operation collected synchronously with the voice information; The parsing module 802 is used to identify the voice information of the current navigation operation to obtain user fault information, and to parse the context information of the current navigation operation to obtain device fault information. The device fault information includes one or more of the following: navigation interface screenshots, navigation operation screen recordings, and navigation logs. The work order module 803 is used to construct a fault work order based on the user fault information and equipment fault information, and send the fault work order to the defect tracking system to report the navigation fault.

[0132] In one embodiment of the present invention, the parsing module 802 is used to perform semantic analysis on the user fault information to obtain the classification label and / or priority of the navigation fault; The work order module 803 is used to determine the type of the fault work order based on the classification label and / or priority of the navigation fault, and fill in the user fault information and equipment fault information according to the type of the fault work order to generate a fault work order.

[0133] In one embodiment of the present invention, the parsing module 802 is used to perform semantic analysis on the user fault information to obtain a classification label for navigation faults; The work order module 803 is used to determine the ticket system included in the defect tracking system corresponding to the classification label of the navigation fault, and to send the fault work order to the defect tracking system in batches / in real time.

[0134] In one embodiment of the present invention, the parsing module 802 is used to generate the fault work order and / or send the fault work order to the defect tracking system, and then feed back a fault work order sending success message to indicate that the fault work order has been sent successfully.

[0135] Figure 9 An exemplary system architecture 900 for a navigation fault reporting method or navigation terminal to which embodiments of the present invention can be applied is shown.

[0136] like Figure 9As shown, the vehicle system architecture 900 may include various systems, such as a driving control system 901, a power system 902, a sensor system 903, a control system 904, a lane change assist system 905, one or more peripheral devices 906, a power supply 907, a computer system 908, and a user interface 909. The navigation fault reporting method provided in this embodiment can be implemented through interaction with the aforementioned systems, or through control of the systems by external devices, or through operation of the systems by a robot driving the vehicle. Optionally, the vehicle system architecture 900 may include more or fewer systems, and each system may include multiple components. Furthermore, each system and component of the vehicle system architecture 900 may be interconnected via wired or wireless means.

[0137] The vehicle system architecture 900 includes a driving control system 901, which can be in a fully or partially automated driving mode. For example, the driving control system 901 can automatically control the vehicle's movement based on control signals or control commands without human interaction, interaction with external devices, or interaction with a robot driving the vehicle.

[0138] The powertrain 902 may include components that provide power to the vehicle. For example, the powertrain 902 may include an engine, an energy source, a transmission, wheels, tires, etc. The engine may be an internal combustion engine, an electric motor, an air-compressed engine, or other combinations of engines, such as a hybrid engine consisting of a gasoline engine and an electric motor, or a hybrid engine consisting of an internal combustion engine and an air-compressed engine. The engine converts the energy source into mechanical energy to supply the transmission. Examples of energy sources may include gasoline, diesel, other petroleum-based fuels, propane, other compressed gas-based fuels, ethanol, solar panels, batteries, and other electrical sources. The energy source may also provide energy to other systems in the vehicle. Furthermore, the transmission may include a gearbox, a differential, a drive shaft, and a clutch, etc.

[0139] The sensor system 903 may include sensors for sensing the vehicle's surrounding environment (such as sensors for detecting the presence of obstacles) and pressure sensors for sensing the presence of passengers in the seats. Examples include a positioning system (which may be a Global Positioning System (GPS), BeiDou Navigation Satellite System, or other positioning systems), radar, a laser rangefinder, an inertial measurement unit (IMU), and cameras. The positioning system can be used to determine the vehicle's geographical location. The IMU is used to sense changes in the vehicle's position and orientation based on inertial acceleration. In one embodiment, the IMU may be a combination of an accelerometer and a gyroscope. The radar can use radio signals to sense objects in the vehicle's surrounding environment. In some embodiments, in addition to sensing objects, the radar can also be used to sense the speed and / or direction of travel of objects.

[0140] To detect environmental information and objects outside the vehicle, cameras can be configured at appropriate locations on the vehicle's exterior. For example, to acquire environmental images of the vehicle's sides, a camera can be mounted on the side mirror. The camera can be a still or video camera.

[0141] The control system 904 may include software systems for implementing vehicle driving control, such as systems for analyzing the vehicle's surrounding environment, pretensioning seat belts, route planning, obstacle avoidance, and image analysis. The control system 904 may also include hardware systems such as an accelerator, steering wheel system, seat belt system, airbag system, and peripheral devices (such as projection equipment and displays). Furthermore, the control system 904 may add or replace components other than those shown and described. Alternatively, some of the components shown above may be reduced.

[0142] In addition, the control system 904 can also interact with external sensors, other autonomous driving devices, other computer systems, or users via peripheral devices 906. Peripheral devices 906 may include wireless communication systems, on-board computers, microphones and / or speakers, cameras, and projectors, etc.

[0143] In some embodiments, peripheral device 906 provides a means for user interaction with the control system 904 via a user interface. For example, an onboard computer may provide information to a user of the vehicle. The user interface may also operate the onboard computer to receive user input. The onboard computer may be operated via a touchscreen. In other cases, peripheral device may provide a means for communicating with other devices located within the vehicle. For example, a microphone may receive audio (e.g., voice commands or other audio input) from a user of the control system. Similarly, a speaker may output audio to a user of the control system.

[0144] Wireless communication systems can communicate wirelessly with one or more devices, either directly or via a communication network. For example, wireless communication systems can use networks such as cellular networks, WiFi, and wireless local area networks (WLANs), or they can use infrared links, Bluetooth, or ZigBee to communicate directly with devices. Other wireless protocols include those used in various autonomous driving communication systems.

[0145] The power source 907 can provide power to various components of the vehicle. The power source 907 can be a rechargeable lithium-ion battery or a lead-acid battery.

[0146] The computer system 908 controls some or all of the vehicle control functions for ramp entry scenarios. The computer system 908 may include at least one processor that executes instructions stored in a non-transitory computer-readable medium such as memory. The computer system 908 provides the aforementioned control system with execution code that implements vehicle control for ramp entry scenarios.

[0147] The processor can be any conventional processor, such as a commercially available central processing unit (CPU). Alternatively, the processor can be a special-purpose device such as an application-specific integrated circuit (ASIC) or other hardware-based processor. Those skilled in the art will understand that the processor, computer, or memory can actually include multiple processors, computers, or memories that may or may not be stored in the same physical housing. For example, memory can be a hard disk drive or other storage media located in a housing different from that of a computer. Therefore, references to processors or computers will be understood to include references to a collection of processors or computers or memories that may or may not operate in parallel. Unlike using a single processor to perform the steps described herein, some components, such as steering and deceleration components, may each have their own processor that performs only determinations related to the component's specific function.

[0148] User interface 909 is used to provide information to or receive information from users of the vehicle. Optionally, user interface 909 may include one or more input / output devices within a set of peripheral devices 906, such as wireless communication systems, on-board computers, microphones, and speakers.

[0149] It should be understood that the components described above are merely an example. In actual applications, components in the various modules or systems mentioned above may be added or removed as needed. Figure 9 This should not be construed as a limitation on the embodiments of this application.

[0150] The following is for reference. Figure 10 It shows a schematic diagram of the structure of a computer system 1000 suitable for implementing a terminal device of the present invention. Figure 10 The terminal device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0151] like Figure 10 As shown, the computer system 1000 includes a central processing unit (CPU) 1001, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 1002 or programs loaded from storage section 1008 into random access memory (RAM) 1003. The RAM 1003 also stores various programs and data required for the operation of the system 1000. The CPU 1001, ROM 1002, and RAM 1003 are interconnected via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0152] The following components are connected to I / O interface 1005: an input section 1006 including a keyboard, mouse, etc.; an output section 1007 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN card, modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to I / O interface 1005 as needed. A removable medium 1011, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 1010 as needed so that computer programs read from it can be installed into storage section 1008 as needed.

[0153] In particular, according to the embodiments disclosed in this invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1009, and / or installed from removable medium 1011. When the computer program is executed by central processing unit (CPU) 1001, it performs the functions defined above in the system of this invention.

[0154] It should be noted that the computer-readable medium shown in this invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0155] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0156] The modules described in the embodiments of the present invention can be implemented in software or hardware. The described modules can also be housed in a processor; for example, a processor may be described as including a control module and an upload module. The names of these modules do not necessarily limit the module itself; for example, the upload module may also be described as "for generating a fault data packet based on the voice information and the context information, and uploading the fault data packet so that the cloud server generates a fault work order based on the fault data packet."

[0157] In another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiments; or it may exist independently and not assembled into the device. The computer-readable medium carries one or more programs, which, when executed by the device, cause the device to include: In response to the trigger operation of voice acquisition, the sound sensor is activated to acquire the user's voice to obtain voice information for the current navigation operation; In response to the triggering operation of the voice acquisition, multimodal data is acquired for the navigation operation on the display to obtain the context information of the current navigation operation. The context information includes one or more of the following: screenshot of the navigation interface, screen recording of the navigation operation, and navigation log. A fault data packet is constructed based on the voice information and the context information, and the fault data packet is uploaded so that the cloud server can generate a fault work order based on the fault data packet.

[0158] According to the technical solution of this invention, in response to the trigger operation of voice acquisition, the context information of the current navigation operation is obtained while acquiring the user's voice, so as to construct a fault data packet, upload it to the cloud server, and generate a fault work order. This allows users to report navigation faults in a timely manner during navigation operations, which helps to speed up the handling of navigation faults.

[0159] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention. It should be noted that the acquisition, storage, and application of user personal information involved in the technical solutions of this disclosure comply with relevant laws and regulations and do not violate public order and good morals.

Claims

1. A method for reporting navigation faults, characterized in that, Applications in vehicles include: In response to the trigger operation of voice acquisition, the sound sensor is activated to acquire the user's voice to obtain voice information for the current navigation operation; In response to the triggering operation of the voice acquisition, multimodal data is acquired for the navigation operation on the display to obtain the context information of the current navigation operation. The context information includes one or more of the following: screenshot of the navigation interface, screen recording of the navigation operation, and navigation log. A fault data packet is constructed based on the voice information and the context information, and the fault data packet is uploaded so that the cloud server can generate a fault work order based on the fault data packet.

2. The method for reporting navigation faults according to claim 1, characterized in that, After the sound sensor is activated to collect user voice data, the method further includes: The sound sensor stops collecting user voice after a preset collection time. The preset collection time includes a preset time after the user's voice ends or a preset time after the user's voice begins. And / or, Receive the fault work order sending success message sent by the cloud server, and display or broadcast the fault work order sending success message.

3. The method for reporting navigation faults according to claim 1, characterized in that, The method further includes: In response to the trigger operation of the voice acquisition, the vehicle-mounted sensors are activated to acquire vehicle parameters, and the vehicle parameters are added to the context information. The vehicle parameters include one or more of the following: current vehicle location, planned route, vehicle speed, traffic conditions, navigation device identifier, navigation application version identifier, and network connection status. And / or, Before generating the fault data packet based on the voice information and the context information, the method further includes: The voice information and the context information are compressed and / or encrypted.

4. A method for reporting navigation faults, characterized in that, Applications in cloud servers, including: Receive a fault data packet, the fault data packet including voice information of the current navigation operation collected in response to the trigger operation, and context information of the current navigation operation collected synchronously with the voice information; User fault information is obtained by recognizing the voice information of the current navigation operation, and device fault information is obtained by parsing the context information of the current navigation operation. The device fault information includes one or more of the following: navigation interface screenshot, navigation operation screen recording, and navigation log. Based on the user fault information and equipment fault information, a fault work order is constructed and sent to the defect tracking system to report the navigation fault.

5. The method for reporting navigation faults according to claim 4, characterized in that, The method further includes: Semantic analysis is performed on the user fault information to obtain the classification labels and / or priorities of navigation faults; The process of constructing a fault work order based on the user fault information and equipment fault information includes: The type of the fault work order is determined based on the classification label and / or priority of the navigation fault, and the user fault information and equipment fault information are filled in according to the type of the fault work order to generate the fault work order.

6. The method for reporting navigation faults according to claim 4, characterized in that, The method further includes: Semantic analysis is performed on the user fault information to obtain classification labels for navigation faults; Sending the fault work order to the defect tracking system includes: The ticket system, which is included in the defect tracking system corresponding to the classification label of the navigation fault, is determined, and the fault work order is sent to the defect tracking system in batches or in real time.

7. The method for reporting navigation faults according to claim 4, characterized in that, The method further includes: After generating the fault work order and / or sending the fault work order to the defect tracking system, feedback information indicating successful sending of the fault work order is provided to indicate that the fault work order was successfully sent.

8. A navigation terminal, characterized in that, The navigation terminal, used in vehicles, includes: The control module, in response to the trigger operation of voice acquisition, activates the sound sensor to acquire the user's voice in order to obtain voice information for the current navigation operation; In response to the triggering operation of the voice acquisition, multimodal data is acquired for the navigation operation on the display to obtain the context information of the current navigation operation. The context information includes one or more of the following: screenshot of the navigation interface, screen recording of the navigation operation, and navigation log. The upload module is used to generate a fault data packet based on the voice information and the context information, and upload the fault data packet so that the cloud server can generate a fault work order based on the fault data packet.

9. A vehicle, characterized in that, Including the navigation terminal as described in claim 8.

10. A server, characterized in that, include: A receiving module is used to receive fault data packets, the fault data packets including voice information of the current navigation operation collected in response to a trigger operation, and context information of the current navigation operation collected synchronously with the voice information; The parsing module is used to identify the voice information of the current navigation operation to obtain user fault information, and to parse the context information of the current navigation operation to obtain device fault information. The device fault information includes one or more of the following: navigation interface screenshots, navigation operation screen recordings, and navigation logs. The work order module is used to construct a fault work order based on the user fault information and equipment fault information, and send the fault work order to the defect tracking system to report the navigation fault.

Images