A method and system for user feedback

Abstract

The embodiments of this specification disclose a method and system for user feedback, the method comprising: obtaining status information, the status information comprising at least one of a vehicle status and a user status; sending a first reminder message to a user terminal based on the status information; obtaining feedback information on the driving condition of a vehicle input by the user at the user terminal in response to the first reminder message, the vehicle being the vehicle in which the user is riding.

Description

Technical Field

[0001] The embodiments of the present specification relate to the field of vehicle control technology, and in particular, to a method and system for user feedback. Background Art

[0002] With the continuous development of computer technology, smart travel (for example, shared travel or autonomous driving, etc.) has also developed rapidly. Taking autonomous driving as an example, an autonomous driving vehicle can refer to a vehicle that can achieve a certain level of driving automation. For example, an autonomous driving vehicle can be controlled by a system (such as background remote control) to achieve autonomous driving. Autonomous driving vehicles can be used as taxis or public transportation. While autonomous driving vehicles provide services to humans, how to collect user feedback on the vehicle in a timely manner to optimize the automatic control of the vehicle and better meet the different needs of users is an urgent problem to be solved. Summary of the invention

[0003] One aspect of an embodiment of the present specification provides a method for user feedback, comprising: obtaining status information, the status information including at least one of a vehicle status and a user status; sending a first reminder message to a user terminal based on the status information; and obtaining feedback information on the driving condition of a vehicle input by the user at the user terminal in response to the first reminder message, the vehicle being the vehicle in which the user is riding.

[0004] One aspect of an embodiment of the present specification provides a user feedback system, including: a first acquisition module, used to acquire status information, the status information including at least one of a vehicle status and a user status; a sending module, used to send a first reminder message to a user terminal based on the status information; and a second acquisition module, used to acquire feedback information on the driving condition of a vehicle input by the user at the user terminal in response to the first reminder message, the vehicle being the vehicle in which the user rides.

[0005] One aspect of an embodiment of the present specification provides a user feedback device, the device comprising at least one processor and at least one memory; the at least one memory is used to store computer instructions; the at least one processor is used to execute at least part of the computer instructions to implement operations corresponding to the user feedback method described in any of the above items.

[0006] One aspect of an embodiment of the present specification provides a computer-readable storage medium, wherein the storage medium stores computer instructions, and when the computer instructions are executed by a processor, operations corresponding to the user feedback method described in any one of the above are implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] This specification will be further described in the form of exemplary embodiments, which will be described in detail by the accompanying drawings. These embodiments are not restrictive, and in these embodiments, the same number represents the same structure, wherein:

[0008] Figure 1 is a schematic diagram of an application scenario of a user feedback system according to some embodiments of this specification;

[0009] Figure 2 is a flow chart of a user feedback method according to some embodiments of this specification;

[0010] Figure 3 is a flowchart of determining state information according to some embodiments of this specification;

[0011] Figure 4 is a flowchart of sending a first reminder message according to some embodiments of this specification;

[0012] Figure 5 is a flowchart of predicting a target position according to some embodiments of this specification;

[0013] Figures 6A-6D is a diagram of a user interface according to some embodiments of the present specification. DETAILED DESCRIPTION

[0014] In order to more clearly illustrate the technical solutions of the embodiments of this specification, the following is a brief introduction to the drawings required for the description of the embodiments. Obviously, the drawings described below are only some examples or embodiments of this specification. For ordinary technicians in this field, this specification can also be applied to other similar scenarios based on these drawings without creative work. Unless it is obvious from the language environment or otherwise explained, the same reference numerals in the figures represent the same structure or operation.

[0015] It should be understood that the "system", "device", "unit" and / or "module" used in this specification is a method for distinguishing different components, elements, parts, parts or assemblies at different levels. However, if other words can achieve the same purpose, the words can be replaced by other expressions.

[0016] As shown in this specification and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "comprise" and "include" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0017] Flowcharts are used in this specification to illustrate the operations performed by the system according to the embodiments of this specification. It should be understood that the preceding or following operations are not necessarily performed precisely in order. Instead, the steps may be processed in reverse order or simultaneously. At the same time, other operations may also be added to these processes, or one or more operations may be removed from these processes.

[0018] Figure 1 1 is a schematic diagram of an application scenario of a user feedback system according to some embodiments of this specification. In some embodiments, the user feedback system 100 may include a server 110, a network 120, a user terminal 130, a database 140, and a vehicle 150. The server 110 may include a processing device 112.

[0019] In some embodiments, the server 110 can be used to process information and / or data related to user feedback. The server 110 can be an independent server or a server group. The server group can be centralized or distributed (e.g., the server 110 can be a distribution system). In some embodiments, the server 110 can be regional or remote. For example, the server 110 can access information and / or data stored in the user terminal 130, the database 140, the vehicle 150, and the detection unit 152 through the network 120. In some embodiments, the server 110 can be directly connected to the user terminal 130, the database 140, the vehicle 150, and the detection unit 152 to access the information and / or data stored therein. In some embodiments, the server 110 can be executed on a cloud platform.

[0020] In some embodiments, the server 110 may include a processing device 112. The processing device 112 may process data and / or information related to user feedback to perform one or more functions described in the specification. For example, the processing device 112 may obtain state information. For another example, the processing device 112 may send a first reminder message to the user terminal based on the state information to obtain feedback information input by the user on the driving condition of the vehicle in the user terminal in response to the first reminder message.

[0021] In some embodiments, the processing device 112 may include one or more sub-processing devices (e.g., a single-core processing device or a multi-core multi-core processing device). As an example only, the processing device 112 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), an application-specific instruction processor (ASIP), a graphics processing unit (GPU), a physical processing unit (PPU), a digital signal processor (DSP), a field programmable gate array (FPGA), a programmable logic circuit (PLD), a controller, a microcontroller unit, a reduced instruction set computer (RISC), a microprocessor, etc., or any combination thereof. In some embodiments, the processing device 122 may be integrated into the vehicle 150 and / or the user terminal 130.

[0022] The network 120 can facilitate the exchange of data and / or information. In some embodiments, one or more components in the system 100 (e.g., the server 110, the user terminal 130, the database 140, the vehicle 150, the detection unit 152) can send data and / or information to other components through the network 120. In some embodiments, the network 120 can be any type of wired or wireless network. For example, the network 120 can include a cable network, a wired network, a fiber optic network, a telecommunications network, an internal network, an Internet network, a local area network (LAN), a wide area network (WAN), a wireless area network (WLAN), a metropolitan area network (MAN), a public switched telephone network (PSTN), a Bluetooth network, a ZigBee network, a near field communication (NFC) network, etc. or any combination thereof. In some embodiments, the network 120 can include one or more network entry and exit points. For example, the network 120 can include a wired or wireless network entry and exit point, such as a base station and / or an Internet exchange point 120-1, 120-2, ..., through which one or more components of the system 100 can connect to the network 120 to exchange data and / or information.

[0023] In some embodiments, the user terminal 130 may be a terminal for the user to input feedback information on the driving condition of the vehicle. In some embodiments, the user may be a service user. For example, service users may include passengers of online car-hailing platforms, passengers of autonomous vehicles, navigation service users, and transportation service users. In some embodiments, the user terminal 130 may include one or any combination of a mobile device 130-1, a tablet computer 130-2, a laptop computer 130-3, a vehicle-mounted device (not shown), a motor vehicle built-in device (not shown), a wearable device (not shown), etc. In some embodiments, the mobile device 130-1 may include a smart home device, a smart mobile device, a virtual reality device, an augmented reality device, etc., or any combination thereof. The smart mobile device may include a smart phone, a personal digital assistant (PDA), a gaming device, a navigation device, a point of sale (POS) device, etc., or any combination thereof. The virtual reality device or augmented reality device may include a virtual reality helmet, virtual reality glasses, virtual reality goggles, an augmented reality helmet, augmented reality glasses, augmented reality goggles, etc., or any combination thereof. In some embodiments, the motor vehicle built-in device may include a vehicle-mounted computer, a vehicle-mounted TV, etc. In some embodiments, the wearable device may include a smart bracelet, smart shoes and socks, smart glasses, smart helmets, smart watches, smart clothing, smart backpacks, smart accessories, etc. or any combination thereof. In some embodiments, the user terminal 130 may be a device with positioning technology for locating the position of the user terminal 130.

[0024] The database 140 may store data and / or instructions. In some embodiments, the database 140 may store data obtained from components such as the user terminal 130, the vehicle 150 detection unit 152, and the processing device 112. In some embodiments, the database 140 may store information and / or instructions for execution or use by the server 110 to execute the exemplary method described in this specification. For example, the database 140 may store the current vehicle state (such as bumps, speed, acceleration, etc.) collected by the detection unit 152. For another example, the database 140 may store feedback information input by the user at the user terminal 130. For another example, the database 140 may also store the target location determined by the processing device 112, etc. In some embodiments, the database 140 may include a large-capacity memory, a removable memory, a volatile read-write memory (such as a random access memory RAM), a read-only memory (ROM), etc., or any combination thereof. In some embodiments, the database 140 may be implemented on a cloud platform. For example, the cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud, a decentralized cloud, an internal cloud, etc., or any combination thereof.

[0025] In some embodiments, the database 140 may be connected to the network 120 to communicate with one or more components of the system 100 (e.g., the server 110, the user terminal 130, the vehicle 150, the detection unit 152, etc.). One or more components of the system 100 may access the data or instructions stored in the database 140 through the network 120. For example, the server 110 may obtain the user status from the database 140 and perform corresponding processing. In some embodiments, the database 140 may be directly connected or communicated with one or more components of the system 100 (e.g., the server 110, the user terminal 130). In some embodiments, the database 140 may be a part of the server 110. In some embodiments, the database 140 may be integrated into the vehicle 110.

[0026] Vehicle 150 can be any type of driving vehicle, for example, an autonomous vehicle, a vehicle driven by a ride-hailing driver, etc. In some embodiments, vehicle 150 can be a vehicle that a user rides in. As used herein, an autonomous vehicle can refer to a vehicle that can achieve a certain level of driving automation. Exemplarily, the driving automation level can include a first level, i.e., the vehicle is mainly supervised by a person and has a specific autonomous function (e.g., autonomous steering or acceleration), a second level, i.e., the vehicle has one or more advanced driver assistance systems (ADAS, e.g., adaptive cruise control system, lane keeping system), which can control braking, steering and / or acceleration of the vehicle, a third level, i.e., the vehicle can automatically drive when one or more certain conditions are met, a fourth level, i.e., the vehicle can operate without human input or negligence, but is still subject to certain restrictions (e.g., limited to a certain area), a fifth level, i.e., the vehicle can operate autonomously in all situations, etc., or any combination thereof.

[0027] In some embodiments, the vehicle 150 may have an equivalent structure that enables the vehicle 150 to move or fly. For example, the vehicle 150 may include the structure of a traditional vehicle, such as a chassis, a suspension, a steering device (e.g., a steering wheel), a braking device (e.g., a brake pedal), an accelerator, etc. For another example, the vehicle 150 may have a body and at least one wheel. The body may be any type of body, such as a sports car, a coupe, a sedan, a light truck, a station wagon, a sports vehicle (SUV), a minivan, or a conversion vehicle. At least one wheel may be configured as all-wheel drive (AWD), front-wheel drive (FWR), rear-wheel drive (RWD), etc. In some embodiments, it is expected that the vehicle 150 may be an electric vehicle, a fuel cell vehicle, a hybrid vehicle, a traditional internal combustion engine vehicle, etc.

[0028] In some embodiments, the vehicle 150 can sense its environment and travel by using one or more detection units 152. At least two detection units 152 may include sensor devices (e.g., radars (e.g., laser radar devices)), global positioning system (GPS) modules, inertial measurement units (IMUs), cameras, etc., or any combination thereof. The radar (e.g., laser radar device) may be configured to scan the surroundings of the vehicle 150 and generate corresponding data. The GPS module may refer to a device that can receive geolocation and time information from GPS satellites and determine the geographic location of the device. IMU may refer to an electronic device that uses various inertial sensors to measure and provide specific forces, angular rates, and sometimes magnetic fields around the vehicle. In some embodiments, various inertial sensors may include acceleration sensors (e.g., piezoelectric sensors), speed sensors (e.g., Hall sensors), distance sensors (e.g., radars, infrared sensors), steering angle sensors (e.g., tilt sensors), traction-related sensors (e.g., force sensors), etc. The camera may be configured to acquire one or more images related to a target (e.g., a person, an animal, a tree, a roadblock, a building, or a vehicle) within the camera range.

[0029] It will be appreciated by those skilled in the art that when an element (or component) of the user feedback system 100 is executed, the element may be executed by an electrical signal and / or an electromagnetic signal. For example, when the user terminal 130 sends feedback information to the server 110, the processor of the user terminal 130 may generate an electrical signal encoding the feedback information. Then, the processor of the user terminal 130 may send the electrical signal to an output port. If the user terminal 130 communicates with the server 110 via a wired network, the output port may be physically connected to a cable, which may also send the electrical signal to an input port of the server 110. If the user terminal 130 communicates with the server 110 via a wireless network, the output port of the user terminal 130 may be one or more antennas that convert the electrical signal into an electromagnetic signal. In an electronic device such as the user terminal 130 and / or the server 110, when its processor processes instructions, issues instructions and / or performs actions, the instructions and / or actions are performed by electrical signals. For example, when the processor retrieves or saves data from a storage medium (e.g., database 140), it can send an electrical signal to a read / write device of the storage medium, which can read or write structured data in the storage medium. The structured data can be transmitted to the processor in the form of an electrical signal via a bus of the electronic device. Here, the electrical signal can refer to one electrical signal, a series of electrical signals, and / or at least two discontinuous electrical signals.

[0030] In some embodiments, a processing device (eg, processing device 112 ) may include a first acquisition module, a sending module, and a second acquisition module.

[0031] The first acquisition module can be used to acquire state information, and the state information includes at least one of a vehicle state and a user state. For specific details of the state information, please refer to step 210 and its related description, which will not be repeated here.

[0032] The sending module can be used to send the first reminder information to the user terminal based on the status information.

[0033] In some embodiments, the sending module can be further used to determine a feedback reminder method, the feedback reminder method is related to at least one of the user status, the user terminal status, the driving environment and the user-related information; and generate the first reminder information based on the feedback reminder method, and send the first reminder information to the user terminal. For details about sending the first reminder information, please refer to Figure 4 And the related description will not be repeated here.

[0034] The second acquisition module may be used to acquire feedback information on the driving condition of the vehicle inputted by the user at the user terminal in response to the first reminder information.

[0035] It should be noted that the above description of the user feedback system and its modules is only for the convenience of description, and this specification cannot be limited to the scope of the embodiments. It is understandable that for those skilled in the art, after understanding the principle of the system, it is possible to arbitrarily combine the various modules, or form a subsystem to connect with other modules without deviating from this principle. In some embodiments, the disclosed first acquisition module, sending module and second acquisition module can be different modules in a system, or a module can implement the functions of two or more modules mentioned above. For example, each module can share a storage module, or each module can have its own storage module. Such variations are all within the scope of protection of this specification.

[0036] Figure 2 is a flow chart of a user feedback method according to some embodiments of this specification. Figure 2 As shown, the process 200 may include the following steps 210, 220, and 230. In some embodiments, the process 200 may be executed by a processing device (eg, the processing device 112).

[0037] Step 210, obtaining state information, where the state information includes at least one of a vehicle state and a user state. In some embodiments, step 210 may be performed by a first obtaining module.

[0038] In some embodiments, the vehicle state is related to the driving condition of the vehicle. For specific details about the driving condition of the vehicle, please refer to step 230 and its related description, which will not be repeated here. The vehicle state may include information such as driving parameters, driving state, driving environment, driving section, driving position, driving route, etc.

[0039] In some embodiments, the vehicle status may include at least one of a current vehicle status, a vehicle status after a preset time, and a vehicle status at a target location.

[0040] In some embodiments, the current vehicle state may reflect part or all of the current driving condition of the vehicle. For example, it may reflect the navigation route the vehicle is using, the road section or position the vehicle is currently in, the current driving condition of the vehicle, the current driving parameters of the vehicle, etc. For specific details about the driving condition, please refer to step 230 and its related description, which will not be repeated here.

[0041] In some embodiments, different data of the current vehicle status can be obtained in different ways. For example, the current driving parameters can be obtained by sensors installed in the vehicle (e.g., speed sensors and acceleration sensors). For another example, the current driving status can be obtained by a monitoring system installed in the vehicle. For another example, the current weather environment and the current time environment are obtained from a weather service platform. For another example, the road condition environment can be reported by users of other vehicles, or obtained from a map or navigation service platform. The current driving section and the current driving position can be obtained through positioning technology, etc.

[0042] The vehicle state after the preset time may reflect part or all of the driving condition of the vehicle after the preset time. The vehicle state at the target location may reflect part or all of the driving condition of the vehicle at the target location. For specific details about the vehicle state after the preset time and the vehicle state at the target location, please refer to step 220 and its related description, which will not be repeated here.

[0043] In some embodiments, the user status is related to one or more of the following information: the user's vehicle usage, the user's free time, and the user's mood. In some embodiments, the user status may include the current user status, the user status after a preset time, and the user status at the target location.

[0044] The user's vehicle usage can reflect the status relationship between the user and the vehicle, for example, the user has boarded the vehicle, the user is waiting for the vehicle to pick him up, the user is about to end using the vehicle (or the user will end using the vehicle after a certain time), etc.

[0045] The user's free time can at least reflect whether the user has time to provide feedback. For example, the user's free time can include very busy, relatively busy, free, etc. The busy level can be set according to needs. For example, if the user is on a phone call, it can be considered very busy. If the user is using the mobile phone for entertainment (such as chatting, playing games, etc.), it can be considered relatively busy, etc.

[0046] The user's emotions may reflect the user's feelings about the driving condition of the vehicle. For example, the user's emotions may include whether he is angry, uncomfortable, panicked, happy, helpless, depressed, etc.

[0047] In some embodiments, the processing device may obtain current status information based on the user's image information and / or the vehicle's audio information. For example, obtain the current user status. For details on obtaining status information, see Figure 3 And the related description will not be repeated here.

[0048] In some embodiments, the processing device may determine the user's emotions based on the data collected by the wearable device (e.g., heartbeat, heart rate, pulse, and blood pressure). For example, if the number of heartbeats within a preset time exceeds a preset threshold, the user's emotions may be nervous. In some embodiments, the processing device may determine the current user state based on the state of the user terminal. For example, the current user state and user emotions are determined based on the web pages browsed or the information viewed by the user.

[0049] The methods of different embodiments can be combined. For example, in some embodiments, the processing device can obtain the user's emotional state based on the data monitored by the wearable device, and obtain the vehicle state based on the storage device; in some embodiments, the processing device can obtain the user's vehicle usage or idle state based on the image data and / or audio data, and obtain the driving parameters based on the sensor.

[0050] Step 220: Send first reminder information to the user terminal based on the state information. In some embodiments, step 220 may be performed by a sending module.

[0051] The first reminder information may be information for guiding the user terminal to provide feedback on the driving condition. The first reminder information includes the content and method of the first reminder information.

[0052] In some embodiments, the content of the first reminder information can be preset. In some embodiments, the content of the first reminder information can be determined based on the driving condition for which the feedback is given. For example, if the sensor of the vehicle detects bumps, the content of the first reminder information may include "please provide feedback on whether the vehicle is stable". For another example, if the target position Q is a turning section, the content of the first reminder information may include "please provide feedback on whether the turn at position Q is stable". In some embodiments, the content of the first reminder information may be determined based on the user state. For example, if the user's state is obtained as body shaking, the content of the first reminder information may include "please provide feedback on whether the vehicle brakes suddenly". In some embodiments, the content of the first reminder information may be determined in combination with the user state and the driving condition for which the feedback is given. For example, if no abnormality of the vehicle is found, but the user is angry, the content of the first reminder information may include "please provide feedback on whether you are satisfied with the current driving condition of the vehicle", etc. Different user states and / or driving conditions for which the feedback is given may set the content of the corresponding first reminder information. After the processing device obtains the user state and / or the driving condition for which the feedback is given, the corresponding content may be directly obtained.

[0053] The first reminder information can be expressed in a variety of ways. For example, the first reminder information can be voice, text, image, video, vibration, etc. For example, the user terminal can display an icon of "click to like" or "click to like", and the icon is used as the first reminder information.

[0054] In some embodiments, the processing device may determine the representation method of the first reminder information according to at least one of the user status, the user terminal status, the driving environment and the user association information. Figure 4 and its related description.

[0055] In some embodiments, the processing device may send the first reminder information to the user terminal based on a preset rule. The preset rule may be specifically set according to actual conditions.

[0056] In some embodiments, the preset rule may include sending the first reminder message at a preset interval. For example, the first reminder message is sent every 5 minutes. The preset time can be set according to the actual situation to avoid poor riding experience of users due to frequent sending of reminder messages, or incomplete feedback information collection due to untimely reminders.

[0057] In some embodiments, the preset rules may include sending the first reminder message at intervals of a preset driving distance. For example, sending the first reminder message at intervals of 50 m of driving distance. The preset rules may also include sending the first reminder message at a preset location. For example, at a red, yellow, or green light. In some embodiments, the preset rules may be adjusted and optimized according to driving conditions. For example, if the weather is rainy or snowy, the preset time or preset driving distance may be shortened.

[0058] The methods of different embodiments can be combined. For example, in some embodiments, the processing device can send the content of the first reminder information related to the vehicle speed to the wearable device worn by the user, and the wearable device can vibrate to prompt. In some embodiments, the processing device can send the first reminder information of "Please provide feedback on the current driving status" to the user terminal at a preset time interval, and the first reminder information can be sent in different ways each time, such as voice, image, text, etc.

[0059] In some embodiments, the processing device may send a first reminder message to the user terminal based on the user portrait. The user portrait may reflect the interests and needs of the user. The user portrait may be determined based on user-related information. For more information about user-related information, please refer to the relevant description below. For example, when the user portrait reflects that the user is interested in the speed of the vehicle, the processing device may send a first reminder message related to the speed of the vehicle. Through the user portrait, reminder messages that the user is more interested in can be sent to increase the user's interest in feedback, thereby obtaining more feedback information.

[0060] In some embodiments, the processing device may determine whether to send the first reminder information to the user terminal, or / and determine the content of the first reminder information to be sent, based on the vehicle state and / or the user state. In some embodiments, the processing device may determine whether to send the first reminder information, or / and determine the content of the first reminder information to be sent, based on current state information (including the current vehicle state and / or the current user state).

[0061] In some embodiments, the processing device may determine whether to send the first reminder information and its content based on the current vehicle state. The processing device may determine whether to send the first reminder information and its content based on a type of information about the current vehicle state. For example, the processing device may determine whether to send the first reminder information based on whether the current vehicle state is an abnormal driving state in which a driving event occurs. Specifically, if the current vehicle state is an abnormal driving state, the first reminder information is sent. The processing device may determine the content of the first reminder information according to the type of driving event of the abnormal driving state. For example, if the current vehicle state has a driving event "bumping", the processing device may send the first reminder information "please feedback whether the vehicle is bumpy". The processing device may determine whether to send the first reminder information and its content based on the current vehicle's driving section. For example, if the vehicle is in a turning section, the first reminder information "please feedback whether the vehicle turns smoothly" is sent. The processing device may determine whether to send the first reminder information and its content based on the current vehicle's driving environment. For example, when the current vehicle's road condition environment is congested, the processing device may send the first reminder information "please feedback whether the vehicle frequently brakes".

[0062] In some embodiments, the processing device may determine whether to send the first reminder message and the content of the first reminder message based on multiple information about the current vehicle status. For example, the processing device may determine whether to send the first reminder message based on preset rules and in combination with multiple information about the current vehicle status. The preset rule may be to score the multiple information about the current vehicle status separately, and then fuse the scores corresponding to the multiple information (e.g., averaging, weighted summing, weighted averaging, etc.), and determine whether to send the first reminder message based on the fused score (e.g., if the fused score is greater than a certain threshold, then send the first reminder message). The content of the first reminder message can be determined based on the information with a score greater than a threshold in the multiple information.

[0063] In some embodiments, the processing device can process various information of the current vehicle status based on the trained first reminder model to determine whether to send the first reminder information and the content of the first reminder information. The processing device inputs various information into the first reminder model, and outputs whether to send the first reminder information and the content theme of the first reminder information. Before the various information is input into the model, the processing device can pre-process the various information, including screening, vector representation, vector normalization, etc. The content theme refers to the driving condition that the user feedback needs to target. Content themes may include: bumps, speeding, sudden braking, etc. The content of the corresponding first reminder information can be pre-set based on the content theme.

[0064] The trained first reminder model can be obtained based on a plurality of first training samples, wherein the first training sample includes a sample vehicle state, and a label includes whether the first reminder message needs to be sent, and the content of the first reminder message. The first reminder model can include but is not limited to a linear regression model, a neural network model, a decision tree, a support vector machine, and a naive Bayes.

[0065] In some embodiments, the processing device may determine whether to send the first reminder message and / or determine the content of the first reminder message based on the current user state. In some embodiments, the processing device may determine whether to send the first reminder message and its content based on a type of information about the current user state. For example, when the user's idle status reflects that the user is idle, the processing device may send a first reminder message "Please evaluate the vehicle's autonomous driving technology." For another example, when the user's emotion reflects that the user's emotion is panic, the processing device may send a first reminder message "Is the vehicle stable?" or "Is the vehicle braking suddenly?", etc. For another example, when the user's vehicle usage reflects that the user has gotten in the car, the processing device may send a first reminder message "Is the vehicle starting smoothly?", etc.

[0066] In some embodiments, the processing device may determine whether to send the first reminder message and the content of the first reminder message based on various information about the current user status. For example, the processing device may determine whether to send the first reminder message and its content based on a preset rule in combination with various information about the current user status. For example, if the user's vehicle usage reflects that the user has boarded the vehicle and the user is busy, the processing device may not need to send the first reminder message. If the user's emotion is panic, but the user is relatively busy, the processing device may send the first reminder message.

[0067] In some embodiments, the processing device may determine whether to send the first reminder message and its content based on one or more information in the current user state and one or more information in the current vehicle state. In some embodiments, the processing device may use preset rules to determine whether to send the first reminder message and the content of the first reminder message based on the current user state and the current vehicle state. The preset rules can be set according to demand. For example, if the vehicle's current driving section is a turning section and the user is in a panic, the processing device may send a first reminder message "Is the current turn smooth?" For another example, if the vehicle's current driving state is speeding and the user is in a uncomfortable mood, the processing device may send a first reminder message "Is the current vehicle speed too fast?" For another example, if the user is in a uncomfortable mood and the vehicle's current in-vehicle ambient temperature is high, the processing device may send a first reminder message "Is the current in-vehicle temperature too high?"

[0068] In some embodiments, the processing device may determine whether to send the first reminder information and its content based on the current user state and the current vehicle state using the second reminder model. The second reminder model is similar to the first reminder model.

[0069] In some embodiments, compared with the first reminder model, the content theme output by the second reminder model may also be related to the user's emotions. Correspondingly, the content of the first reminder information may include content that matches the user's emotions. For example, if the user is angry, the content of the first reminder information may include content that comforts the user.

[0070] By comprehensively considering the current vehicle state and the current user state to determine whether to send the first reminder information, a more accurate assessment of the current driving condition of the vehicle can be ensured. This allows the user to be reminded in a timely manner when user feedback is required, providing a basis for optimizing subsequent driving. Moreover, interference to the user can be avoided.

[0071] In some embodiments, the processing device may determine whether to send the first reminder message based on the user's willingness to provide feedback. For example, when the user's willingness to provide feedback reflects that the user is unwilling to provide feedback, the processing device does not send the first reminder message to the user terminal. The user's willingness to provide feedback can be determined based on user-related information. User-related information refers to information related to the user. User-related information includes: user basic information and user historical behavior information. User basic information includes user credit rating, user gender, age, occupation, hobbies, length of time the user has registered on the platform, user feedback intention and degree of feedback intention, etc. User historical behavior information includes: user historical order information, user historical evaluation information, and user historical feedback information, etc. In some embodiments, the processing device may obtain user-related information from a storage device or platform (for example, an online car-hailing platform).

[0072] The methods of different embodiments may be combined. For example, in some embodiments, the processing device may determine whether to send the first reminder information and the content of the first reminder information based on the vehicle state and the user profile. For example, the user profile reflects that the user is sensitive to speed, and when the vehicle's driving speed is detected to be speeding, the processing device may send the first reminder information.

[0073] The methods of different embodiments can be combined. For example, in some embodiments, the processing device can send the first reminder information to the vehicle terminal in the form of text based on the vehicle status; in some embodiments, the processing device can send the first reminder information to the user's wearable device in the form of vibration based on the user status; in some embodiments, the processing device can send the first reminder information to the user's mobile phone in the form of an image based on the vehicle status and the user status, and after receiving the first reminder information, the user clicks on the icon to provide feedback.

[0074] In some embodiments, the processing device may determine whether to send the first reminder information, and the content and / or method of the first reminder information based on subsequent status information. The subsequent status information includes the vehicle status after a preset time, the user status after a preset time, the vehicle status at the target location, and / or the user status at the target location.

[0075] The vehicle state after the preset time may be the vehicle state after the preset time starting from the current time. The preset time may be one or more. The preset time may be preset, for example, 10 minutes, 20 minutes, etc. The preset time may be determined according to the driving condition, the user state and / or the user-related information. For example, if the current road is congested, the preset time may be extended. For another example, if it is determined according to the user-related information that the user is not positive about the feedback, the preset time may be extended.

[0076] In some embodiments, the vehicle state after a preset time can be predicted and obtained. The vehicle state after the preset time is predicted based on the current vehicle state and the preset time. For example, the prediction of whether there is a bumpy driving event after the preset time can be determined based on the possible driving position of the vehicle after the preset time. The predicted driving position after the preset time can be determined based on the current driving speed and driving route.

[0077] In some embodiments, the prediction may be implemented by a first vehicle state prediction model. The first vehicle state prediction model may process the current vehicle state and a preset time to obtain the vehicle state after the preset time.

[0078] The second training sample used in the training of the first vehicle state prediction model includes the current vehicle state of the sample and the preset time of the sample, and the label represents the vehicle state after the preset time. The first vehicle state prediction model may include but is not limited to a linear regression model, a neural network model, a decision tree, a support vector machine, and a naive Bayesian.

[0079] In some embodiments, the input of the first vehicle state prediction model may include, in addition to the preset time and the current vehicle state, the navigation route currently used by the vehicle (ie, the first navigation route), etc. Correspondingly, the second training sample also includes the sample first navigation route.

[0080] In some embodiments, the processing device may also optimize or update the vehicle state after the preset time through the historical vehicle driving conditions at the location where the vehicle arrives after the preset time. For example, the processing device predicts that the vehicle will arrive at location X after the preset time, and the vehicle state after the preset time predicted by the processing device includes an abnormal driving state of "bumping", and the historical vehicle driving conditions at location X that exceed the threshold do not include "bumping", then the vehicle state after the preset time may be updated to "not bumpy". By optimizing or updating the vehicle state after the preset time through the historical vehicle driving conditions at the location where the vehicle arrives after the preset time, the vehicle state after the preset time can be obtained more in line with the actual situation through the driving conditions of multiple historical vehicles, thereby improving the prediction accuracy of the vehicle state after the preset time.

[0081] The user status after the preset time is similar to the vehicle status after the preset time, which will not be repeated here. In some embodiments, predicting the user status after the preset time can be achieved by a first user status prediction model. The first user status prediction model can process the current user status and the preset time to obtain the user status after the preset time. The first user status prediction model can also process the user status, the preset time and the user-related information to obtain the user status after the preset time. For user-related information, please refer to the relevant description above. The first user status prediction model is similar to the first vehicle status prediction model, which will not be repeated here.

[0082] The target position in the vehicle state at the target position or the user state at the target position may be a preset position, for example, a position preset or determined by a user terminal, a driving terminal or a processing device, etc. The target position may include: a point, an area or a road section.

[0083] The target location may be one or more. In some embodiments, the target location may be set based on a preset rule. The preset rule may be to determine a target location at every preset distance. The preset rule may also be to use the location where a driving event may occur as the target location. For example, a turning location, a pedestrian congestion (such as a school, etc.) section or location, etc. are used as the target location. The processing device, user terminal or driving terminal may obtain the navigation route (i.e., the first navigation route) that the vehicle is currently traveling, and determine the location where a driving event may occur from the navigation route as the target location.

[0084] In some embodiments, the target position may be predicted based on feedback information, historical information, etc. For specific details on predicting the target position, please refer to step 520 and related descriptions, which will not be repeated here.

[0085] In some embodiments, the vehicle state at the target position can be predicted and obtained. According to the current vehicle state and the target position, the vehicle state at the target position is predicted. In some embodiments, the prediction is achieved by a second vehicle state prediction model. The second vehicle state prediction model can process the current vehicle state and the target position to obtain the vehicle state at the target position. The third training sample used to train the second vehicle state prediction model includes the sample current vehicle state and the sample target position, and the label is used to characterize the vehicle state at the target position. In some embodiments, the second vehicle state prediction model may include but is not limited to a linear regression model, a neural network model, a decision tree, a support vector machine, and a naive Bayes.

[0086] In some embodiments, the processing device may also determine the driving condition at the target location based on the historical driving condition at the target location. For example, the historical driving condition at the target location, the current vehicle state, and the target location are input into the second vehicle state prediction model to determine the driving condition at the target location.

[0087] In some embodiments, the processing device can also calculate the preset time for the vehicle to travel to the target position based on the target position and current driving parameters, and then use the first vehicle state prediction model to obtain the vehicle state at the target position based on the preset time and the current state of the vehicle.

[0088] In some embodiments, the prediction of the user state at the target location can be achieved by a second user state prediction model. The second user state prediction model can process the current user state and the target location to obtain the user state at the target location. The second user state prediction model is similar to the second vehicle state prediction model and will not be described in detail here.

[0089] In some embodiments, similar to determining whether the first reminder message occurs based on the current vehicle state, the processing device may determine whether to send the first reminder message and its content based on the vehicle state after a preset time. For example, the processing device may determine whether a driving event occurs based on the vehicle state after a preset time, and may send the first reminder message if it occurs.

[0090] In some embodiments, similar to determining whether to send the first reminder information based on the current user status, the processing device may determine whether to send the first reminder information and its content based on the user status after a preset time.

[0091] In some embodiments, similar to determining whether to send the first reminder message based on the current user status and the current vehicle status, the processing device can determine whether to send the first reminder message and its content based on the vehicle status after a preset time and the user status after a preset time.

[0092] In some embodiments, similar to determining whether to send the first reminder message based on the current vehicle status, the processing device may determine whether to send the first reminder message and its content based on the vehicle status at the target location.

[0093] In some embodiments, similar to determining whether to send the first reminder information based on the current user status, the processing device may determine whether to send the first reminder information and its content based on the user status at the target location.

[0094] In some embodiments, similar to determining whether to send the first reminder information based on the current user status and the current vehicle status, the processing device can determine whether to send the first reminder information and its content based on the vehicle status at the target location and the user status at the target location.

[0095] The first reminder information determined based on the subsequent status information is an advance reminder information sent to the user, that is, to remind the user in advance to provide feedback on the subsequent driving conditions. At this time, the time for sending the first reminder information to the user terminal can be adjusted according to the actual situation. For example, it can be sent at a preset distance from the target location. For another example, it can be sent at a certain time before the preset time is reached. For another example, the sending time can be determined based on the user status, the user terminal status, the driving environment and / or the user-related information. For example, it is sent when the user status or the user terminal status is idle before the preset time or the target location is reached.

[0096] In some embodiments, the processing device may determine the manner of the first reminder information based on the subsequent user status, the subsequent user terminal status, the subsequent driving environment, and the user association information. Figure 4 And the related description will not be repeated here.

[0097] In some embodiments, the subsequent user terminal status may include the user terminal status after a preset time and the user terminal status at the target location. Figure 4 and its related description.

[0098] In some embodiments, the user terminal state after a preset time or the user terminal state at the target location can be predicted. For example, the chat content can be identified to predict whether the chat will end after a preset time. In some embodiments, the processing device can predict the subsequent user terminal state based on the current user terminal state, user association information and / or the current user state.

[0099] The methods of different embodiments can be combined. For example, in some embodiments, the subsequent user terminal state can be predicted based on the user emotion and the current user terminal state. For example, if the user emotion is "happy" and the current user terminal state is playing games, the predicted subsequent user terminal state is playing games. In some embodiments, the subsequent user terminal state can be predicted based on the user idleness, user emotion and the current user terminal state.

[0100] In some embodiments, the subsequent driving environment may include a driving environment after a preset time and a driving environment at a target location. In some embodiments, the driving environment after a preset time and the driving environment at a target location may be obtained from a service platform (e.g., a map service or a weather service, etc.).

[0101] The methods of different embodiments can be combined. For example, in some embodiments, the processing device can send the first reminder information to the user terminal in real time based on the state information; in some embodiments, the processing device can send the first reminder information to the user terminal in real time in the form of an image based on the current state information, and send the first reminder information to the user terminal in the form of a voice at a certain time before reaching the preset time or the target location based on the subsequent state information.

[0102] In some embodiments, the first reminder information determined based on the subsequent state information can be directly displayed on the navigation interface of the user terminal. In this way, the user can know the content and location of the feedback in advance by checking the navigation route.

[0103] like Fig. 6A -D, Figure 6B is the vehicle's moving interface, such as Figure 6BAs shown in the figure, the vehicle is traveling from the current position A to the position B1, then the processing device can display the estimated time from the current position to the next position on the interface of the user terminal to prompt the user to provide feedback when arriving at the next position. For example, the interface of the user terminal can display "traveling to B1, estimated time to provide feedback: 20 minutes".

[0104] In some embodiments, the processing device can display the road conditions at the driving position corresponding to the subsequent vehicle state (i.e., the vehicle state after a preset time and the vehicle state at the target position) on the current position feedback interface and the vehicle travel interface. Fig. 6A For example, the road section icon from position A to position B1 displayed on the user terminal interface indicates that the road section is a delayed driving section. For example, the road section requires longer driving time than usual due to muddy roads or construction; the road section icon from position B2 to position B3 indicates that the road section has a school; the road section icon from position B3 to position B4 indicates that the road section has many traffic lights.

[0105] In some embodiments, the processing device may also display the road conditions at the current location of the vehicle on the vehicle's travel interface. Figure 6B As shown, Figure 6B The floating window on the middle right can show the vehicle's current muddy / construction road conditions.

[0106] In some embodiments, the processing device may dynamically adjust the current position feedback interface and the vehicle's driving interface based on the user's preset instructions. The preset instructions may be instructions generated after the user provides feedback on the driving conditions at the current position. Fig. 6A For example, if the user clicks on the emoticon icon, that is, provides feedback on the driving conditions at the current location, the processing device can switch the current location feedback interface to the vehicle's driving interface.

[0107] In some embodiments, the user can zoom in and out on the current position feedback interface and the vehicle's travel interface on the user terminal interface. Fig. 6A or Figure 6B For example, the user can click the “-” icon and the “+” icon displayed on the current position feedback interface and the vehicle's travel interface to zoom in and out the interface.

[0108] exist Fig. 6AIn the user interface shown, the current vehicle position is position A, and the vehicle positions after the preset time are B1, B2, and B3, where B1, B2, and B3 are the vehicle positions after 10 minutes, 20 minutes, and 30 minutes, respectively. On the current user interface, the first reminder message for feedback on the driving conditions after 10 minutes is displayed, "Expected to arrive at position B1 in 10 minutes, please prepare to provide feedback on the driving conditions after 10 minutes." Users can click on the three emoticon icons on the interface to provide feedback, where the three emoticon icons represent very satisfied, generally satisfied, and dissatisfied from left to right. After 10 minutes, the user interface is updated to Figure 6B , the current vehicle position is updated to position B1, B2 and B3 are the vehicle positions after 10 minutes and 20 minutes respectively, and the first reminder message on the current interface is updated to "It is expected to arrive at position B2 in 10 minutes. Please prepare to provide feedback on the driving conditions after 10 minutes."

[0109] like Figure 6C In the user interface shown, the current vehicle position is position A, and the target positions include positions C1, C2, C3, C4, C5 and C6, where C1, C2, C3, C4, C5 and C6 are turning position, turning position, school section, turning position, traffic light position and turning position respectively. On the current user interface, the first reminder message of feedback on the driving condition of position C1 is displayed, "Please prepare to provide feedback on whether the turn at C1 is smooth." The user can click on the three emoticon icons on the interface to provide feedback. When the vehicle drove to C1, the user provided feedback on the driving condition of the position (specifically, whether the turn was smooth). And, the user interface enters the Fig.6D , the current vehicle position is updated to C1, the target position is updated to positions C2, C3, C4, C5 and C6, and the first reminder message on the current interface is updated to "Please prepare to provide feedback on whether the turn at C2 is smooth."

[0110] It can be seen from this that the vehicle status after the preset time and the vehicle status at the target position are both used to reflect the future driving conditions of the vehicle. By sending the first reminder information based on the future driving conditions, the user can be reminded to provide feedback in advance before reaching the preset time or the target position, thereby avoiding the situation where the user does not send feedback information when the vehicle has passed the position at the preset time or the target position due to the delay in sending the first reminder information.

[0111] Step 230, obtaining feedback information on the driving condition of the vehicle input by the user in the user terminal in response to the first reminder information. In some embodiments, step 230 may be performed by a second obtaining module.

[0112] A user refers to a passenger riding in a vehicle. A vehicle can be a vehicle used in smart travel, that is, a vehicle ridden by a user. For example, a vehicle can be a shared vehicle in an online car-hailing service. For another example, a vehicle can be an autonomous vehicle, etc. For details about autonomous vehicles, please refer to the above description, which will not be repeated here.

[0113] A user terminal is a terminal for users to operate or receive information. In some embodiments, a user terminal may be a terminal that provides online services to users. For example, a user terminal may be a terminal that provides online car-hailing services to users. For another example, a user terminal may be a terminal that provides autonomous driving services to users. User terminals may include, but are not limited to, one or more combinations of notebooks, tablet computers, mobile phones, personal digital assistants (PDAs), in-vehicle camera devices, dedicated touch devices, and wearable devices. In some embodiments, a user terminal may also be a device terminal inside a vehicle for collecting or providing information, such as a vehicle-mounted terminal, an electronic device, or a remote control device.

[0114] The driving condition of the vehicle may include information related to the driving of the vehicle. The driving condition may include: a driving route, a driving position, a driving section, a driving environment, a driving state, a driving parameter, and a combination of one or more of the above.

[0115] The driving route may be a route that a vehicle travels from a departure point to a destination. In some embodiments, the driving route may be at least one navigation route automatically generated by a processing device based on a departure point and a destination. The departure point and the destination may be obtained in a variety of ways. For example, by a user inputting the information into a user terminal.

[0116] The driving position may be a position that the vehicle has passed. The position may include a point, an area, or a road section. In some embodiments, the driving position may be a position that the vehicle has passed, a current position, or a position that the vehicle is about to pass. The driving position may be a track point in the driving route. For example, the driving position may include a departure point, a destination, and a current position, etc.

[0117] A road segment can be a route consisting of at least two position points on a road network. The at least two position points can be adjacent or non-adjacent. For example, a road segment can be a section or the entirety of a road.

[0118] In some embodiments, the driving section may be a section that the vehicle has traveled through. In some embodiments, the driving section may be a section that the vehicle has traveled through, a section where the vehicle is currently located, or a section that the vehicle is about to travel through. In some embodiments, the driving section may also be a preset section included in the driving route. The preset section may be specifically set according to actual needs, for example, the preset section may include at least one of a turning section, a roundabout section, a no-parking section, a one-way driving section, and a section with many accidents.

[0119] The driving environment may be environmental information related to the vehicle or the driving of the vehicle. The driving environment may include the in-vehicle environment when the vehicle is driving and the external environment when the vehicle is driving. In some embodiments, the driving environment may include the in-vehicle air quality, the in-vehicle humidity, the in-vehicle temperature, the number of passengers in the vehicle, whether the vehicle is clean, the in-vehicle light, and the appearance of the passengers in the vehicle. In some embodiments, the driving environment may include at least one of the weather environment, the road condition environment, and the time environment.

[0120] In some embodiments, the weather environment may reflect temperature, air pressure, humidity, visibility, etc. For example, the weather environment may include rain, snow, clear weather, etc.

[0121] In some embodiments, the road condition environment may reflect the smoothness of vehicle driving. For example, the road condition environment may reflect the steepness of the road, the number of lanes, the degree of road congestion, road closures, road construction conditions, and traffic control conditions. For example, the road condition environment may include traffic volume, the number of traffic lights, construction ahead, etc.

[0122] In some embodiments, the time environment may reflect the time information of the vehicle's travel, for example, whether it is a holiday, whether it is a working day, whether it is a rush hour, etc.

[0123] The driving state may be the state information of the vehicle during the driving process. In some embodiments, the driving state may include: a normal driving state or an abnormal driving state, etc. An abnormal driving state may be driving with a driving event, and a normal driving state may be driving without a driving event. A driving event may be an event related to the vehicle operation and an event that affects the user's riding experience during the driving process. In some embodiments, the driving event may include events such as starting, stopping, accelerating, decelerating, sudden acceleration, speeding, sudden braking, turning, and bumping of the vehicle.

[0124] Driving parameters refer to parameters that reflect the driving conditions of the vehicle. In some embodiments, the driving parameters include, but are not limited to, speed parameters and acceleration parameters.

[0125] The feedback information may reflect the user's attitude towards the vehicle's driving condition. In some embodiments, the feedback information may include whether the user is satisfied with the vehicle's driving condition and / or the degree of satisfaction (e.g., very satisfied, generally satisfied, dissatisfied, etc.). In some embodiments, the feedback information may also reflect other information, such as the reasons for satisfaction or dissatisfaction, suggestions for the vehicle's driving condition, etc.

[0126] In some embodiments, the user can input feedback information in the user terminal according to his / her own wishes or needs. For example, the user inputs feedback information in the user terminal according to the driving condition of the vehicle (for example, the vehicle is driving too fast or the vehicle brakes suddenly). For another example, the user inputs feedback information in the user terminal at any time.

[0127] In some embodiments, the user may input feedback information based on the first reminder information sent by the processing device.

[0128] The methods of different embodiments can be combined. For example, in some embodiments, the user can input feedback information on the driving parameters of the vehicle in the mobile terminal; in some embodiments, the processing device can determine whether to send the first reminder information based on the vehicle status after a preset time, and the user can input feedback information on the driving route of the vehicle in the vehicle terminal at an appropriate time after receiving the first reminder information.

[0129] In some embodiments, the user can input feedback information in a variety of ways. For example, the user can send feedback information by clicking an icon or text button, sending text or voice, or making gestures or facial movements. For example, the "like" and "dislike" icons are displayed on the user terminal interface, and the user sends feedback information representing dissatisfaction by clicking the "dislike" icon. The user makes a preset gesture corresponding to "satisfied" facing the camera of the terminal as the feedback information sent.

[0130] The methods of different embodiments can be combined. For example, in some embodiments, the user can click an icon on the mobile phone interface to input feedback information on the driving event; in some embodiments, the processing device can determine whether to send the first reminder information based on the vehicle status at the target location. If sent, the processing device sends the first reminder information to the user's wearable device when the vehicle is at a preset distance from the target location. After receiving the first reminder information, the user can send voice as feedback information on the driving route through the wearable device.

[0131] After the user inputs the feedback information, the feedback information can be stored in the storage device or directly sent to the processing device. The processing device (eg, the second acquisition module) can directly acquire the feedback information from the user terminal or read the feedback information from the storage device.

[0132] In some embodiments, after the processing device obtains the feedback information, it can process the feedback information to determine the attitude reflected by the feedback information. For example, the processing device can determine the attitude represented by the icon clicked by the user on the user terminal through the preset relationship between the icon and the attitude. For another example, the processing device can identify (for example, through a model or algorithm, etc.) the text, picture, video or voice sent by the user terminal to determine the attitude reflected.

[0133] The methods of different embodiments can be combined. For example, in some embodiments, the processing device can determine whether to send the first reminder information based on the current user state (for example, the user's current mood), and the user can input feedback information in the mobile phone based on the first reminder information received by the vehicle-mounted device; in some embodiments, the processing device can determine whether to send the first reminder information based on the vehicle state after a preset time, and if sent, the first reminder information is sent to the mobile phone at a certain time before the preset time, and the user can input feedback information in the mobile phone based on the first reminder information received by the mobile phone. When the feedback information reflects that the user is dissatisfied with the navigation route, the navigation route can be updated.

[0134] Some embodiments of the present specification obtain status information and promptly send a first reminder message to a user terminal, so that the user responds to the first reminder message and inputs feedback information on the vehicle's condition in the user terminal, which facilitates the subsequent operation of the vehicle's driving system by the processing device, improves the driving conditions corresponding to the user's dissatisfied feedback, and enhances the user's riding experience; at the same time, through the user's timely feedback during the journey, it can avoid the trip control automatically set by the autonomous driving vehicle failing to meet the user's actual needs, thereby further improving the user's riding experience.

[0135] Figure 3 FIG. 1 is a flowchart of determining status information according to some embodiments of this specification. Figure 3 As shown, the process 300 may include the following steps 310 to 340. In some embodiments, the process 300 may be performed by a processing device (eg, the processing device 112). In some embodiments, the process 300 may be performed by a first acquisition module.

[0136] Step 310: Acquire user's image information from the first terminal.

[0137] The first terminal may be a terminal device for collecting images, for example, a camera device, a monitoring device, a user terminal with a camera, or a vehicle-mounted device with a camera. The image information of the user may be an image of the user taken by the first terminal or an image frame included in a video of the user taken by the first terminal.

[0138] In some embodiments, the first terminal can obtain the user's image information in real time and upload it to the database. The processing device can obtain the user's image information in real time from the first terminal. The processing device can also obtain the user's image information from the database.

[0139] Step 320: Acquire vehicle-related audio information from the second terminal.

[0140] In some embodiments, the second terminal may be a terminal device for acquiring audio information, such as a recording device, a camera device, a user terminal with a recording function, or a vehicle-mounted device with a recording function. The second terminal and the first terminal may be the same or different.

[0141] Vehicle-related audio information refers to audio data emitted by the vehicle or emitted in the vehicle's internal environment. In some embodiments, vehicle-related audio information may include: audio emitted by the vehicle (e.g., engine sound, brake sound, etc.), audio emitted or received by the user using the user terminal, or audio of communication between the user and the driver of the vehicle, etc.

[0142] In some embodiments, the second terminal can obtain the audio information related to the vehicle in real time and upload it to the database. The processing device can obtain the audio information related to the vehicle in real time from the second terminal. The processing device can also obtain the audio information related to the vehicle from the database.

[0143] Step 330: extracting image features of the image information and audio features of the audio information.

[0144] In some embodiments, the image features may include color features, texture features, shape features, and spatial features. In some embodiments, the processing device may extract the image features through an image feature extraction algorithm or an image feature extraction model. For example, the image feature extraction model may include a convolutional neural network model.

[0145] In some embodiments, the audio features may include sampling frequency, bit rate, number of channels, frame rate, zero crossing rate, short-time energy, and short-time autocorrelation coefficient. In some embodiments, the processing device may extract the audio features through an audio feature extraction algorithm. The audio feature extraction algorithm may include, but is not limited to, linear prediction analysis (Linear Prediction Coefficients, LPC), perceptual linear prediction coefficients (Perceptual Linear Predictive, PLP), Linear Predictive Cepstral Coefficients (Linear Predictive Cepstral Coefficient, LPCC) and Mel-Frequency Cepstrum Coefficients (Mel-Frequency Cepstrum Coefficient, MFCC), etc.

[0146] In some embodiments, the processing device pre-processes the image information or the audio information before extracting the audio features or the image features. The pre-processing of the image information includes: smoothing, noise elimination, edge enhancement, edge feature extraction, etc. The pre-processing of the audio information includes: pre-emphasis, framing, windowing, etc.

[0147] Step 340, determining state information based on processing the image features and the audio features.

[0148] In some embodiments, the processing can be performed by a pre-trained feature processing model. The pre-trained feature processing model can determine the state information based on the image features and the audio features. In some embodiments, the fourth training sample of the feature processing model includes sample image features and sample audio features of the sample vehicle, and the label is used to characterize the state information of the sample vehicle. When the determined state information is the vehicle state, the label represents the vehicle state, for example, the label represents the vehicle's driving events: starting, stopping, accelerating or turning, etc. When the determined state information is the user state, the label represents the user state, for example, the label represents the user's emotions: angry, uncomfortable, panic or happy, etc. The processing device can iteratively update the parameters of the initial feature processing model based on multiple training samples so that the loss function of the model meets the preset conditions, for example, the loss function converges, or the loss function value is less than the preset value. When the loss function meets the preset conditions, the model training is completed, and a trained feature processing model is obtained.

[0149] In some embodiments, the image feature extraction model and the feature processing model can be trained in an end-to-end training manner. Specifically, the sample image information of the sample user is input into the initial image feature extraction model, the sample audio features are input into the initial feature processing model, a loss function is constructed based on the label representing the state information of the sample vehicle and the result output by the initial feature processing model, and the parameters of the initial feature processing model are iteratively updated based on the loss function until the preset conditions are met.

[0150] In some embodiments of the present specification, the current user status and the current vehicle status can be determined in real time by acquiring user image information and vehicle audio information in real time, so as to facilitate the subsequent real-time acquisition of current status information and the sending of the first reminder information.

[0151] Figure 4 FIG. 1 is a flowchart of sending a first reminder message according to some embodiments of this specification. Figure 4 As shown, the process 400 may include the following steps 410 and 420. In some embodiments, the process 400 may be executed by a processing device (eg, the processing device 112).

[0152] Step 410, determining a feedback reminder method. In some embodiments, step 610 may be performed by a sending module.

[0153] In some embodiments, the feedback reminder method can be used to determine the representation method of the first reminder information. For example, voice, text, image, video, vibration, etc. The feedback reminder method can be related to at least one of the user status, the user terminal status, the driving environment, and the user-related information. For specific details about the user status, please refer to the above step 210 and its related description, which will not be repeated here. For specific details about the driving environment, please refer to the above step 230 and its related description, which will not be repeated here.

[0154] The user terminal status can reflect the terminal battery level, terminal standby time, terminal usage time, whether the terminal is being used, and the current usage method of the terminal (watching videos, listening to music, etc.).

[0155] In some embodiments, the feedback reminder method can be determined based on the user status.

[0156] The methods of different embodiments can be combined. For example, in some embodiments, when the user's idle state reflects that the user is reading a book, the feedback reminder method can be voice; in some embodiments, when the user's idle state reflects that the user is listening to music, the feedback reminder method can be text display or image; in some embodiments, when the user's emotion is anger, the feedback reminder method can be voice.

[0157] In some embodiments, the feedback reminder method can be determined based on the user terminal status.

[0158] The methods of different embodiments can be combined. For example, in some embodiments, when the user terminal status reflects that the user is using a speaker to make a call or play music, the feedback reminder method can be an image display or text; in some embodiments, when the user terminal status reflects that the user's mobile phone is in low power or standby mode, the feedback reminder method can be performed through a vehicle-mounted device or a wearable device, for example, through a vehicle-mounted device or a smart watch voice display, image display, or text display.

[0159] In some embodiments, the feedback reminder method can be determined based on the driving environment.

[0160] The methods of different embodiments can be combined. For example, in some embodiments, when the in-car environment information reflects that the environment is noisy and it is inconvenient to listen to the audio, the feedback reminder method can be through image display or text, and when the in-car environment information reflects that the light in the car is dim or too bright, the feedback reminder method can be voice; in some embodiments, when the road condition environment is a congested section, the vehicle brakes frequently, and looking at the mobile phone screen for a long time may cause discomfort to the user, and the feedback reminder method can be voice or vibration.

[0161] In some embodiments, the feedback reminder method can also be determined based on user-related information. For example, if the user-related information reflects that the user's age exceeds the preset age, the text and sound in the feedback reminder method can be amplified. For another example, if the user information reflects that the user's commonly used language is English, the text and sound in the feedback reminder method can be switched to English. For another example, if the user information reflects that the user is Chinese, the text and sound in the feedback reminder method can be switched to Chinese.

[0162] In some embodiments, the feedback reminder method can also be determined based on multiple information of the user status, user terminal status, driving environment and user-related information. The processing device can determine the feedback reminder method based on preset rules or a trained feedback reminder method determination model, process multiple information of the user status, user terminal status, driving environment and user-related information, and determine the feedback reminder method.

[0163] For example, as mentioned above, each of the user status, user terminal status, driving environment and user-related information may include multiple types, and each type may have a preset corresponding feedback reminder method. The preset rule may be to score the multiple information in the user status, user terminal status, driving environment and user-related information separately, and to fuse the scores of the same type of feedback reminder methods (e.g., averaging, weighted summing, weighted averaging, etc.), and determine the final feedback reminder method based on the fused scores of the different types of feedback reminder methods (e.g., selecting the feedback reminder method with the largest fused score).

[0164] For another example, the processing device can simultaneously process multiple information including user status, user terminal status, driving environment and user-related information based on preset rules or trained feedback reminder mode determination models to determine multiple initial feedback reminder modes; based on preset screening rules, determine the final feedback reminder mode from multiple initial feedback reminder modes. The preset screening rules can be random screening or set according to needs.

[0165] In some embodiments, the feedback reminder mode can be changed in real time based on the above actual situation. For example, when the user is answering a call, the feedback reminder mode can be changed from the original voice display to text display or image display.

[0166] Step 420: Generate first reminder information based on the feedback reminder mode, and send the first reminder information to the user terminal. In some embodiments, step 420 may be performed by a sending module.

[0167] For details on generating the first reminder information and sending it to the user terminal, please refer to the above step 220 and its related description, which will not be repeated here. In some embodiments, the processing device can send the first reminder information to the user terminal via a transmission method such as SMS, network or Bluetooth.

[0168] In some embodiments of the present specification, the corresponding feedback reminder method is intelligently and dynamically determined based on various information (for example, user status, user terminal status, driving environment, and user-related information, etc.), thereby avoiding fixed feedback reminder methods from disturbing the user and improving the user's riding experience.

[0169] Figure 5 is a flowchart of predicting a target position according to some embodiments of this specification. Figure 5 As shown, the process 500 may include the following steps 510 and 520. In some embodiments, the process 500 may be performed by a processing device (eg, the processing device 112). In some embodiments, the process 500 may be performed by a first acquisition module.

[0170] Step 510, obtaining historical feedback information related to at least one of the track point on the first navigation route and the user, and historical driving conditions corresponding to the historical feedback information.

[0171] In some embodiments, the first navigation route may be a navigation route currently being used by the vehicle. As described in step 230 above, the driving route may include at least one navigation route. In some embodiments, the first navigation route may be a navigation route selected by a user from the driving routes, or may be an optimal navigation route automatically selected by a processing device from the driving routes (e.g., a navigation route with the shortest journey, a navigation route with the shortest time spent, or a navigation route with the least amount of consumption, etc.).

[0172] In some embodiments, historical feedback information of historical users on historical driving conditions of historical vehicles can be stored in a storage device (e.g., database 140) or a platform. It is understandable that there is at least one corresponding historical feedback information for the historical driving conditions of historical vehicles. For example, the correspondence between historical users, historical vehicles, historical driving conditions, and historical feedback information can be stored in a storage device.

[0173] The track point on the first navigation route may be a location point included on the first navigation route. In some embodiments, the acquisition module may acquire historical feedback information related to the track point on the first navigation route. For example, the acquisition module may acquire historical feedback information corresponding to the track point on the first navigation route and historical driving conditions corresponding to the historical feedback information from a storage device or platform.

[0174] In some embodiments, the acquisition module may acquire historical feedback information related to the user. For example, the acquisition module may acquire historical feedback information of the user from a storage device or platform, and historical driving conditions corresponding to the historical feedback information.

[0175] In some embodiments, the acquisition module may acquire historical feedback information related to the user and the track point on the first navigation route. For example, the acquisition module acquires historical feedback information of the historical user for the track point on the first navigation route and the historical driving conditions corresponding to the historical feedback information from a storage device or platform, wherein the historical user is the user.

[0176] Step 520: predicting a target location on the first navigation route based on the historical feedback information and the historical driving conditions.

[0177] In some embodiments, the processing device may predict the target location on the first navigation route based on historical feedback information (which may be referred to as "first historical feedback information") related to the track points and / or users on the first navigation route. For example, the processing device may determine the track point corresponding to the negative feedback (e.g., unsatisfactory feedback) in the first historical feedback information as the target location. For example, if the user or other users have provided unsatisfactory feedback information on the historical driving conditions of track points 1 and track points 2 included in the first navigation route in the past, the processing device may determine track points 1 and 2 as the target locations. For another example, the processing device may determine the track points in the first historical feedback information that include a number of negative feedback greater than a threshold as the target location. The preset number of times can be set according to actual needs, for example, 3 times or 5 times.

[0178] In some embodiments, the processing device may predict the target location on the first navigation route based on the historical driving condition corresponding to the first historical feedback information (which may be referred to as the "first historical driving condition"). For example, the processing device may determine the track point reflecting the driving event in the historical driving condition as the target location. For example, if the driving event "turn" is reflected at track points 2 and 3, track points 2 and 3 may be determined as the target location.

[0179] In some embodiments, the processing device may predict the target location on the first navigation route based on the first historical feedback information and the first historical driving condition. In some embodiments, the processing device may determine as the target location a track point that reflects a driving event in the first historical driving condition and whose corresponding first historical feedback information is negative feedback. Still taking the above example, track point 2 may be determined as the target location.

[0180] In some embodiments, the processing device may predict the target position on the first navigation route based on the trained target position prediction model. Specifically, the target position prediction model may process the first historical feedback information and the first historical driving condition and output the target position. The target position prediction model may include but is not limited to a graph neural network model, a linear regression model, a neural network model, a decision tree, a support vector machine, and a naive Bayes.

[0181] When the target location prediction model is a graph neural network model, the nodes of the graph neural network model are trajectory points on the first navigation route, and the characteristics of the nodes may include at least: the first historical feedback information, the first historical driving conditions, and the attributes of the trajectory points. The attributes of the trajectory points may reflect the type of trajectory points, for example, turning sections, congested sections, near schools, etc. The edges of the graph neural network model are the relationships between trajectory points, and the characteristics of the edges may include distance relationships, whether they appear on the same navigation route and / or whether they are on the same section, etc. The characteristics of the edges and the characteristics of the nodes are input into the graph neural network model, and the node output of the graph neural network model indicates whether the corresponding trajectory point is the target location.

[0182] As mentioned above, the first historical feedback information can be divided into multiple categories: the user's past feedback information on the track points on the first navigation route, other users' past feedback information on the track points on the first navigation route, and the user's past feedback information on other locations (i.e., locations other than the track points on the first navigation route). In some embodiments, when determining the target location based on historical feedback information, weights can be set for different historical feedback information. The weights can be set according to the situation. For example, the user's past feedback information on the track points has the highest weight, so that the determined target location can be more in line with the user's needs.

[0183] In some embodiments of the present specification, the target position is determined by historical feedback information and historical driving conditions. A trajectory point on the navigation route currently being used by the vehicle that has been historically fed back or historically determined to have a driving event can be determined as the target position, and then user feedback information on the target position is collected to facilitate subsequent optimization of vehicle control to enhance the user's riding experience.

[0184] An embodiment of the present specification also provides a user feedback device, which includes at least one processor and at least one memory; the at least one memory is used to store computer instructions; the at least one processor is used to execute at least part of the computer instructions to implement operations corresponding to the user feedback method described in any of the preceding items.

[0185] The embodiments of the present specification also provide a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed by a processor, operations corresponding to the method for user feedback as described in any of the preceding items are implemented.

[0186] The basic concepts have been described above. Obviously, for those skilled in the art, the above detailed disclosure is only for example and does not constitute a limitation of this specification. Although not explicitly stated here, those skilled in the art may make various modifications, improvements and corrections to this specification. Such modifications, improvements and corrections are suggested in this specification, so such modifications, improvements and corrections still belong to the spirit and scope of the exemplary embodiments of this specification.

[0187] At the same time, this specification uses specific words to describe the embodiments of this specification. For example, "one embodiment", "an embodiment", and / or "some embodiments" refer to a certain feature, structure or characteristic related to at least one embodiment of this specification. Therefore, it should be emphasized and noted that "one embodiment" or "an embodiment" or "an alternative embodiment" mentioned twice or more in different positions in this specification does not necessarily refer to the same embodiment. In addition, certain features, structures or characteristics in one or more embodiments of this specification can be appropriately combined.

[0188] In addition, it will be understood by those skilled in the art that various aspects of this specification may be illustrated and described by a number of patentable categories or situations, including any new and useful process, machine, product or combination of substances, or any new and useful improvements thereto. Accordingly, various aspects of this specification may be performed entirely by hardware, entirely by software (including firmware, resident software, microcode, etc.), or by a combination of hardware and software. The above hardware or software may be referred to as "data blocks", "modules", "engines", "units", "components" or "systems". In addition, various aspects of this specification may be represented as a computer product located in one or more computer-readable media, which includes computer-readable program code.

[0189] A computer storage medium may include a propagated data signal containing computer program code, for example, in baseband or as part of a carrier wave. The propagated signal may be in a variety of forms, including electromagnetic, optical, etc., or a suitable combination. A computer storage medium may be any computer-readable medium other than a computer-readable storage medium, which can be connected to an instruction execution system, device or apparatus to communicate, propagate or transmit the program for use. The program code on the computer storage medium may be transmitted via any suitable medium, including radio, cable, fiber optic cable, RF, or similar media, or any combination of the above media.

[0190] The computer program code required for the operation of each part of this specification can be written in any one or more programming languages, including object-oriented programming languages ​​such as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C++, C#, VB.NET, Python, etc., conventional procedural programming languages ​​such as C language, Visual Basic, Fortran2003, Perl, COBOL2002, PHP, ABAP, dynamic programming languages ​​such as Python, Ruby and Groovy, or other programming languages, etc. The program code can be run entirely on the user's computer, or run on the user's computer as an independent software package, or run partially on the user's computer and partially on a remote computer, or run entirely on a remote computer or processing device. In the latter case, the remote computer can be connected to the user's computer through any network form, such as a local area network (LAN) or a wide area network (WAN), or connected to an external computer (e.g., via the Internet), or in a cloud computing environment, or used as a service such as software as a service (SaaS).

[0191] In addition, unless explicitly stated in the claims, the order of the processing elements and sequences described in this specification, the use of alphanumeric characters, or the use of other names are not intended to limit the order of the processes and methods of this specification. Although the above disclosure discusses some invention embodiments that are currently considered useful through various examples, it should be understood that such details are only for illustrative purposes, and the attached claims are not limited to the disclosed embodiments. On the contrary, the claims are intended to cover all modifications and equivalent combinations that are consistent with the essence and scope of the embodiments of this specification. For example, although the system components described above can be implemented by hardware devices, they can also be implemented only by software solutions, such as installing the described system on an existing processing device or mobile device.

[0192] Similarly, it should be noted that in order to simplify the description disclosed in this specification and thus help understand one or more embodiments of the invention, in the above description of the embodiments of this specification, multiple features are sometimes combined into one embodiment, figure or description thereof. However, this disclosure method does not mean that the features required by the subject matter of this specification are more than the features mentioned in the claims. In fact, the features of the embodiments are less than all the features of the single embodiment disclosed above.

[0193] In some embodiments, numbers describing the number of components and attributes are used. It should be understood that such numbers used in the description of the embodiments are modified by the modifiers "about", "approximately" or "substantially" in some examples. Unless otherwise specified, "about", "approximately" or "substantially" indicate that the numbers are allowed to vary by ±20%. Accordingly, in some embodiments, the numerical parameters used in the specification and claims are approximate values, which may change according to the required features of individual embodiments. In some embodiments, the numerical parameters should take into account the specified significant digits and adopt the general method of retaining digits. Although the numerical domains and parameters used to confirm the breadth of their range in some embodiments of this specification are approximate values, in specific embodiments, the setting of such numerical values ​​is as accurate as possible within the feasible range.

[0194] Each patent, patent application, patent application publication, and other materials, such as articles, books, specifications, publications, documents, etc., cited in this specification are hereby incorporated by reference in their entirety. Except for application history documents that are inconsistent with or conflicting with the contents of this specification, documents that limit the broadest scope of the claims of this specification (currently or later attached to this specification) are also excluded. It should be noted that if the descriptions, definitions, and / or use of terms in the materials attached to this specification are inconsistent or conflicting with the contents described in this specification, the descriptions, definitions, and / or use of terms in this specification shall prevail.

[0195] Finally, it should be understood that the embodiments described in this specification are only used to illustrate the principles of the embodiments of this specification. Other variations may also fall within the scope of this specification. Therefore, as an example and not a limitation, alternative configurations of the embodiments of this specification may be considered consistent with the teachings of this specification. Accordingly, the embodiments of this specification are not limited to the embodiments explicitly introduced and described in this specification.

Claims

1. A method for user feedback, comprising: Acquiring status information, wherein the status information includes vehicle status and user status; Based on the state information, sending first reminder information to the user terminal includes: Determine a feedback reminder mode, the feedback reminder mode is related to at least one of the user state, the user terminal state and the driving environment; wherein the user state includes the user's idle state, the user's vehicle usage and the user's mood, and the user's idle state includes whether the user is reading a book and whether the user is listening to music; the user terminal state includes the terminal power, the terminal standby time, the terminal usage time, whether the terminal is being used and the current terminal usage mode; the driving environment includes the in-vehicle environment and the road condition environment, and the in-vehicle environment includes the light inside the vehicle; and Generate the first reminder information based on the feedback reminder mode, and send the first reminder information to the user terminal; Acquire feedback information on the driving condition of a vehicle input by the user at the user terminal in response to the first reminder information, the vehicle being the vehicle in which the user rides.

2. According to the method of claim 1, the vehicle state includes at least one of a current vehicle state, a vehicle state after a preset time, and a vehicle state at a target position; and the vehicle state is related to the driving condition of the vehicle.

3. The method according to claim 2, wherein the driving condition comprises: At least one of a driving route, a driving position, a driving section, the driving environment, a driving state and a driving parameter.

4. The method according to claim 2, wherein the target position is obtained through a prediction process, wherein the prediction process comprises: Acquire historical feedback information related to the track point on the first navigation route and at least one of the users, and historical driving conditions corresponding to the historical feedback information; as well as Based on the historical feedback information and the historical driving conditions, the target position on the first navigation route is predicted by a trained target position prediction model, wherein the target position prediction model is a graph neural network model, wherein: The first navigation route is a navigation route currently being used by the vehicle; The nodes of the graph neural network model are trajectory points on the first navigation route, and the characteristics of the nodes include: first historical feedback information, first historical driving conditions and attributes of the trajectory points; the attributes of the trajectory points reflect the type of the trajectory points; the edges of the graph neural network model are the relationships between the trajectory points, and the characteristics of the edges include distance relationships, whether they appear on the same navigation route and / or whether they are on the same road section; the characteristics of the edges and the characteristics of the nodes are input into the graph neural network model, and the node output of the graph neural network model determines whether the corresponding trajectory point is the target location.

5. The method according to claim 1, wherein the vehicle is an autonomous driving vehicle.

6. A user feedback system, comprising: A first acquisition module, used to acquire status information, wherein the status information includes vehicle status and user status; A sending module, used for sending first reminder information to a user terminal based on the state information, comprising: Determine a feedback reminder mode, the feedback reminder mode is related to at least one of the user state, the user terminal state and the driving environment; wherein the user state includes the user's idle state, the user's vehicle usage and the user's mood, and the user's idle state includes whether the user is reading a book and whether the user is listening to music; the user terminal state includes the terminal power, the terminal standby time, the terminal usage time, whether the terminal is being used and the current terminal usage mode; the driving environment includes the in-vehicle environment and the road condition environment, and the in-vehicle environment includes the light inside the vehicle; and Generate the first reminder information based on the feedback reminder mode, and send the first reminder information to the user terminal; The second acquisition module is used to acquire feedback information on the driving condition of the vehicle input by the user in the user terminal in response to the first reminder information.

7. A user feedback device, the device comprising at least one processor and at least one memory; The at least one memory is used to store computer instructions; The at least one processor is configured to execute at least part of the computer instructions to implement the method according to any one of claims 1 to 5.

8. A computer-readable storage medium storing computer instructions, which implement the method according to any one of claims 1 to 5 when the computer instructions are executed by a processor.

Images