Data processing method and apparatus, and device

Abstract

Some embodiments of the present application provide a data processing method, apparatus, and device, including: acquiring first data and first historical driving characteristics, the first data including first driving data of a vehicle for a first time period, the first historical driving characteristics being determined based on the historical driving data of the vehicle; determining that the vehicle is abnormal for the first time period based on the first data and the first historical driving characteristics; and annotating a driving record video of the vehicle based on the first time period, which improves efficiency of searching video clips.

Description

[Technical field]

[0001] This application relates to the field of intelligent driving and intelligent vehicle technology, and in particular to data processing methods and apparatus, and devices. [Background technology]

[0002] Currently, driving recorders are installed in many vehicles, and are capable of taking and storing video of the vehicle during the driving process.

[0003] The video captured by the driving recorder and the video capture time of the video may be stored in a preset storage space. In special cases, such as when a vehicle accident occurs or when a user remembers a driving scenario, the user needs to search for the corresponding video clip in the preset storage space. In the conventional technology, the user usually estimates an approximate time and searches for the corresponding video clip in the preset storage space based on the estimated time. However, searching for the video clip by this method is not efficient. Summary of the Invention

[0004] Certain embodiments of the present application provide a data processing method and apparatus, and a device, which improves the efficiency of searching for video clips.

[0005] According to a first aspect, one embodiment of the present application provides a data processing method, the method including: acquiring first data and first historical driving characteristics, determining that a vehicle is not normal for a first time period based on the first data and the first historical driving characteristics, and annotating a driving record video of the vehicle based on the first time period, where the first data includes first driving data of the vehicle for the first time period, and the first historical driving characteristics are determined based on the historical driving data of the vehicle.

[0006] In the above process, the first data can reflect the driving situation of the vehicle in the first time period, and the first historical driving characteristics can reflect the driving habits of the driver of the vehicle. Because the driving habits of the user are usually fixed, it is possible to accurately determine whether the vehicle is abnormal in the first time period based on the first driving data and the first historical driving characteristics. When determining that the vehicle is abnormal in the first time period, the driving record video captured by the image capture device may be annotated to record that the vehicle is abnormal in the first time period. In this way, the driving record video can be accurately annotated. The user can accurately and quickly search for video clips based on the annotation result of the driving record video, thereby improving the searching efficiency of the video clips.

[0007] In one possible implementation, the first operating data includes at least one of status data of a component in the vehicle, operation data of a component in the vehicle, or sensor data collected by a component in the vehicle.

[0008] In the above process, the first driving data includes one or more of component status data, operation data, or sensor data, which are generated during the operation process of the vehicle and can accurately reflect the driving state of the vehicle. Therefore, based on the first driving data, it is possible to accurately determine whether the vehicle is abnormal in a first time period.

[0009] In one possible implementation, the first historical driving characteristic includes a plurality of historical data curves corresponding to a plurality of types of historical driving data, the historical data curves being used to indicate a distribution rule of the historical driving data, and the historical driving data includes at least one of historical state data of the components in the vehicle, historical operation data of the components in the vehicle, or historical sensor data collected by the components in the vehicle.

[0010] In the above process, the historical driving data includes one or more of historical state data of components, historical operation data, or historical sensor data. The data is generated during the historical operation process of the vehicle and can accurately reflect the historical driving state of the vehicle. The historical driving state of the vehicle reflects the driving habits of a user. Therefore, based on the first historical driving characteristic, it can accurately determine whether the vehicle is abnormal in a first time period.

[0011] In one possible implementation, the first historical driving characteristics may be obtained by processing the historical driving data using a first model to obtain the first historical driving characteristics, the first model being obtained by learning a plurality of groups of first samples, each of the plurality of groups of first samples including sample driving data and sample historical driving characteristics.

[0012] In the above process, a plurality of groups of the first samples are learned, so that the first model has the function of acquiring the driving characteristics of the driving data. Therefore, the first model can be used to process the historical sample data to accurately acquire the first historical driving characteristics.

[0013] In one possible implementation, it may be determined that the vehicle is not normal during the first time period based on the first data and the first historical driving characteristics by determining a similarity between the first data and the first historical driving characteristics, and determining that the vehicle is not normal during the first time period based on the similarity.

[0014] In the above process, the similarity between the first data and the first historical driving characteristics reflects the similarity between the vehicle's operating state and the user's driving habits in the first time period. Because the user's driving habits are usually fixed, it is possible to accurately determine whether the vehicle is abnormal in the first time period based on the similarity.

[0015] In one possible implementation, the similarity between the first data and the first historical driving characteristics may be determined by processing the first data and the first historical driving characteristics using a second model to obtain the similarity, the second model being obtained by learning a plurality of groups of second samples, each of the plurality of groups of second samples including sample data, sample driving characteristics, and sample similarities, and the sample data including sample driving data.

[0016] In the above process, a plurality of groups of the second samples are learned, so that the second model has the function of determining the similarity between the data and the driving characteristics. Therefore, by using the second model to process the first data and the first historical driving characteristics, it is possible to accurately determine the similarity between the first data and the first historical driving characteristics.

[0017] In one possible implementation, the step of processing the first data and the first historical driving characteristics by using a second model to obtain the similarity includes the steps of determining a first data curve corresponding to the first data by using the second model, and comparing the historical data curve and the first data curve for the first historical driving characteristics by using the second model to obtain the similarity.

[0018] In the above process, the first data curve can accurately reflect the first data, so that the similarity can be accurately obtained by comparing the history data curve with the first data curve.

[0019] In one possible implementation, the first data further includes scenario data, and the scenario data includes at least one of the following information: time information, location information, road condition information, or weather information.

[0020] In the above process, the user's driving habits in multiple different scenarios (for example, under different weather conditions or different road conditions) may be different. Therefore, when the first data further includes scenario data, the scenario in which the user is driving in the first time period may be determined based on the scenario data. In this way, it is possible to more accurately determine whether the vehicle is abnormal in the first time period based on the first data.

[0021] In one possible implementation, the first historical driving characteristic may be obtained by determining the first historical driving characteristic from a plurality of historical driving characteristics based on the scenario data.

[0022] In the above process, the first historical driving characteristics being acquired are associated with scenario data, so that the scenario corresponding to the first data is the same as the scenario corresponding to the first historical driving characteristics, and the accuracy of determining whether the vehicle is abnormal in the first time period is increased.

[0023] In one possible implementation, annotating a driving record video of the vehicle based on the first time period may be performed by generating anomaly information and annotating the anomaly information in the driving record video based on the first time period.

[0024] In one possible implementation, the abnormality information includes at least one of the following: an abnormality level, an abnormality type, or an abnormality description information.

[0025] In the above process, the abnormality information in the driving record video is annotated, so that the user can accurately and quickly search the video clips based on the annotation results in the driving record video, which improves the searching efficiency of the video clips.

[0026] According to a second aspect, an embodiment of the present application provides a data processing apparatus including an acquisition module, a determination module, and an annotation module.

[0027] The acquisition module is configured to acquire first data and first historical driving characteristics, the first data including first driving data of a vehicle for a first time period, and the first historical driving characteristics are determined based on the historical driving data of the vehicle.

[0028] The determination module is configured to determine that the vehicle is not normal during the first time period based on the first data and the first historical driving characteristics.

[0029] The annotation module is configured to annotate driving footage of the vehicle based on the first time period.

[0030] In one possible implementation, the first operational data comprises: status data for components within the vehicle; Operational data of components within the vehicle; or and sensor data collected by components within the vehicle.

[0031] In one possible implementation, the first historical driving characteristic includes a plurality of historical data curves corresponding to a plurality of types of historical driving data, the historical data curves being used to indicate a distribution rule of the historical driving data, and the historical driving data being: historical condition data for said components within said vehicle; Historical operating data of the components in the vehicle; or and historical sensor data collected by the components in the vehicle.

[0032] In one possible implementation, the acquisition module is particularly A first model is configured to process the historical driving data to obtain the first historical driving characteristic.

[0033] The first model is obtained by learning a plurality of groups of first samples, each of the plurality of groups of first samples including sample driving data and sample historical driving characteristics.

[0034] In one possible implementation, the decision module is particularly determining a similarity between the first data and the first historical driving characteristics; and and determining that the vehicle is not normal during the first time period based on the similarity.

[0035] In one possible implementation, the decision module is particularly A second model is configured to process the first data and the first historical driving characteristics to obtain the similarity.

[0036] The second model is obtained by learning a plurality of groups of second samples, each of the plurality of groups of second samples including sample data, sample driving features, and sample similarities, and the sample data includes sample driving data.

[0037] In one possible implementation, the decision module is particularly determining a first data curve corresponding to the first data by using the second model; and and configured to compare the historical data curve and the first data curve for the first historical driving characteristic by using the second model to obtain the similarity.

[0038] In one possible implementation, the first data further includes scenario data, the scenario data being: The information includes at least one of time information, location information, road condition information, and weather information.

[0039] In one possible implementation, the decision module is particularly and configured to determine the first historical driving characteristic from a plurality of historical driving characteristics based on the scenario data.

[0040] In one possible implementation, the annotation module may, inter alia: Generate anomaly information, and and annotating the anomaly information in the driving record video based on the first time period.

[0041] In one possible implementation, the abnormality information comprises: The information includes at least one of the following: a level of abnormality, a type of abnormality, or a description of the abnormality.

[0042] According to a third aspect, an embodiment of the present application provides a data processing apparatus, the data processing apparatus including a memory and a processor, the memory storing computer program instructions, and the processor executing the computer program instructions to perform operations according to any implementation of the first aspect.

[0043] According to a fourth aspect, an embodiment of the present application provides a computer storage medium comprising computer instructions which, when executed by a processor, implement a method according to any implementation of the first aspect.

[0044] According to a fifth aspect, an embodiment of the present application provides a computer program product which, when executed by a processor, implements a method according to any implementation of the first aspect.

[0045] According to the data processing method and apparatus and device provided by the embodiments of this application, a first data and a first historical driving characteristic of a vehicle in a first time period can be obtained, and based on the first data and the first historical driving characteristic, it is determined that the vehicle is not normal in the first time period, and annotate a driving record video of the vehicle based on the first time period. The first data can reflect the driving state of the vehicle in the first time period, and the first historical driving characteristic can reflect the driving habits of the driver of the vehicle. Since the driving habits of a user are usually fixed, it is possible to determine whether the vehicle is not normal in the first time period based on the first driving data and the first historical driving characteristic. When it is determined that the vehicle is not normal in the first time period, the driving record video captured by the image capture device can be annotated to record that the vehicle is not normal in the first time period. In this way, the driving record video can be accurately annotated. A user can accurately and quickly search for video clips based on the annotation results of the driving video, thereby improving the efficiency of searching for video clips. [Brief description of the drawings]

[0046] [Figure 1] FIG. 1 is a schematic diagram of an application scenario according to an embodiment of the present application. [Figure 2A] FIG. 1 is a schematic diagram of a video capture scheme according to one embodiment of the present application. [Figure 2B] FIG. 2 is a schematic diagram of another video capture scheme according to an embodiment of the present application. [Figure 2C] FIG. 2 is a schematic diagram of yet another video capture scheme according to an embodiment of the present application. [Diagram 3] 1 is a schematic flowchart of a data processing method according to an embodiment of the present application. [Figure 4]FIG. 2 is a schematic diagram of driving data and driving characteristics according to an embodiment of the present application. [Figure 5A] FIG. 2 is a schematic diagram of a scheme for acquiring historical driving data according to one embodiment of the present application. [Figure 5B] FIG. 2 is a schematic diagram of another manner of acquiring historical driving data according to an embodiment of the present application. [Figure 6] FIG. 2 is a schematic diagram of a history operation characteristic according to an embodiment of the present application. [Figure 7A] FIG. 2 is a schematic diagram of similarity determination according to one embodiment of the present application; [Figure 7B] FIG. 2 is a schematic diagram of similarity determination according to an embodiment of the present application; [Figure 8A] 1 is a schematic diagram of a video viewing method according to one embodiment of the present application. [Figure 8B] 1 is a schematic diagram of a video viewing method according to one embodiment of the present application. [Figure 8C] 1 is a schematic diagram of a video viewing method according to one embodiment of the present application. [Figure 9] 1 is a schematic diagram of a video annotation scheme according to one embodiment of the present application; [Figure 10A] 1 is a schematic diagram of a video viewing method according to one embodiment of the present application. [Figure 10B] 1 is a schematic diagram of a video viewing method according to one embodiment of the present application. [Figure 10C] 1 is a schematic diagram of a video viewing method according to one embodiment of the present application. [Figure 10D] 1 is a schematic diagram of a video viewing method according to one embodiment of the present application. [Figure 10E] 1 is a schematic diagram of a video viewing method according to one embodiment of the present application. [Figure 10F] 1 is a schematic diagram of a video viewing method according to one embodiment of the present application. [Figure 11] 4 is a schematic flowchart of another data processing method according to an embodiment of the present application. [Figure 12A] 1 is a schematic diagram of a configuration of a driving recorder according to an embodiment of this application; [Figure 12B] FIG. 11 is a schematic diagram of another driving recorder configuration according to an embodiment of the present application. [Figure 13A] FIG. 2 is a schematic diagram of device connections according to an embodiment of the present application. [Figure 13B] FIG. 13 is a schematic diagram of another device connection according to an embodiment of the present application. [Figure 13C] FIG. 13 is a schematic diagram of yet another device connection according to an embodiment of the present application. [Figure 14A] FIG. 1 is a schematic diagram of a vehicle configuration according to one embodiment of the present application. [Figure 14B] FIG. 2 is a schematic diagram of another configuration of a vehicle according to an embodiment of the present application. [Figure 15] 1 is a schematic diagram of a configuration of a data processing apparatus according to an embodiment of this application; [Figure 16] 1 is a schematic diagram of a hardware configuration of a data processing device according to an embodiment of this application; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0047] To facilitate understanding, we first explain several concepts in this application.

[0048] Vehicles: In this application, vehicles include cars, buses, cargo vans, passenger cars, motorcycles, and the like.

[0049] Components in a vehicle: Components in a vehicle may be parts or sensors in the vehicle, and the parts may be components that keep the vehicle running. For example, the parts may include an engine, a generator, a shock absorber, or vehicle lights, etc. The sensors may include a temperature sensor, a speed sensor, or an acceleration sensor, etc.

[0050] Driving data: Data generated during the driving process of a vehicle. The driving data may include at least one of status data of a component, operation data of a component, or sensor data collected by a component.

[0051] Status data is used to indicate the status of a component, for example, vehicle light status data is used to indicate that the vehicle light status is on or off, engine status data is used to indicate that the engine status is starting or paused, and solenoid valve status data is used to indicate that the solenoid valve status is on or off.

[0052] Operational data is used to refer to data generated during the operational process of a component, for example, engine operation data may include engine rotation speed, electronic transmission operation data may include electronic transmission gears, and battery operation data may include remaining battery power.

[0053] Sensor data refers to data collected by a sensor. For example, sensor data may include temperature collected by a temperature sensor, speed collected by a speed sensor, and air-fuel ratio collected by an air-fuel ratio sensor.

[0054] Different components may generate different data, and components may generate one or more of status data, operational data, and sensor data. For example, some of the components may generate status data, some of the components may generate status data and operational data, and some of the components may generate sensor data. For example, a ceiling switch may generate status data, an engine may generate status data and operational data, and a temperature sensor may generate sensor data.

[0055] Driving Recording Video: Video obtained by photographing the vehicle during the driving process. The video may include video obtained by photographing the surroundings of the vehicle body and video obtained by photographing the interior of the vehicle. The video is captured by an image capturing device. The image capturing device may be a device built into the vehicle or a device installed on the vehicle.

[0056] For ease of understanding, the following description will refer to FIG. 1 to explain an application scenario to which this application is applicable.

[0057] 1 is a schematic diagram of an application scenario according to an embodiment of this application. As shown in FIG. 1, a vehicle includes an engine electronic system, a chassis electronic system, a body electronics system, a safety and comfort system, an entertainment and communication system, and an image capture device.

[0058] The engine electronic system may include an engine management electronic control unit (ECU), battery, generator, starter, temperature sensors, knock sensors, air-fuel ratio sensors, oxygen sensors, engine harness, cooling system, ignition system, intake and exhaust systems, transmission system, electric feed pump, etc.

[0059] The chassis electronic systems may include a steering system, a suspension system, a braking system, and the like.

[0060] Body electronics may include business continuity management (BCM) systems, relays / fuses, floor wiring, door wiring, ceiling wiring, dashboard wiring, universal serial bus (USB) / high-definition multimedia interface (HDMI) cables, power rearview mirrors, window lifting motors, power rear door pillars, door and window switches, wiper motors, sunroof motors, on-board diagnostics (OBD), lighting systems, and switches, etc.

[0061] The safety and comfort systems may include safety systems, seat adjustment motors, active noise cancellation units, horns, air conditioning systems, and the like.

[0062] Entertainment and communication systems may include human-machine interaction (HMI) systems and communication systems, among others.

[0063] The image capture device may be located outside the vehicle and / or inside the vehicle, and there may be one or more image capture devices. For example, the image capture device may be located at point A, point B, point C, or point D of the vehicle. The image capture device may capture a video. For example, the image capture device may capture a video during the running (or starting) process of the vehicle. Optionally, a storage space may be set in the vehicle, and the video captured by the image capture device may be stored in the storage space. Optionally, a wireless communication module may be located in the vehicle, and the wireless communication module may transmit the video captured by the image capture device to a cloud server, or transmit the video captured by the image capture device to a user's terminal device (such as a device such as a mobile phone or a computer).

[0064] It should be noted that the above descriptions are merely examples of components that may be contained within a vehicle and do not constitute limitations on the components within the vehicle.

[0065] During a driving process (or an operation process) of a vehicle, driving data of the vehicle may be obtained. A historical driving characteristic of the vehicle may be determined based on the historical driving data of the vehicle, and the historical driving characteristic of the vehicle may represent a driving habit of a driver of the vehicle. The first driving data of the vehicle in a first time period (e.g., a time period before a current time) may indicate a driving state of the vehicle in the first time period. Since a user's driving habit is usually fixed, it is possible to determine whether the vehicle is abnormal in the first time period by comparing the historical driving characteristic of the vehicle and the first driving data. When determining that the vehicle is abnormal in the first time period, annotate the driving record video captured by the image capture device to record that the vehicle is abnormal in the first time period. Optionally, the abnormality of the vehicle may include a vehicle malfunction, a car accident, or an abnormal driving of the user. The abnormal driving of the user may include the user driving at an excessively high speed, the user driving at an excessively low speed, the user driving the vehicle in a wandering manner in an area, and the like.

[0066] A user can use the terminal device to watch a driving record video (which is a video taken by an image capture device or a video obtained after annotating the video taken by an image capture device). Optionally, the terminal device may obtain the driving record video through multiple ways. The following description will describe some of the multiple ways of obtaining the driving record video by the terminal device with reference to Figures 2A to 2C.

[0067] 2A is a schematic diagram of a video capture method according to an embodiment of this application. As shown in FIG. 2A, there is a vehicle A, a removable hard disk B, and a terminal device C.

[0068] The driving record video may be stored in a pre-set storage space in the vehicle A. In this case, when a user needs to watch the driving record video by using the terminal device C, the user may copy the driving record video in the storage space to the removable hard disk B, and then copy the driving record video to the terminal device C.

[0069] The driving record video may be stored in a removable hard disk B. For example, the removable hard disk B is inserted into a vehicle, and the video captured by the image capture device is stored in the removable hard disk B. In this case, when a user needs to watch the driving record video by using a terminal device C, the user may remove the removable hard disk B from the vehicle, and then copy the driving record video in the removable hard disk B to the terminal device C.

[0070] FIG. 2B is a schematic diagram of another video acquisition method according to an embodiment of this application. As shown in FIG. 2B, there is a vehicle A and a terminal device B. Driving record videos may be stored in a pre-configured storage space in the vehicle A. A communication module is disposed in the vehicle A, and the vehicle A may communicate with the terminal device B by using the communication module. For example, the vehicle A may communicate with the terminal device B by Bluetooth or Wi-Fi. An application may be installed in the terminal device B, and a user may use the application to control the transmission of the videos in the pre-configured storage space of the vehicle A to the terminal device B. For example, a user may select a driving record video of a certain time period by using the application, so that the vehicle transmits the driving record video of the certain time period to the terminal device.

[0071] FIG. 2C is a schematic diagram of yet another video acquisition method according to an embodiment of this application. As shown in FIG. 2C, there are a vehicle A, a terminal device B, and a cloud server C. A communication module is disposed in the vehicle A. The vehicle A uploads driving record videos to the cloud server C by using the communication module, and the cloud server C stores the driving record videos. An application may be installed in the terminal device B, and a user may control the terminal device B to download the driving record videos from the cloud server C by using the application. For example, a user may select a driving record video of a certain time period by using the application, so that the terminal device B requests to download the driving record video of the certain time period from the cloud server C.

[0072] It should be noted that Figures 2A to 2C merely illustrate several examples of ways in which a terminal device may obtain driving footage, and do not constitute limitations on those ways.

[0073] The following description will use several specific embodiments to describe several technical solutions of this application in detail. It should be noted that the following specific embodiments may exist independently or may be combined with each other, and the same or similar contents will not be described repeatedly in several different embodiments.

[0074] 3 is a schematic flowchart of a data processing method according to an embodiment of this application. Please refer to FIG. 3. The method may include the following steps:

[0075] S301: A first model is obtained.

[0076] The execution subject of this embodiment of this application may be a vehicle, a driving recorder, or a data processing device disposed in the vehicle or the driving recorder. The data processing device may be implemented by using software, or may be implemented by using a combination of software and hardware. For example, the data processing device may be a processing chip.

[0077] For ease of explanation, the following description uses one example in which the executing subject is a driving recorder for illustration purposes.

[0078] The first model is used to process the driving data to obtain corresponding driving characteristics. The driving characteristics may be represented by using a curve. For example, the driving characteristics may represent a distribution rule of the driving data.

[0079] With reference to FIG. 4, the following description will explain the driving data and driving characteristics by using one example in which the driving data is acceleration.

[0080] FIG. 4 is a schematic diagram of driving data and driving characteristics according to an embodiment of this application. As shown in FIG. 4, the horizontal axis of the coordinate system represents a road condition index, which may be determined based on parameters such as road congestion and weather. The vertical axis of the coordinate system represents normalized acceleration. The points in the coordinate system represent discrete accelerations (driving data), which may be accelerations collected for multiple different road conditions in a preset time period. The curves in the coordinate system represent driving characteristics (which may also be referred to as acceleration characteristics), which may indicate the distribution rule of acceleration.

[0081] Alternatively, the first model can be obtained in the following manner.

[0082] First method: A device other than the driving recorder generates the first model, and the driving recorder obtains the first model from the device.

[0083] Alternatively, the device may be a server or a terminal device, etc.

[0084] The device may generate the first model by obtaining a plurality of groups of first samples and learning the plurality of groups of first samples to obtain the first model, each group of first samples including sample driving data and sample driving features.

[0085] Optionally, the sample driving data of a group of first samples may be one type of driving data. The type of driving data may be speed, acceleration, engine speed, or engine temperature, etc. For example, the sample driving data in a group of first samples may be speed during a preset time period. In a group of sample data, a sample driving feature corresponds to the sample driving data, and the sample driving feature is used to reflect the feature of the sample driving data. Optionally, the sample driving feature may be represented by a sample curve, and the sample curve is used to show the distribution rule of the sample driving data. The sample driving feature may be obtained by manually annotating the sample driving data.

[0086] The algorithm for training the first model may include a logistic regression algorithm, a decision tree algorithm, a vector machine algorithm, a naive Bayes algorithm, and the like.

[0087] After obtaining the first model by learning a plurality of groups of the first samples, the first model has the function that after inputting driving data into the first model, the first model is capable of processing the driving data and outputting driving features corresponding to the driving data.

[0088] After a device other than the driving recorder acquires the first model by learning, the driving recorder may acquire the first model from the device. The first model may be an offline model.

[0089] Second method: The driving recorder generates the first model.

[0090] It should be noted that the process of the driving recorder generating the first model should be referred to the first method, and the details will not be described repeatedly in this specification.

[0091] S302: Obtain historical driving data of the vehicle.

[0092] The historical operation data may be data generated during a historical driving process of the vehicle. The historical driving process may be a driving process of the vehicle in a historical time period. The historical time period may be one week, one month, half a year, or one year before the current time. The historical time period may be set based on an actual situation.

[0093] Optionally, the historical operating data includes at least one of historical condition data of components in the vehicle and historical operating data of components in the vehicle or historical sensor data collected by components in the vehicle.

[0094] It should be noted that for the description of the historical driving data, reference should be made to the above description of the driving data, and the details will not be repeated in this specification.

[0095] Optionally, after starting the vehicle, most of the components in the vehicle start to operate and generate driving data. The driving recorder can acquire the historical driving data generated by the vehicle. For ease of understanding, the following description will refer to Figures 5A and 5B to describe the manner in which the driving recorder acquires the historical driving data.

[0096] FIG. 5A is a schematic diagram of acquiring historical driving data according to an embodiment of the present application. As shown in FIG. 5A, a vehicle includes a plurality of components and a vehicle body state system, and the plurality of components includes a plurality of parts and a plurality of sensors. After the vehicle is started, most of the components can start to operate. The components that start to operate can generate driving data, and the vehicle body state system can acquire and store the driving data generated by the components. The driving recorder and the vehicle body state system may communicate with each other. For example, the driving recorder and the vehicle body state system may be connected by a wired manner or a wireless manner, so that the driving recorder may acquire the historical driving data from the vehicle body state system.

[0097] FIG. 5B is a schematic diagram of another manner of acquiring historical driving data according to an embodiment of the present application. As shown in FIG. 5B, a vehicle includes multiple components and a vehicle body state system, and the multiple components include multiple parts and multiple sensors. After the vehicle is started, most components can start to operate. The components that start to operate can generate driving data. The vehicle body state system may acquire the driving data generated by the components, and transmit the driving data generated by the components to a cloud server. The cloud server stores the driving data. When the driving recorder needs to acquire the historical driving data, the driving recorder may request to acquire the historical driving data from the cloud server.

[0098] It should be noted that Figures 5A and 5B are merely examples of how a driving recorder may obtain historical driving data and do not constitute limitations.

[0099] S303: Process the historical driving data by using a first model to obtain a first historical driving characteristic.

[0100] The historical driving data may include multiple types of historical driving data, such as speed, acceleration, engine speed, engine temperature, and the like.

[0101] The first historical driving feature includes driving features corresponding to multiple types of historical driving data. For example, the first historical driving feature includes driving features corresponding to past speed, driving features corresponding to past acceleration, or driving features corresponding to past engine speed. The driving features corresponding to past speed are used to indicate a distribution rule of past speed, and the driving features corresponding to past acceleration are used to indicate a distribution rule of past acceleration.

[0102] For a vehicle, the vehicle is usually driven by one or more fixed users. For any user, the driving habits of the user are usually fixed, and as a result, the driving data of the vehicle is usually distributed as a fixed curve. For example, for the same user and the same vehicle, the speed at which the user drives the vehicle under different road conditions is usually distributed as a fixed speed curve, and the acceleration at which the user drives the vehicle under different road conditions is usually distributed as a fixed acceleration curve. Therefore, the first historical driving characteristic determined based on the historical driving data may represent the driving habits of the user, or the first historical driving characteristic may also be referred to as the user profile of the user who drives the vehicle.

[0103] The historical driving data may be input to a first model, and the first model outputs a first historical driving characteristic corresponding to the historical driving data. The first historical driving characteristic includes a plurality of historical data curves corresponding to different types of historical driving data. Each of the plurality of historical data curves is used to indicate a distribution rule of the historical driving data. That is, each of the plurality of historical data curves is used to represent a driving characteristic corresponding to the historical driving data.

[0104] The following description will explain the first historical operating characteristic with reference to FIG.

[0105] FIG. 6 is a schematic diagram of a historical driving feature according to an embodiment of this application. As shown in FIG. 6, there are multiple curves in a coordinate system, each curve representing one historical driving feature, and one historical driving feature corresponding to one type of historical driving data. It should be noted that for different curves, the horizontal and vertical axes represent different meanings. For example, when the historical driving feature corresponds to acceleration, the horizontal axis may represent a road condition index, and the vertical axis may represent acceleration. When the historical driving feature corresponds to speed, the horizontal axis may represent a road condition index, and the vertical axis may represent speed.

[0106] FIG. 6 merely illustrates one example of the first historical driving characteristic and does not constitute a limitation on the first historical driving characteristic.

[0107] S304: Obtain a second model.

[0108] The second model is used to obtain a similarity between the data and the driving characteristics. The data may include driving data, or the data may include driving data and scenario data.

[0109] Alternatively, the second model may be obtained in the following manner.

[0110] First method: A device other than the driving recorder generates the second model, and the driving recorder obtains the second model from that device.

[0111] Alternatively, the device may be a server or a terminal device, etc.

[0112] The device may generate the second model by obtaining a plurality of groups of second samples, and learning the plurality of groups of second samples to obtain a second model, each group of second samples including sample data, sample driving features, and sample similarities.

[0113] Alternatively, the sample data may include sample driving data, or the sample data may include sample driving data and sample scenario data. The sample scenario data includes at least one of sample time information, sample location information, sample road condition information, or sample weather information.

[0114] The sample similarity may be a similarity between the sample data and the sample driving characteristics, and the sample similarity may be a manually annotated similarity.

[0115] Optionally, the sample driving data in one group of the second samples may be one type of driving data. The type of driving data may be speed, acceleration, rotation speed of a transmission, or engine temperature, etc. For example, the sample driving data in one group of the first samples may be speed in a preset time period. In one group of sample data, the sample driving characteristics correspond to the sample driving data. Optionally, the sample driving characteristics may be represented by using a sample curve.

[0116] Algorithms for training the second model include a logistic regression algorithm, a decision tree algorithm, a vector machine algorithm, a naive Bayes algorithm, and the like.

[0117] After obtaining the second model by learning a plurality of groups of the second samples, the second model has the function that after inputting data and driving features into the second model, the second model can process the input data and driving features to obtain similarity between the data and the driving features.

[0118] After a device other than the driving recorder acquires the second model by learning, the driving recorder may acquire the second model from the device. The second model may be an offline model.

[0119] Second method: The driving recorder generates a second model.

[0120] It should be noted that the process of the driving recorder generating the second model should refer to the first method, and the details will not be described repeatedly in this specification.

[0121] Optionally, the second model and the first model may be integrated into one model. For example, the model obtained by the integration may have the functions of the first model and the functions of the second model at the same time. Alternatively, the second model and the first model may be two independent models.

[0122] The process from S301 to S304 may be a one-time execution process or a periodic execution process, i.e., before S305, the process from S301 to S304 only needs to be executed once and does not need to be executed in real time.

[0123] S305: The first data is acquired.

[0124] Optionally, the first data includes first driving data of the vehicle in a first time period. Alternatively, the first data may include the first driving data and scenario data, and the scenario data includes at least one of time information, location information, road condition information, or weather information.

[0125] The first time period may be a time period that is prior to and closest to the current time point, for example, the first time period may be 5 minutes, 10 minutes, 30 minutes, 1 hour, etc., prior to the current time point.

[0126] It should be noted that for the manner of obtaining the first data by the driving recorder, refer to the manner of obtaining the historical driving data by the driving recorder in S302. Details are not repeated in this specification.

[0127] S306: Process the first data and the first historical driving characteristics by using a second model to obtain a similarity between the first data and the first historical driving characteristics.

[0128] The second model may input the first data and the first historical driving characteristics, so that the second model outputs a similarity between the first data and the first historical driving characteristics. For example, after inputting the first data and the first historical driving characteristics into the second model, the second model may first determine a first data curve corresponding to the first data, and then compare the historical data curve in the first historical driving characteristics with the first data curve to obtain the similarity. When the first data further includes scenario data, the second model may further obtain a historical data curve corresponding to the scenario data from the first historical driving characteristics, and then compare the historical data curve corresponding to the scenario data with the first data curve to obtain the similarity.

[0129] Optionally, the first data may include the first driving data, or the first data may include the first driving data and the scenario data. When the contents included in the first data are different, the process of determining the similarity between the first data and the first historical driving characteristics is different accordingly. The following two cases may be included:

[0130] Case 1: The first data includes first driving data.

[0131] The first driving data may include a plurality of types of driving data, and the first historical driving characteristics include historical driving characteristics corresponding to each type of driving data. A similarity between each type of driving data and the corresponding historical driving characteristics may be obtained separately, and then a similarity between the first data and the first historical driving characteristics is determined based on the similarity between each type of driving data and the corresponding historical driving characteristics.

[0132] For any type of driving data, a first data curve corresponding to that type of driving data may be obtained, a historical data curve corresponding to that type of driving data may be obtained from the first historical driving characteristics, and then a similarity between the first data curve and the corresponding historical data curve may be determined as a similarity between the type of driving data and the corresponding historical driving characteristics.

[0133] The following description, with reference to FIG. 7A, describes a process for determining the similarity between the first data and the first historical driving characteristics.

[0134] 7A is a schematic diagram of determining similarity according to an embodiment of the present application. As shown in FIG. 7A, the first data includes first driving data. The first driving data may include multiple types of driving data. Driving data 1, driving data 2, driving data 3, ..., driving data N (N is an integer greater than 1) are N different types of driving data. For example, driving data 1 may be speed, driving data 2 may be acceleration, and driving data 3 may be engine speed.

[0135] A data curve corresponding to each type of driving data may be obtained. Optionally, a second model may be used to obtain a data curve corresponding to each type of driving data. For example, driving data 1 may be input to the second model, whereby the second model outputs data curve 1 corresponding to driving data 1. Driving data 2 may be input to the second model, whereby the second model outputs data curve 2 corresponding to driving data 2. By analogy, it is possible to obtain N data curves corresponding to N types of driving data.

[0136] The first historical driving characteristic includes a plurality of historical data curves. For example, the first historical driving characteristic includes a historical data curve 1 corresponding to driving data 1 and a historical data curve 2 corresponding to driving data 2. The data curve 1 may be compared with the historical data curve 1 to obtain a similarity 1, the data curve 2 may be compared with the historical data curve 2 to obtain a similarity 2, and by analogy, N similarities may be obtained.

[0137] The similarity between the first data and the first historical driving characteristic may be determined as an average or weighted average of the N similarities.

[0138] Case 2: The first data includes first operating data and scenario data.

[0139] The first driving data may include multiple types of driving data. For example, the first driving data may include speed, acceleration, and engine speed. The scenario data includes at least one of time information, location information, road condition information, or weather information. A corresponding scenario type may be determined and obtained based on the scenario data. For example, the scenario types may include rainy weather, morning rush hour, evening rush hour, holiday, working day, and unblocked road conditions.

[0140] The first historical driving characteristics include a plurality of historical driving characteristics corresponding to each type of driving data, and the plurality of historical driving characteristics correspond to a plurality of different scenario types. For example, assuming the driving data is speed, the first historical driving characteristics may include a plurality of historical speed driving characteristics corresponding to speed, and the plurality of historical speed driving characteristics may include a historical speed driving characteristic corresponding to rainy weather, a historical speed driving characteristic corresponding to morning rush hour, and a historical speed driving characteristic corresponding to holidays, etc.

[0141] Based on the scenario data, a similarity between each type of driving data and the corresponding historical driving characteristics may be obtained separately, and then a similarity between the first data and the first historical driving characteristics may be determined based on the similarity between each type of driving data and the corresponding historical driving characteristics.

[0142] For any type of driving data, a first data curve corresponding to the driving data of that type may be obtained. From the first historical driving characteristics, a plurality of historical data curves corresponding to the driving data of that type are obtained. A scenario type is determined based on the scenario data. From the plurality of historical data curves, a first historical data curve corresponding to the scenario type is determined. A similarity between the first data curve and the first historical data curve is determined as a similarity between the driving data of that type and the historical driving characteristics.

[0143] The following description, with reference to FIG. 7B, describes a process for determining the similarity between the first data and the first historical driving characteristics.

[0144] 7B is a schematic diagram of determining similarity according to an embodiment of the present application. As shown in FIG. 7B, the first data includes the first driving data and the scenario data. The first driving data may include multiple types of driving data. Driving data 1, driving data 2, driving data 3, ..., driving data N (N is an integer greater than 1) are N different types of driving data. For example, driving data 1 may be speed, driving data 2 may be acceleration, and driving data 3 may be engine speed.

[0145] A data curve corresponding to each type of driving data may be obtained. Optionally, a second model may be used to obtain a data curve corresponding to each type of driving data. For example, driving data 1 may be input to the second model, whereby the second model outputs data curve 1 corresponding to driving data 1. Driving data 2 may be input to the second model, whereby the second model outputs data curve 2 corresponding to driving data 2. By analogy, it is possible to obtain N data curves corresponding to N types of driving data.

[0146] The first historical driving characteristic includes a plurality of historical data curves corresponding to each type of driving data. For example, the first historical driving characteristic includes a plurality of historical data curves corresponding to driving data 1 (e.g., historical data curve 11, historical data curve 12, ..., historical data curve 1X, etc.) and a plurality of historical data curves corresponding to driving data 2 (e.g., historical data curve 21, historical data curve 22, ..., historical data curve 2K, etc.). Based on the scenario data, a first historical data curve (assumed to be historical data curve 11) corresponding to the scenario data may be selected from the plurality of historical data curves corresponding to driving data 1, and a first historical data curve (assumed to be historical data curve 21) corresponding to the scenario data may be selected from the plurality of historical data curves corresponding to driving data 2 based on the scenario data. By analogy, a first historical data curve corresponding to each type of driving data is selected based on the scenario data. Data curve 1 may be compared to historical data curve 1 to obtain similarity 1, data curve 2 may be compared to historical data curve 2 to obtain similarity 2, and by analogy, N similarities may be obtained.

[0147] The similarity between the first data and the first historical driving characteristic may be determined as an average or weighted average of the N similarities.

[0148] It should be noted that Figures 7A and 7B show only one example of determining the similarity between the first data and the first historical driving characteristics, and do not constitute a limitation on the determination of the similarity. The similarity may be determined in other ways.

[0149] Furthermore, it should be noted that the similarity between the first data and the first historical driving characteristics may be obtained according to an algorithm. The process of obtaining the similarity according to an algorithm is similar to the process of obtaining the similarity by using the second model. The details are not repeated in this specification.

[0150] S307: Determine that the vehicle is not normal in the first time period based on the similarity.

[0151] The case where the vehicle is abnormal in the first time period may include the case where the vehicle malfunctions in the first time period, the case where the vehicle is involved in a vehicle accident in the first time period, or the case where the user drives the vehicle abnormally in the first time period. The case where the user drives the vehicle abnormally may include the case where the user drives at an excessively high speed, the case where the user drives at an excessively low speed, the case where the user drives the vehicle in a wandering manner in an area, etc.

[0152] Optionally, if the similarity is less than or equal to a similarity threshold, it is determined that the vehicle is not normal during the first time period.

[0153] S308: Annotate vehicle driving video based on the first time period.

[0154] A driving video of the vehicle may be annotated in a manner that generates anomaly information and annotates the anomaly information in the driving video based on the first time period.

[0155] The abnormality information includes at least one of abnormality indicator, abnormality level, abnormality type, or abnormality description information. The abnormality indicator is used to indicate that the vehicle is not normal. The abnormality level is used to indicate the degree of abnormality of the vehicle. For example, the lower the similarity determined in S306, the higher the degree of abnormality of the vehicle and the higher the abnormality level. The abnormality type may include abnormal braking, abnormal turning, abnormal acceleration, abnormal engine, abnormal driving path, etc. The abnormality description information is used to describe the specific situation of abnormality of the vehicle. For example, the abnormality description information may include five sudden brakings within one minute, accelerations greater than a first threshold, ten round trips in the same road section within one hour, ten lane changes within three minutes, etc.

[0156] After annotating the drive record video, the user may view the annotated drive record video by using a terminal device (such as a computer or cell phone). The following description describes the process of annotating a drive record video and a user viewing the annotated drive record video.

[0157] First video annotation method:

[0158] The image capture device can capture multiple driving record videos of the vehicle. For example, one run of the vehicle (from the time the vehicle is started to the time the vehicle stalls) may correspond to one driving record video, or one preset driving duration of the vehicle (e.g., one hour) may correspond to one driving record video. Multiple driving record videos of the vehicle for multiple runs may be stored in the list. When annotating the driving record videos, anomaly information is generated, and the anomaly information is used as attribute information of the driving record videos. In this way, the anomaly information may be displayed when displaying the driving record videos in the list. Furthermore, some information of the anomaly information (e.g., anomaly description information or some content of the anomaly description information) may be added to a corresponding position in the driving record video as a subtitle or a pop-up.

[0159] In this video annotation approach, the following description, with reference to Figures 8A-8C, illustrates how a user may view annotated drive footage video.

[0160] 8A to 8C are schematic diagrams of a video viewing method according to an embodiment of the present application. As shown in FIG. 8A to FIG. 8C, there are interfaces 801 to 803.

[0161] Referring to the interface 801, assuming that the manner in which the terminal device acquires the driving record video is shown in Fig. 2A, after the terminal device acquires the driving record video, the terminal device may display a folder corresponding to the driving record video. When the user needs to watch the driving record video, the user may tap the folder corresponding to the driving record video to display the driving record video list on the terminal device.

[0162] As shown in interface 802, there is a driving record video list. The list includes five driving record videos and attribute information of the five driving record videos. The attribute information includes a shooting time, a non-normality indicator, a non-normality level, a non-normality type, and a non-normality description information. Of course, the attribute information may further include other information. This is not limited thereto. The five driving record videos may be videos corresponding to five runs of a vehicle. From the driving record video list, it can be known that during the time periods corresponding to driving record video 1, driving record video 4, and driving record video 5, the vehicle is normal, and the non-normality indicators corresponding to the three driving record videos are "normal". During the time periods corresponding to driving record video 2 and driving record video 3, the vehicle is not normal, and the non-normality indicators corresponding to the two driving record videos are "non-normal". The two driving record videos further have attributes of a non-normality level, a non-normality type, and a non-normality description information. Optionally, the abnormality description information may be simple description information, and the user may tap the simple description information to display detailed description information on the terminal device. For example, after the user taps the simple description information of "frequent braking", the terminal device may display detailed description information of "eight sudden brakes between 20 minutes and 22 minutes". Optionally, in order to facilitate the user to watch the "abnormal" driving record video, the identifiers and attribute information of the "normal" driving record video and the "abnormal" driving record video may be displayed by a plurality of different display manners, and the display manners include font color, font type, font size, etc. For example, the identifiers and attribute information of the "normal" driving record video may be displayed in red font, and the identifiers and attribute information of the "abnormal" driving record video may be displayed in black font. The user may tap the identifier of the driving record video in the driving record video list to play the driving record video on the terminal device. For example, when the user needs to watch the driving record video 2, the user may tap the identifier of the driving record video 2 to play the driving record video 2 on the terminal device.

[0163] As shown in interface 803, the terminal device plays driving record video 2. A user may determine that the vehicle is not normal between 20 minutes and 22 minutes of driving record video 2 based on the abnormality description information of driving record video 2, and the user may drag the video playback progress bar based on the abnormality description information. When the user drags the progress bar to 20 minutes, the partial content of the abnormality description information (sudden braking) is displayed as a subtitle in the video.

[0164] In the above process, the user can easily obtain the driving record video corresponding to the abnormality in the vehicle based on the driving record video list, and can know the time period in the driving record video in which the vehicle becomes abnormal based on the abnormality description information, and can accurately identify the time period of the driving record video. These operations are convenient and flexible to improve the user experience.

[0165] It should be noted that the drive record videos may alternatively be videos of a preset duration. For example, each drive record video may have a duration of one hour. Optionally, the drive record videos with the non-normality indicator "non-normal" may alternatively be videos taken during the time period in which the vehicle experiences the non-normality. For example, drive record video 2 may have a duration of two minutes, and the vehicle experiences the non-normality within two minutes.

[0166] Secondary video annotation method:

[0167] One driving record video captured by the image capture device may correspond to one video file. When the vehicle becomes abnormal during the capture time period of the driving record video, a non-normality information file is generated. The non-normality information file may include one or more pieces of non-normality information, and the non-normality information file may be used as a configuration file of the video file. One piece of non-normality information in the non-normality information file may correspond to one non-normality time period (or non-normality time point) in the driving record video, and there is a relationship between the non-normality time period (or non-normality time point) in the driving record video and the non-normality information. For example, a corresponding non-normality time period (or non-normality time point) may be identified based on the non-normality information.

[0168] With reference to FIG. 9, the following describes the process of annotating a driving video.

[0169] FIG. 9 is a schematic diagram of a video annotation method according to an embodiment of this application. As shown in FIG. 9, when a vehicle is not normal in a shooting time period corresponding to a driving record video, a video file and an anomaly information file may be generated. For example, the video file may be an 80-minute video, and the anomaly information file may include three anomaly information items, anomaly information 1, anomaly information 2, and anomaly information 3, respectively. The anomaly information 1 may correspond to a time point t1 in the driving record video, i.e., the time point t1 in the driving record video may be identified based on the anomaly information 1. The anomaly information 2 may correspond to a time point t2 in the driving record video, i.e., the time point t2 in the driving record video may be identified based on the anomaly information 2. Anomaly Information 3 may correspond to time t3 in the driving record video, i.e., time t3 in the driving record video may be identified based on the anomaly information 3.

[0170] In this video annotation approach, the following description, with reference to Figures 10A-10F, illustrates how a user may view annotated drive footage video.

[0171] 10A to 10F are schematic diagrams of a video viewing method according to an embodiment of the present application. As shown in FIG. 10A to 10F, the video viewing method includes interfaces 1001 to 1006.

[0172] Assume that the manner in which the terminal device obtains the driving record video is as shown in FIG. 2B or FIG. 2C, as shown in the interface 1001. Install an application in the terminal device. Assume that the driving record video is transmitted to the terminal device by using the application. When a user needs to watch the driving record video by using the terminal device, the user may launch the application in the terminal device. The application includes a "video playback" icon. The user may tap the icon to display the interface 1002 on the terminal device. The interface 1001 may further include other icons. The icons included in the interface 1001 are not particularly limited in this embodiment of the application.

[0173] As shown in interface 1002, there are an "all videos" icon and a "non-normality video" icon. The page corresponding to the "all videos" icon includes identifiers of all videos, and the page corresponding to the "non-normality video" icon includes identifiers of the non-normality video. When a user needs to view a non-normality video, the user may tap the "non-normality video" icon to display interface 1003 on the terminal device. A non-normality video is a video whose captured time period includes a time period during which the vehicle is not normal.

[0174] As shown in interface 1003, it includes a non-normal video list. A user may tap any non-normal video to play the non-normal video on the terminal device. For example, when a user needs to watch video 2, the user taps video 2 to display interface 1004 on the terminal device.

[0175] As shown in the interface 1004, it includes a video playback window M and at least one "abnormality point" icon. The number of "abnormality point" icons corresponds to the number of abnormalities in the abnormality video. For example, assume that video 2 includes three abnormalities, and accordingly includes three "abnormality point" icons in the interface, including an "abnormality point 1" icon, an "abnormality point 2" icon, and an "abnormality point 3" icon. Optionally, the three "abnormality point" icons may be generated based on the three abnormality information in FIG. 9. When a user needs to watch the video clip corresponding to the abnormality point 1, the user may tap the "abnormality point 1" icon to display the interface 1005 on the terminal device.

[0176] As shown in interface 1005, the video playback progress bar in the video playback window M jumps to time t1, and the abnormality description information (severe braking) is displayed in the video. When a user needs to watch the video clip corresponding to abnormal point 2, the user may tap the "abnormal point 2" icon to display interface 1006 on the terminal device.

[0177] As shown in the interface 1006, the playback progress bar of the animation in the animation playback window M jumps to time t2, and the abnormality description information (sharp turn) is displayed in the animation.

[0178] In the above process, the user can conveniently watch the abnormality video by using the application in the terminal device, and can quickly identify the abnormality time period based on the "abnormality point" icon. These operations are convenient and flexible, improving the user experience.

[0179] According to the data processing method provided by the embodiments of the present application, first data and first historical driving characteristics of a vehicle in a first time period can be obtained, and based on the first data and the first historical driving characteristics, it can be determined that the vehicle is not normal in the first time period, and a driving record video of the vehicle can be annotated based on the first time period. The first data can reflect the driving state of the vehicle in the first time period, and the first historical driving characteristics can reflect the driving habits of the driver of the vehicle. Since the driving habits of a user are usually fixed, it can be determined whether the vehicle is not normal in the first time period by comparing the first driving data and the historical driving characteristics of the vehicle. When determining that the vehicle is not normal in the first time period, the driving record video captured by the image capture device can be annotated to record that the vehicle is not normal in the first time period. In this way, the driving record video can be accurately annotated. A user can accurately and quickly search for video clips based on the annotation result of the driving record video, thereby improving the efficiency of searching for video clips.

[0180] It should be noted that the processing steps (S301 to S308) shown in the embodiment in Fig. 3 do not constitute any particular limitation to the data processing process. In some of the other embodiments of this application, the data processing process may include more or fewer steps than those in the embodiment in Fig. 3. For example, the data processing process may include some of the steps in the embodiment in Fig. 3, or may replace some of the steps in the embodiment in Fig. 3 with steps having the same functions, or may branch some of the steps in the embodiment in Fig. 3 into multiple steps.

[0181] The following describes another possible data processing method with reference to FIG.

[0182] 11 is a schematic flowchart of another data processing method according to an embodiment of the present application. Please refer to FIG. 11. The method may include the following steps.

[0183] S1101: Obtain first data and first historical driving characteristics.

[0184] The execution subject of this embodiment of this application may be a vehicle, a driving recorder, or a data processing device disposed in the vehicle or the driving recorder. The data processing device may be implemented by using software, or may be implemented by using a combination of software and hardware. For example, the data processing device may be a processing chip.

[0185] The first data includes first driving data for the vehicle during a first time period, and the first historical driving characteristics are determined based on the historical driving data for the vehicle.

[0186] It should be noted that for the process of acquiring the first historical driving characteristics, refer to the process of executing S303. For the process of acquiring the first data, refer to the process of executing S305. Details will not be described repeatedly in this specification.

[0187] S1102: Determine that the vehicle is not normal during the first time period based on first data and first historical driving characteristics.

[0188] It should be noted that the process of performing S1102 should refer to the process of performing S306 and S307, and details will not be described in this specification.

[0189] S1103: Annotate vehicle driving video based on a first time period.

[0190] It should be noted that the process of executing S1103 should refer to the process of executing S308, and details will not be described in this specification.

[0191] In the embodiment shown in FIG. 11, first data and first historical driving characteristics of a vehicle in a first time period may be obtained, and based on the first data and the first historical driving characteristics, it is determined that the vehicle is not normal in the first time period, and annotate a driving record video of the vehicle based on the first time period. The first data may reflect a driving state of the vehicle in the first time period, and the first historical driving characteristics may reflect a driving habit of a driver of the vehicle. Since a user's driving habit is usually fixed, it is possible to determine whether the vehicle is not normal in the first time period by comparing the first driving data and the historical driving characteristics of the vehicle. When determining that the vehicle is not normal in the first time period, the driving record video captured by the image capture device may be annotated to record that the vehicle is not normal in the first time period. In this way, the driving record video may be accurately annotated. A user may accurately and quickly search for video clips based on the annotation result of the driving record video, thereby improving the efficiency of searching for video clips.

[0192] An embodiment of the present application further provides a driving recorder. The following description will explain the driving recorder with reference to Figures 12A and 12B.

[0193] 12A is a schematic diagram of a configuration of a driving recorder according to an embodiment of this application. As shown in FIG. 12A, the driving recorder 120 includes a processing chip 121 and an image capture device 122.

[0194] The image capture device 121 is configured to capture a driving record video.

[0195] The processing chip 122 is configured to obtain first data and first historical driving characteristics, determine that the vehicle is not normal during the first time period based on the first data and the first historical driving characteristics, and annotate a driving record video of the vehicle based on the first time period, where the first data includes first driving data of the vehicle during the first time period, and the first historical driving characteristics are determined based on the historical driving data of the vehicle.

[0196] Optionally, the processing chip 122 may further execute the data processing methods shown in the above method embodiments, whose processes and beneficial effects are similar, and the details are not repeated in this specification.

[0197] Fig. 12B is a schematic diagram of another driving recorder configuration according to an embodiment of the present application. Based on Fig. 12A, as shown in Fig. 12B, the driving recorder 120 further includes a memory 123, and the memory 123 is configured to store driving record video.

[0198] Optionally, after image capture device 121 captures and acquires the drive recording video, it may transmit the drive recording video to memory 123 where it may be stored.

[0199] Optionally, the memory 123 may further store program instructions, and the processing chip 122 may execute the technical solutions shown in the above method embodiments according to the program instructions.

[0200] As shown in FIG. 12B, the driving recorder 120 may further include a communication module 124, which may communicate with other devices (e.g., the vehicle or a cloud server).

[0201] Optionally, the driving recorder 120 may further include a data collection device. The data collection device may include a camera, a millimeter wave radar, a laser radar, an inertial sensor, etc. The data collection device is capable of acquiring the first data.

[0202] In the practical application process, the driving recorder 120 is usually placed in a vehicle. The following description will refer to Figures 13A to 13C to describe the connection relationship between the driving recorder and the vehicle. In Figures 13A to 13C, the driving recorder in Figure 12B is used for description.

[0203] 13A is a schematic diagram of device connections according to an embodiment of the present application. As shown in FIG. 13A, the interface may be located in the communication module 124 of the driving recorder 120, or the interface may be located in the vehicle.

[0204] In an actual application process, an interface on the driving recorder 120 may be connected to an interface on the vehicle, so that the driving recorder and the vehicle can communicate with each other by using those interfaces.

[0205] It should be noted that the interface may alternatively be located at other positions in the driving recorder, which is not particularly limited in this embodiment of the application.

[0206] Fig. 13B is a schematic diagram of another device connection according to an embodiment of the present application. As shown in Fig. 13B, the communication module is disposed in the vehicle, and the communication module 124 in the driving recorder 120 may communicate (e.g., by wireless communication) with the communication module in the vehicle. For example, the vehicle may transmit driving data to the driving recorder by using the communication module.

[0207] FIG. 13C is a schematic diagram of yet another device connection according to an embodiment of the present application. As shown in FIG. 13C, a communication module is disposed in a vehicle, and the communication module in the vehicle may communicate with a server. For example, the vehicle may send driving data to a cloud server by using the communication module. The driving recorder 120 may communicate with the cloud server by using the communication module 124. For example, the driving recorder 120 may obtain driving data from the cloud server by using the communication module 124.

[0208] It should be noted that Figures 13A-13C depict only one example of a vehicle configuration and do not constitute a limitation on the vehicle configuration.

[0209] An embodiment of the present application further provides a vehicle. The following description will explain the configuration of the vehicle with reference to Figures 14A and 14B.

[0210] 14A is a schematic diagram of a vehicle configuration according to an embodiment of the present application. As shown in FIG. 14A, a vehicle 140 may include a processing chip 141 and an image capture device 142.

[0211] Image capture device 142 is configured to capture a driving record video.

[0212] Processing chip 141 is configured to obtain first data and first historical driving characteristics, determine that the vehicle is not normal during the first time period based on the first data and the first historical driving characteristics, and annotate the drive record video based on the first time period, where the first data includes first driving data of the vehicle during the first time period, and the first historical driving characteristics are determined based on the historical driving data of the vehicle.

[0213] Optionally, the processing chip 141 may further implement the data processing methods shown in the above method embodiments, whose processes and beneficial effects are similar, and the details are not described repeatedly in this specification.

[0214] Fig. 14B is a schematic diagram of another configuration of a vehicle according to an embodiment of the present application. Based on Fig. 14A, as shown in Fig. 14B, the vehicle may further include a memory 143, which is configured to store driving record video.

[0215] Optionally, an image capture device 142 After the driver captures and acquires the drive record video, the driver may transmit the drive record video to memory 143 where the drive record video is stored.

[0216] Optionally, the memory 143 may further store program instructions, such as a processing chip. 141 may implement the technical solutions shown in the above method embodiments according to those program instructions.

[0217] 14B, the vehicle further includes a vehicle body status system 144. The vehicle body status system 144 may obtain the first driving data and transmit the first driving data to the processing chip 141. The vehicle further includes components. The components include a plurality of parts and a plurality of sensors. The vehicle body status system 144 may obtain the first driving data from the components of the vehicle.

[0218] Optionally, vehicle 140 may further include a communications module (not shown) that may communicate with other devices (e.g., a cloud server, etc.).

[0219] Optionally, the vehicle 140 may further include a data collection device. The data collection device may include a camera, a millimeter wave radar, a laser radar, an inertial sensor, etc. The data collection device is capable of acquiring the first data.

[0220] 15 is a schematic diagram of a configuration of a data processing device according to an embodiment of this application. As shown in FIG. 15, the data processing device 150 may include an acquisition module 151, a determination module 152, and an annotation module 153.

[0221] The acquisition module 151 is configured to acquire first data and first historical driving characteristics, the first data including first driving data of the vehicle in a first time period, and the first historical driving characteristics determined based on the historical driving data of the vehicle.

[0222] The determination module 152 is configured to determine that the vehicle is abnormal during a first period of time based on the first data and the first historical driving characteristics.

[0223] The annotation module 153 is configured to annotate the vehicle drive footage based on the first time period.

[0224] Optionally, the acquisition module 151 may execute S303 and S305 in the embodiment shown in FIG. 3, and S1101 in the embodiment shown in FIG.

[0225] Optionally, the determination module 152 may execute S306 and S307 in the embodiment shown in FIG. 3, and S1102 in the embodiment shown in FIG.

[0226] Optionally, the annotation module 153 may execute S308 in the embodiment shown in FIG. 3 and S1103 in the embodiment shown in FIG.

[0227] The data processing device shown in this embodiment of this application may implement the technical solutions shown in the above method embodiments. The implementation principles and beneficial effects of the data processing device are similar to those of the multiple technical solutions, and the details are not described repeatedly in this specification.

[0228] In one possible implementation, the first operational data includes: status data of components within the vehicle; Operational data of components within the vehicle, or and sensor data collected by components within the vehicle.

[0229] In one possible implementation, the first historical driving characteristic includes a plurality of historical data curves corresponding to a plurality of types of historical driving data, the historical data curves are used to indicate a distribution rule of the historical driving data, and the historical driving data is: Historical condition data of components within the vehicle; Historical operating data of components within the vehicle; or and historical sensor data collected by components within the vehicle.

[0230] In one possible implementation, the acquisition module comprises, inter alia: The first model is configured to process the historical driving data to obtain a first historical driving characteristic by using the first model.

[0231] The first model is obtained by learning a plurality of groups of first samples, each of the plurality of groups of first samples including sample driving data and sample historical driving characteristics.

[0232] In one possible implementation, the decision module 152 may, inter alia: determining a similarity between the first data and the first historical driving characteristics; and and determining that the vehicle is not normal during the first period of time based on the similarity.

[0233] In one possible implementation, the decision module 152 may, inter alia: The second model is configured to process the first data and the first historical driving characteristics to obtain the similarity by using the second model.

[0234] The second model is obtained by learning a plurality of groups of second samples, each of the plurality of groups of second samples including sample data, sample driving features, and sample similarities, and the sample data includes sample driving data.

[0235] In one possible implementation, the decision module 152 may, inter alia: determining a first data curve corresponding to the first data by using the second model; and The second model is configured to compare the historical data curve and the first data curve in the first historical driving characteristic to obtain a similarity.

[0236] In one possible implementation, the first data further includes scenario data, the scenario data comprising: The information includes at least one of time information, location information, road condition information, and weather information.

[0237] In one possible implementation, the decision module 152 may, inter alia: A first historical driving characteristic is configured to be determined from the plurality of historical driving characteristics based on the scenario data.

[0238] In one possible implementation, the annotation module 153 may, inter alia: Generate anomaly information, and The method is configured to annotate the anomaly information in the driving record video based on the first time period.

[0239] In one possible implementation, the anomaly information is The information includes at least one of the following: a level of abnormality, a type of abnormality, or a description of the abnormality.

[0240] It should be noted that the data processing device shown in this embodiment of this application may implement the technical solutions shown in the above method embodiments, and the implementation principles and beneficial effects of the data processing device are similar to those of those technical solutions, and the details are not repeated in this specification.

[0241] Fig. 16 is a schematic diagram of a hardware configuration of a data processing device according to an embodiment of this application. As shown in Fig. 16, the data processing device 160 may include a processor 161 and a memory 162. The processor 161 and the memory 162 can communicate with each other. For example, the processor 161 and the memory 162 communicate with each other by using a communication bus 163. The memory 162 is configured to store program instructions, and the processor 161 is configured to execute the program instructions in the memory to execute the data processing method shown in any one of the above method embodiments.

[0242] Alternatively, the processor 161 may execute the steps of the embodiments shown in FIG. 3 or FIG.

[0243] The data processing device 160 may further include a transmitter and / or a receiver.

[0244] The processor may be a Central Processing Unit (CPU) or other general purpose processor, a Digital Signal Processor (DSP) or an Application-Specific Integrated Circuit (ASIC), etc. The general purpose processor may be a microprocessor or the processor may be any conventional processor, etc. The steps of the methods disclosed with reference to this application may be directly implemented by a hardware processor or may be implemented by a combination of hardware and software modules in the processor.

[0245] Alternatively, the data processing device 160 may be a chip, a driving recorder, a vehicle, a component in a driving recorder, or a component in a vehicle, etc.

[0246] When the data processing device 160 is a driving recorder, the data processing device may further include at least one of an image capture device or a data collection device. Optionally, the data collection device may be in the form of a sensor. For example, the data collection device may include a camera, a millimeter wave radar, a laser radar, or an inertial sensor. The image capture device may be configured to capture a driving record video, the data collection device may be configured to acquire the first data, and the memory may further be configured to store the driving record video. For example, the configuration of the data processing device 160 may be shown as in FIG. 12A or FIG. 12B.

[0247] When the data processing device 160 is a vehicle, the data processing device may further include at least one of an image capture device or a data collection device. The image capture device may be configured to capture a driving record video, the data collection device may be configured to acquire the first data, and the memory may be further configured to store the driving record video. The data collection device may be in the form of a sensor. For example, device may include a camera, a millimeter wave radar, a laser radar, an inertial sensor, etc. The data collection device may further be a vehicle body state system configured to acquire the first driving data. For example, the configuration of the data processing device may be shown as in FIG. 14A or FIG. 14B.

[0248] This application provides a readable storage medium, which stores a computer program, which is used to implement a data processing method according to any one of the above embodiments.

[0249] One embodiment of this application provides a computer program product, which includes instructions that, when executed, enable a computer to perform the above data processing method.

[0250] All or part of the steps of the above method embodiments may be implemented by a program instructing associated hardware. The above program may be stored in a readable memory. When the program is executed, it performs the steps of the above method embodiments. The memory (storage medium) includes a read-only memory (ROM), a RAM, a flash memory, a hard disk drive, a solid-state drive, a magnetic tape, a floppy disk, an optical disc, and any combination thereof.

[0251] The embodiments of the present application will be described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that computer program instructions may be used to implement each process and / or each block in the flowcharts and / or block diagrams, and combinations of processes and / or blocks in the flowcharts and / or block diagrams. The computer program instructions may be provided to a processing unit of a general purpose computer, a special purpose computer, an embedded processor, or other programmable data processing unit to generate a machine, whereby the instructions executed by the processing unit of the computer or other programmable data processing unit generate an apparatus for implementing a particular function in one or more processes in the flowcharts and / or one or more blocks in the block diagrams.

[0252] Alternatively, computer program instructions may be stored in the computer readable memory that can instruct a computer or other programmable data processing device to operate in a certain manner, whereby the instructions stored in the computer readable memory generate an artifact that includes an instruction apparatus that implements a certain function in one or more steps in the flowcharts and / or one or more blocks in the block diagrams.

[0253] Alternatively, computer program instructions may be loaded into a computer or other programmable data processing device, which causes the computer or other programmable device to execute a series of operations and steps to generate a computer-implemented process. Thus, the instructions executed by the computer or other programmable device provide steps for implementing a particular function in one or more procedures in the flowcharts and / or one or more blocks in the block diagrams.

[0254] It is obvious that those skilled in the art can make various modifications and variations to the embodiments of this application without departing from the scope of this application. As long as the modifications and variations of the embodiments of this application fall within the scope of protection defined by the appended claims and their equivalent technologies, this application intends to cover such modifications and variations.

[0255] In this application, the word "comprise" and its variations may refer to an open inclusion relationship, and the word "or" and its variations may refer to "and / or". In this application, words such as "first" and "second" are intended to distinguish between multiple similar objects, but are not necessarily intended to indicate a particular order or sequence. In this application, "multiple" refers to two or more than two. The word "and / or" describes a relationship between multiple related objects, and indicates that three relationships may exist. For example, A and / or B may represent three cases: when only A is present, when both A and B are present, and when only B is present. The symbol " / " generally indicates an "or" relationship between multiple related objects.

Claims

1. A data processing method performed by a data processing device of a vehicle, comprising: acquiring first data and first historical driving characteristics, the first data including first driving data of the vehicle for a first time period, the first historical driving characteristics being determined based on the historical driving data of the vehicle, the first historical driving characteristics being indicative of driving habits of a driver while operating the vehicle; determining a non-normality of the vehicle during the first time period based on the first data and the first historical driving characteristics; annotating a driving video of the vehicle based on the first time period; annotating the vehicle driving record video based on the first time period, generating an abnormality video file including a video of the driving record corresponding to the first time period and information describing the abnormality; generating abnormality information as attribute information of the driving record video to indicate a correspondence between the first time period and a description of the abnormality; or adding the description of the abnormality in the form of a subtitle to footage of the driving video relating to the first time period; method.

2. The first data further includes scenario data, and the scenario data includes: The method of claim 1 , comprising at least one of the following information: time information, location information, road condition information, or weather information.

3. The step of acquiring a first historical driving characteristic includes: The method of claim 2 , comprising determining the first historical driving characteristic from a plurality of historical driving characteristics based on the scenario data.

4. The first operating data is status data for components within the vehicle; Operational data of components within the vehicle; or 4. The method of claim 1, further comprising at least one of: sensor data collected by components within the vehicle.

5. The first historical driving characteristic includes a plurality of historical data curves corresponding to a plurality of types of historical driving data, the historical data curves are used to indicate a distribution rule of the historical driving data, and the historical driving data is: historical condition data for said components within said vehicle; Historical operating data of the components in the vehicle; or and historical sensor data collected by the components in the vehicle.

6. The step of acquiring a first historical driving characteristic includes: processing the historical driving data by using a first model to obtain the first historical driving characteristics; 6. The method of claim 1, wherein the first model is obtained by learning a plurality of groups of first samples, each of the plurality of groups of first samples including sample driving data and sample historical driving characteristics.

7. The step of determining a non-normality of the vehicle during the first time period based on the first data and the first historical driving characteristics includes: determining a similarity between the first data and the first historical driving characteristics; and determining the abnormality in the first time period based on the similarity.

8. The step of determining a similarity between the first data and the first historical driving characteristics includes: processing the first data and the first historical driving characteristics by using a second model to obtain the similarity; 8. The method of claim 7, wherein the second model is obtained by learning a plurality of groups of second samples, each of the plurality of groups of second samples including sample data, sample driving features, and sample similarities, and the sample data includes sample driving data.

9. The step of processing the first data and the first historical driving characteristics by using a second model to obtain the similarity includes: determining a first data curve corresponding to the first data by using the second model; and comparing the historical data curve and the first data curve for the first historical driving characteristic by using the second model to obtain the similarity.

10. A data processing device including an acquisition module, a determination module, and an annotation module, the data processing device being disposed in a vehicle; The acquisition module is configured to acquire first data and first historical driving characteristics, the first data including first driving data of the vehicle during a first time period, the first historical driving characteristics being determined based on the historical driving data of the vehicle, the first historical driving characteristics being indicative of driving habits of a driver while operating the vehicle; the determination module is configured to determine a non-normality of the vehicle during the first time period based on the first data and the first historical driving characteristics; the annotation module is configured to annotate driving record video of the vehicle based on the first time period; The annotation module includes: generating an abnormality video file including a video of the driving record corresponding to the first time period and information for describing the abnormality; Generate abnormality information as attribute information of the driving record video to indicate a correspondence between the first time period and a description of the abnormality; or adding the description of the abnormality in the form of a subtitle to footage of the driving video relating to the first time period; Device.

11. The first data further includes scenario data, and the scenario data includes: The apparatus of claim 10, further comprising at least one of the following information: time information, location information, road condition information, or weather information.

12. The decision module: The apparatus of claim 11 , configured to determine the first historical driving characteristic from a plurality of historical driving characteristics based on the scenario data.

13. The first operating data is status data for components within the vehicle; Operational data of components within the vehicle; or 13. The apparatus of claim 10, further comprising at least one of: a) sensor data collected by a component within the vehicle;

14. The first historical driving characteristic includes a plurality of historical data curves corresponding to a plurality of types of historical driving data, the historical data curves are used to indicate a distribution rule of the historical driving data, and the historical driving data is: historical condition data for said components within said vehicle; Historical operating data of the components in the vehicle; or and historical sensor data collected by the components in the vehicle.

15. The acquisition module includes: configured to process the historical driving data by using a first model to obtain the first historical driving characteristics; 15. The apparatus of claim 10, wherein the first model is obtained by learning a plurality of groups of first samples, each of the plurality of groups of first samples including sample driving data and sample historical driving characteristics.

16. The decision module: determining a similarity between the first data and the first historical driving characteristics; and The apparatus of claim 14 , configured to determine the abnormality in the first time period based on the similarity.

17. The decision module: configured to process the first data and the first historical driving characteristics by using a second model to obtain the similarity; 17. The apparatus of claim 16, wherein the second model is obtained by learning a plurality of groups of second samples, each of the plurality of groups of second samples including sample data, sample driving features, and sample similarities, and the sample data includes sample driving data.

18. The decision module: determining a first data curve corresponding to the first data by using the second model; and 20. The apparatus of claim 17, configured to compare the historical data curve and the first data curve for the first historical driving characteristic by using the second model to obtain the similarity.

19. A data processing apparatus comprising a memory and a processor, the memory storing instructions of a computer program and the processor executing the instructions of the computer program to perform the method of any one of claims 1 to 9.

20. A computer readable storage medium containing computer instructions which, when executed by a processor, perform the method of any one of claims 1 to 9.

Images