A driving event labeling system and method for intelligent connected vehicle road traffic regulations compliance testing

Abstract

The present invention discloses a driving event label marking system and method for a road traffic regulations compliance test for an intelligent networked vehicle. The marking system comprises: an identity authentication module, a visualization module, a data recording trigger module, a label management module, a data acquisition module, a data processing module and a data storage module; the identity authentication module is used to authenticate the user identity type; the visualization module is used to display the interaction interface between the user and the system; the data recording trigger module is used to trigger data recording under preset conditions for data recording; the label management module is used to adjust labels and update a voice library according to user selection; the data acquisition module is used to collect data from a test vehicle based on the data recording trigger module and the label management module; the data processing module is used to process the collected raw data into format data required for the traffic regulations compliance test; and the data storage module is used to store the processed data and perform label marking.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent networked vehicle testing, and in particular relates to a driving event label marking system and method for testing the compliance of intelligent networked vehicles with road traffic regulations. Background Art

[0002] Prior art 1 A data collection method for road test verification of autonomous driving, which installs a data acquisition device on the vehicle to be tested, records event information and its timestamp when the data meets the preset conditions, and collects autonomous driving data and environmental data in real time. There is a lack of human intervention, and there is no standard label system and standard voice library that meet the traffic police's needs for compliance assessment of intelligent connected vehicle road traffic regulations, and it cannot be combined with the actual law enforcement experience of the traffic police.

[0003] Prior art 2 is a driving event recording system and method based on driving data triggering, which collects vehicle driving data through the vehicle-mounted intelligent terminal and collects the vehicle's internal and external environment video through the video acquisition terminal. The usage scenario is accident handling, which is aimed at ordinary vehicles, involving different usage scenarios and objects; it uses artificial intelligence technology for identification and judgment, and its accuracy depends entirely on the accuracy of the algorithm, without human intervention and the subjective perception of the traffic police on the vehicle's driving status, which is prone to omissions or misjudgments; driving data cannot be obtained when the trigger conditions are not met; there is no human-computer interaction effect.

[0004] Prior art 3 Vehicle driving data recording method, device, electronic device and medium. By combining the global navigation satellite system (GNSS) and inertial measurement unit (IMU), the vehicle's driving trajectory, acceleration, linear velocity and angular velocity and other driving data can be recorded in real time. It is impossible to selectively record driving event data, there is no human-computer interaction effect, and there is no standard label system and standard voice library that meet the traffic police's compliance assessment requirements for intelligent connected vehicle road traffic regulations.

[0005] The existing technology 4 driving data acquisition method and device obtains key driving data such as vehicle control instructions by communicating with the vehicle's own control system, but requires additional sensor equipment to realize data collection, and there is no standard label system and standard voice library that meet the traffic police's needs for compliance assessment of intelligent connected vehicle road traffic regulations.

[0006] The existing technology 5 server and media resource marking method detects and identifies the character actions in the video frame by frame by comparing with the action library and generates marking files and marking tags, calculates the similarity between the user's action and the target action, and obtains the user's action score. Due to different fields involved, there is no standard label system and standard voice library that meets the traffic police's compliance assessment requirements for intelligent connected vehicle road traffic regulations.

[0007] From the above content, we can see that there are currently two main conventional solutions, as shown below:

[0008] Solution 1: The dotting device uses an actual physical button and a drone interactive interface. When the operator detects an abnormality, he presses the button to do the dotting. The dotting information only contains the dotting timestamp, and there is no corresponding tag information for supplementary explanation. At the same time, the dotting behavior is only used as a trigger signal to record the test data, and it is not stored locally, making it difficult to trace the source for analysis afterwards.

[0009] Solution 2: The management method uses office communication software and a chat interface as the interactive interface. When the operator detects an abnormality, he or she enters a voice message on the interactive device and sends it. The voice message will form label information together with the time the message was sent and transmitted to the cloud platform. However, the label description information is more colloquial and non-standard, making it difficult to conduct statistical analysis.

[0010] Intelligent connected vehicles must comply with relevant laws and regulations such as the Road Traffic Law when driving on the road. The existing technology proposes to use data acquisition devices to obtain key data during vehicle driving, trigger storage records when the data reaches preset values, and conduct post-analysis. However, the driving scenes on real roads are complex and changeable, and existing technologies alone cannot cope with them. Only preset conditions cannot meet all the evaluation requirements of the intelligent connected vehicle's ability to comply with traffic regulations. All objective evaluations by machines often preset fixed standards, cannot fully understand the meaning of behavior in specific situations, and cannot adapt to special situations in different scenarios. It is easy to prompt autonomous driving system algorithm developers to pay more attention to passing tests rather than improving the safety level of driving on the road. Analogous to human subject two and subject three exams, in addition to machine evaluation, examiners are still required to conduct manual scoring on the vehicle. Similarly, the evaluation of the compliance of intelligent connected vehicles with road traffic regulations should be combined with the evaluation of traffic police personnel on the vehicle and the automatic evaluation of machines. On the one hand, the on-board traffic police can combine their accumulated law enforcement experience to make a comprehensive assessment of the actual operation, emergency response and driving behavior of the vehicle based on the autonomous driving system, which are difficult for automated evaluation machines to fully cover; on the other hand, current sensors and algorithms may not be able to fully and accurately capture all scene details, especially in complex traffic environments, where the system may misjudge or miss judgments due to technical failures, software defects or external interference. Therefore, it is necessary to combine manual judgment by traffic police with machine judgment to ensure the comprehensiveness of the compliance assessment of intelligent connected vehicles with road traffic regulations. Summary of the invention

[0011] In order to solve the above technical problems, the present invention proposes a driving event labeling system and method for road traffic regulations compliance testing of intelligent connected vehicles, which aims to solve the problem of accurately capturing data items and record contents required for road traffic regulations compliance evaluation according to the time frame recorded by the manual marking of the on-board traffic police and the preset trigger conditions when the intelligent connected vehicles are conducting actual vehicle tests for road traffic regulations compliance, and attaching corresponding labels to ensure time synchronization and data synchronization between manual and automated evaluations, avoid spending a lot of time on sorting and analysis after the test, and improve evaluation efficiency.

[0012] The present invention provides a driving event label marking system for intelligent network-connected vehicle road traffic regulations compliance test, including: an identity authentication module, a visualization module, a data recording trigger module, a label management module, a data acquisition module, a data processing module and a data storage module;

[0013] The identity authentication module is used to authenticate the user identity type;

[0014] The visualization module is used to display the interaction interface between the user and the system;

[0015] The data recording trigger module is used to trigger data recording under preset conditions or to record data according to user selection on the visual interface;

[0016] The label management module is used to adjust labels and update the voice library according to user selection;

[0017] The data acquisition module is used to collect data from the test vehicle based on the data recording trigger module and the tag management module;

[0018] The data processing module is used to process the collected raw data into data in a format required for traffic rule compliance testing;

[0019] The data storage module is used to store and label the processed data.

[0020] Optionally, the identity authentication module includes: a smart connected vehicle digital identity unit and a traffic police identity unit;

[0021] The intelligent connected vehicle digital identity unit is used to perform identity authentication using a digital certificate issued by a certification authority CA and stored in the active carrier certificate area;

[0022] The traffic police identity includes: a mobile digital certificate using the national secret algorithm SM2, supplemented by biometric identification authentication.

[0023] Optionally, the visual interface includes: electronic map, real-time vehicle positioning, vehicle speedometer, location tracking and dot labeling;

[0024] The electronic map is used to display the vehicle position and driving path in real time;

[0025] The real-time positioning of the vehicle is used for combined positioning using a global navigation satellite system and an inertial navigation system;

[0026] The vehicle speedometer is used to display the vehicle speed in the form of a pointer combined with text;

[0027] The position tracking is used to track the vehicle on the electronic map;

[0028] The dotting tag is used to perform dotting through a tag selection page.

[0029] Optionally, the data recording trigger module includes: a manual dot trigger submodule and a system preset condition trigger submodule;

[0030] The manual tapping trigger submodule includes: a button tapping trigger unit and a speech recognition unit;

[0031] The system preset condition trigger submodule is used to trigger data recording according to time, speed, acceleration and distance.

[0032] Optionally, the click button trigger unit is used to upload a label file composed of label information and a timestamp of the marking moment to the cloud platform by clicking to select a combined event label;

[0033] The speech recognition unit is used to perform speech recognition based on sound source localization based on arrival time difference and a deep learning model.

[0034] Optionally, the label management module includes: a standard label system and a standard voice library;

[0035] The standard labeling system is used to classify situations involved in the road traffic regulations compliance assessment during vehicle testing;

[0036] The standard speech library is used to construct a standard pronunciation model, and the speech signal after syllable segmentation and fundamental frequency marking is used to form the standard speech library according to the standard pronunciation model.

[0037] Optionally, the data acquisition module includes: collecting perception data, positioning data, video surveillance data and vehicle-mounted data.

[0038] Optionally, processing the collected data includes:

[0039] The collected data is subjected to fusion perception, data annotation and structured processing to obtain the vehicle status data, traffic environment data, traffic participant data, road data, traffic sign markings and signal light data during the vehicle's driving process.

[0040] The present invention also provides a method for marking driving event labels for compliance testing of road traffic regulations for intelligent networked vehicles, comprising:

[0041] The user performs digital identity authentication for the smart connected car and the traffic police. After successful authentication, the user enters the visual interface.

[0042] Determine whether to adjust the label and the trigger condition on the visualization interface, and determine the determination result;

[0043] According to the judgment result, a real vehicle test is carried out. During the test, the system preset and manual marking trigger data recording are carried out according to the actual test vehicle operation situation and the experience of the traffic police. If the system preset marking time and the manual marking time are less than 20 seconds apart, the union data of the two time intervals are collected;

[0044] Data is collected according to the triggering mode, the collected data is processed, and the processed data is stored and output.

[0045] Compared with the prior art, the present invention has the following advantages and technical effects:

[0046] 1. The present invention adopts a human-computer interaction interface, combined with components such as maps and speedometers to display vehicle status information, can provide objective image support, has a friendly interface, and can be adapted to portable devices such as mobile phones and tablets;

[0047] 2. Based on the objective judgment of driving data, the present invention adds manual intervention and the subjective perception and judgment of the traffic police on the driving status of the vehicle, which can be combined with the actual law enforcement experience of the traffic police;

[0048] 3. The present invention establishes a standard label system and a standard voice library that meet the traffic police's needs for compliance assessment of intelligent connected vehicle road traffic regulations;

[0049] 4. The present invention adopts a combination of radio button, multiple button and drop-down box to standardize the label system, which is convenient for operation and facilitates unified description and subsequent statistical analysis;

[0050] 5. The present invention uses clicking the label button as the marking moment, and the label operation time does not affect the marking delay. At the same time, the label file will be locally stored while uploading the platform to facilitate traceability and evidence storage. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0052] Figure 1This is a structural diagram of a driving event labeling system for testing compliance with road traffic regulations for intelligent connected vehicles according to an embodiment of the present invention;

[0053] Figure 2 is a flow chart of the function implementation process of the identity authentication module of an embodiment of the present invention;

[0054] Figure 3 is a schematic diagram of a visualization interface of an embodiment of the present invention;

[0055] Figure 4 is a schematic diagram of positioning two microphones in a far-field model of an embodiment of the present invention;

[0056] Figure 5 is a schematic diagram of positioning of three microphones in a near-field model of an embodiment of the present invention;

[0057] Figure 6 is a schematic diagram of a tag selection page according to an embodiment of the present invention;

[0058] Figure 7 is a schematic diagram of an example of a fault cause selection page according to an embodiment of the present invention;

[0059] Figure 8 is a schematic diagram of a label information page according to an embodiment of the present invention;

[0060] Fig. 9 The present invention is a flowchart of a method for marking driving event labels for compliance testing of road traffic regulations for intelligent connected vehicles according to an embodiment of the present invention. DETAILED DESCRIPTION

[0061] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0062] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0063] The present invention proposes a driving event labeling system for testing the compliance of intelligent connected vehicles with road traffic regulations, such as Figure 1 As shown, specifically including:

[0064] Identity authentication module, visualization module, data recording trigger module, tag management module, data acquisition module, data processing module and data storage module;

[0065] Identity authentication module, used to authenticate the user's identity type;

[0066] Visualization module, used to display the interaction interface between users and the system;

[0067] A data recording trigger module, which is used to trigger data recording according to preset conditions or to record data according to user selection on a visual interface;

[0068] The label management module is used to adjust labels and update the voice library according to user selection;

[0069] A data acquisition module, used to collect data from the test vehicle based on the data recording trigger module and the tag management module;

[0070] A data processing module is used to process the collected raw data into data in the format required for traffic rule compliance testing;

[0071] The data storage module is used to store and label the processed data.

[0072] Specifically, the intelligent connected vehicle involved in this embodiment refers to a vehicle equipped with advanced on-board sensors, which uses an on-board computing platform to process the perceived information, make driving decisions and control through actuators, integrates wireless communication, intelligent interconnection, on-board networks and vehicle-road collaboration and other technologies to achieve intelligent information exchange and sharing between vehicles, vehicles and roads, vehicles and people, and vehicles and the cloud, and has assisted driving or automatic driving functions.

[0073] The label marking system of this embodiment is composed of seven parts: a visualization module, an identity authentication module, a data recording trigger module, a label management module, a data acquisition module, a data processing module and a data storage module.

[0074] This embodiment can realize marking operation and visualization based on mobile smart terminals such as mobile phones and tablets, and can be applied to any scenario that requires precise positioning of smart connected vehicle actual vehicle operation safety event data collection, recording and evaluation, including but not limited to actual road traffic regulations compliance evaluation of smart connected vehicles.

[0075] Furthermore, user identity types include: intelligent connected car digital identity and traffic police identity;

[0076] Authentication of the digital identity of an intelligent connected vehicle includes: using a digital certificate issued by a certification authority CA and stored in the active carrier certificate area for identity authentication;

[0077] Traffic police identity authentication includes: mobile digital certificate using national secret algorithm SM2, supplemented by biometric authentication.

[0078] Specifically, Figure 2 As shown, identity authentication is required to log in and use the tag-marking system, including digital identity authentication for smart connected vehicles and traffic police identity authentication.

[0079] Smart connected vehicle digital identity authentication: When performing smart connected vehicle digital identity authentication, a digital certificate issued by a certification authority CA and stored in the active carrier certificate area is used for identity authentication, which includes verification of coded information such as the vehicle registration number and vehicle identification code that uniquely identify the vehicle, as well as verification of coded information that uniquely identifies the version number of the autonomous driving system. The digital identity of the smart connected vehicle is confirmed by checking whether the digital certificate validity period is less than the vehicle inspection validity period, using the digital certificate to digitally sign the message sent to the tag dotting system, and verifying the legitimacy of the digital signature of the message sent by the tag dotting system.

[0080] Traffic police identity authentication: Traffic police identity authentication is based on a mobile digital certificate using the national secret algorithm SM2, supplemented by biometric authentication of facial images or fingerprints. The certificate storage method uses a hardware password module or a software password module, and the facial image or fingerprint collection has liveness detection capabilities.

[0081] Identity authentication based on mobile digital certificates first verifies the electronic signature, uses the root certificate to verify the signature integrity of the user certificate, and then checks the validity of the signature to see if the user certificate is currently within the validity period defined in the certificate structure, checks whether the user certificate is invalid, and finally verifies the consistency of the digital certificate that generates the signature and the user association, confirms the certificate purpose in the user certificate structure, and accepts the certificate if all verifications are passed.

[0082] Biometric authentication based on facial images or fingerprints includes feature collection, feature processing, feature recognition and verification. The facial image or fingerprint features of traffic police approved for registration must be stored in the biometric database as a template in advance, the user's identification, auxiliary data, facial image or fingerprint feature template must be bound, a biometric data package for storage must be created, and a unique user identifier must be assigned to each data package.

[0083] Furthermore, the visual interface includes: electronic map, real-time vehicle location, vehicle speedometer, location tracking and dot labeling;

[0084] Electronic maps, used to display vehicle locations and driving routes in real time;

[0085] Real-time vehicle positioning, for positioning using a combination of global navigation satellite systems and inertial navigation systems;

[0086] Vehicle speedometer, used to display vehicle speed in the form of a pointer combined with text;

[0087] Position tracking, used to track the vehicle on an electronic map;

[0088] Dot tag, used to dot through the tag selection page.

[0089] Specifically, Figure 3 As shown, after the identity authentication is completed and the system is logged in, the electronic map, real-time vehicle positioning and driving path, vehicle speedometer, location tracking, dot labeling and corresponding selection page can be seen in the visualization module of the mobile terminal.

[0090] The electronic map supports zooming in and out and dragging. The vehicle will be displayed as a small black car on the map, and the vehicle location and driving path will be displayed in real time.

[0091] Vehicle real-time positioning technology is a combined navigation system that uses a global navigation satellite system (GNSS) and an inertial navigation system (INS) for combined positioning. INS uses inertial sensors such as accelerometers and gyroscopes to provide continuous high-frequency data in short-term carrier high-mobility situations, which can make up for the shortcomings of GNSS that are susceptible to interference in dynamic environments. GNSS uses satellite signals to provide discrete and accurate absolute position and time information over a long period of time, which can correct the drift error of INS. The GNSS and INS data are fused through Kalman filtering, and the optimal state estimate is obtained through multiple iterations, so that more accurate vehicle position information and trajectory information can be calculated in real time.

[0092] The vehicle speedometer will display the vehicle speed in the form of a pointer combined with text. At the same time, when the vehicle speed exceeds the speed limit of the current road, the speed text will turn red.

[0093] When you click the location tracking button, the map will automatically track the location of your vehicle and use your vehicle as the center point of the map.

[0094] When you click the Mark Tag button, the Mark Tag selection page will pop up in a pop-up window. When reviewing the Mark Tag records, you can filter by tags and Mark Tag time range.

[0095] Further, the data recording trigger module includes: a manual dot trigger submodule and a system preset condition trigger submodule;

[0096] The manual tapping trigger submodule includes: a button tapping trigger unit and a speech recognition unit;

[0097] The system presets conditional trigger submodule, which is used to trigger data recording according to time, speed, acceleration and distance.

[0098] Specifically, data recording triggers include manual dot triggering and system preset trigger conditions, recording driving data, timestamps and additional tags at the trigger time. Through time synchronization technology, ensure that the data recording of this system is synchronized with the time trigger of vehicle-mounted data recording such as CAN bus.

[0099] The time synchronization method is NTP network timing. The tag marking system is connected to the same vehicle LAN as the data acquisition system via WIFI, and the system time is used as the timestamp information.

[0100] Further, a button-clicking trigger unit is used to upload a label file composed of label information and a timestamp of the marking moment to the cloud platform by clicking and selecting the combined event label;

[0101] Speech recognition unit, used for speech recognition based on sound source localization and deep learning model based on arrival time difference.

[0102] Wherein, the speech recognition unit includes: a sound source localization subunit and a speech processing subunit;

[0103] The sound source collection subunit is used to The sensor array composed of microphones collects sound source signals at the same time, filters and reduces noise on the signals, calculates the delay difference of the sound source signals reaching different microphone array elements, and combines the speed of sound propagation in the air. The geometric relationship between the microphone arrays is used to obtain the coordinate position of the sound source, obtain the speaker's position information, and lock the voice zone of the traffic police on the passenger seat;

[0104] The speech processing subunit is used to perform pre-processing such as filtering, noise reduction, and frame division on the collected speech signals, and to extract acoustic feature information based on the speech to be recognized. It can use common acoustic feature extraction methods such as Mel-frequency cepstral coefficients and linear prediction cepstral coefficients, and use deep learning models such as deep full-sequence convolutional neural networks and feedforward sequence memory networks to learn acoustic features, capture the timing information in the speech signal, and automatically recognize the speech content, so as to match it with the standard speech library.

[0105] Specifically, the manual triggering method includes two methods: the co-pilot traffic police clicking a button and voice recognition.

[0106] Click the button to select a physical button or a visual interface button. After clicking the button, the label selection page will be displayed in the form of a pop-up window. You can select the combined event label by clicking on the screen. It also supports voice input and photo storage. When the mark is confirmed, the label information will be formed into a label file together with the timestamp of the mark time, which will be saved locally and uploaded to the platform.

[0107] Speech recognition is triggered by a wake-up word and uses sound source localization based on arrival time difference and a deep learning model.

[0108] Sound source localization is performed by The sensor array composed of microphones simultaneously collects the sound source signal , filter and reduce noise on the signal, and calculate the sound source signal Reaching different microphone array elements The delay difference, combined with the speed of sound in air The geometric relationship between the microphone arrays is used to obtain the coordinate position of the sound source, obtain the speaker's position information, and lock the voice zone of the traffic police on the passenger seat. The sound source signals received by the two microphones are and :

[0109]

[0110]

[0111] , is the sound propagation attenuation factor, is the sound source signal, , is the received noise signal, , is the time difference between the sound source reaching the two microphones, then and The cross-correlation function for:

[0112]

[0113]

[0114] It is usually considered that the sound source signal and and noise and Uncorrelated and noisy and It is also considered that the randomness is unrelated, and the above formula is simplified to:

[0115]

[0116] The two microphones receive the same sound source signal , the cross-correlation function can be transformed into an autocorrelation function, let , which is simplified to:

[0117]

[0118] in ,when hour is the maximum value, at this time is the time delay difference between the two microphones, that is

[0119]

[0120] Due to the external reverberation interference, the cross-correlation function of the microphone array The waveform will have multiple peaks, which will affect the judgment of delay, and a weighting function needs to be introduced , highlighting the true peak of the cross-correlation function waveform and suppressing the false peak. Perform a short-time Fourier transform:

[0121]

[0122]

[0123] is the cross-power spectrum function of the two signals. Finally, the corresponding value of the weighted cross-correlation function is obtained. , which is the time delay between the two microphones:

[0124]

[0125] like Figure 4 As shown in Figure 2, for the far-field model, the waveform of the sound source reaching the microphone array is considered as a plane wave. The time difference of arrival at the microphone array After that, the sound source position can be solved according to the spatial arrangement relationship between the microphones. is the distance between the two microphones, is the speed of sound, then the sound azimuth is , the sound source can be located.

[0126]

[0127]

[0128] like Figure 5 As shown, for the near-field model, the waveform of the sound source reaching the microphone array is considered to be a spherical wave, and three microphones are required. and is the time difference between the first, second and third microphones:

[0129]

[0130]

[0131]

[0132]

[0133] By calculating the time difference and , the above trigonometric functions can be solved , , and , using the law of sines

[0134]

[0135]

[0136] Available and , the sound source can be located.

[0137] After locking and confirming the voice triggered by the traffic police, the collected voice signal is preprocessed by filtering, noise reduction, framing, etc., and the acoustic feature information is extracted according to the voice to be recognized. Common acoustic feature extraction methods such as Mel cepstral coefficients and linear prediction cepstral coefficients can be used. Deep learning models such as deep full-sequence convolutional neural networks and feedforward sequence memory networks are used to learn acoustic features, capture the time sequence information in the voice signal, and automatically recognize the voice content, so as to match it with the standard voice library.

[0138] The system preset trigger conditions include time trigger, speed trigger, acceleration trigger and distance trigger. The trigger indicators and thresholds can be set in advance, as shown in Table 1.

[0139] Table 1

[0140]

[0141] Furthermore, the tag management module includes: a standard tag system and a standard voice library;

[0142] A standard labelling system for classifying situations involving road compliance assessment during vehicle testing;

[0143] The standard speech library is used to construct a standard pronunciation model. According to the standard pronunciation model, the speech signal after syllable segmentation and fundamental frequency annotation constitutes the standard speech library.

[0144] Specifically, the standard label system: The standard label system divides the situations that may be involved in the compliance assessment of road traffic regulations during the actual vehicle test of intelligent connected vehicles into six categories: traffic violations, traffic accidents, manual intervention takeover, minimum risk strategy, risk control, and system prompt takeover. Each parent label further distinguishes the sub-labels under the corresponding conditions for reclassification. For example, risk control can be subdivided into improper braking, improper steering, improper throttle control, etc. At the same time, you can also choose to involve traffic participants, vehicle driving behavior, road type, etc. Supplementary information, if the operator has other supplementary explanation items, can also be supplemented by voice or text input in the supplementary explanation items.

[0145] Table 2

[0146]

[0147] The label code is in hexadecimal, see the table below for details. In manual intervention takeover, perception limitation takeover includes situations within the design operating range but the sensor may be temporarily blocked (such as mud and water splashing, foreign object blocking) or encounter complex visual interference; logical judgment takeover refers to situations where the system cannot judge in the face of complex traffic behaviors or interactive scenarios (such as courtesy to pedestrians, non-standardized hand gestures); safety policy takeover includes encountering edge scenarios within the design operating range, or situations with collision risks; failure or degradation takeover includes vehicle failure within the design operating range, or situations where the vehicle is in degradation; management requirement takeover refers to specific situations such as temporary traffic control and special area regulations requiring human driving.

[0148] Table 3

[0149]

[0150] In the visual interface, the mandatory items include the above six categories of label selection, illegal faults and fault descriptions, and the optional items include traffic participants involved and road types. Road types include highways, urban expressways, general urban roads, internal roads, public parking lots and others. Traffic participants involved include large and medium-sized buses, small and micro buses, heavy and medium-sized trucks, light trucks, motorcycles, trailers, special operation vehicles, special vehicles, non-motor vehicles, pedestrians, roadblocks, traffic police, others and not involved.

[0151] like Figure 6-7 As shown, the overall page combines elements such as radio button, drop-down box button, cascading list, key switch, dialog multiple-choice box and editable input box. When the confirmation button is clicked, it will first determine whether the data is filled in in compliance. Otherwise, a corresponding message will pop up to prompt modification. After confirming compliance, the current elements will be used to form a label file.

[0152] like Figure 8 As shown, the tag information includes the main events of the current tag and the corresponding description, as well as the operator, operation time and other supplementary description items.

[0153] In order to build a standard pronunciation model, it is necessary to use a recording device to capture the standard pronunciation of a professional speaker, convert the pronunciation into a speech signal, and store it in a memory as a digital signal or electrical signal after syllable segmentation and fundamental frequency annotation to form a standard speech library. The recording material should include the following table.

[0154] Table 4

[0155]

[0156] Furthermore, the data collection module includes: collecting perception data, positioning data, video surveillance data and vehicle-mounted data.

[0157] Specifically, the data collected by the data acquisition module include: perception data obtained by multiple sensors such as laser radar and intelligent perception cameras; positioning data output by the integrated navigation system; video monitoring data output by the vehicle surrounding environment monitoring camera and the in-vehicle monitoring camera; vehicle data output through the vehicle OBD interface, vehicle Ethernet, CAN / CANFD and other means;

[0158] According to the manual point trigger or the system preset trigger condition judgment, the collected data from 15 seconds before the trigger time to 5 seconds after the trigger time is stored. The above signal transmission has a strong fault redundancy capability. If there are abnormal situations such as missing video signals and camera obstruction, it can automatically warn. In the case of temporary network interruption and unable to transmit data normally, the system automatically stores the data in the vehicle-mounted industrial computer and automatically retransmits it when the wireless network is restored.

[0159] Furthermore, processing the collected data includes:

[0160] The collected data is subjected to fusion perception, data annotation and structured processing to obtain the vehicle status data, traffic environment data, traffic participant data, road data, traffic sign markings and signal light data during the vehicle's driving process.

[0161] Specifically, the data processing module: The data processing system is responsible for the fusion perception, data labeling and structured processing of the collected data, and ultimately outputs the vehicle status data, traffic environment data, traffic participant data, road data, traffic signs and markings, and signal light data during the vehicle's driving process.

[0162] Table 5

[0163]

[0164] Data storage module: The data storage module can continuously fragment the structured data locally and upload the data to the visual interface in real time. The data storage module can automatically generate operation logs, including operation date, operation time, operator, operation content, etc. The operation log should not be deleted, modified or overwritten in the visual interface.

[0165] The present invention also provides a method for marking driving event labels for intelligent networked vehicle road traffic regulations compliance testing, the specific steps are as follows: Fig. 9 As shown:

[0166] The user performs digital identity authentication for the smart connected car and the traffic police. After successful authentication, the user enters the visual interface.

[0167] Determine whether to adjust the label and trigger conditions in the visual interface and confirm the judgment result;

[0168] A real vehicle test is carried out based on the judgment results. During the test, the system preset and manual marking trigger data recording is carried out according to the actual test vehicle operation conditions and the traffic police experience. Among them, if the system preset marking time and the manual marking time are less than 20 seconds apart, the union data of the two time intervals are collected; if the interval between the two times is greater than 20 seconds, it is recorded normally, regardless of whether it is system marking or manual marking. If the marking time interval is too close, avoid repeated recording.

[0169] Data is collected according to the triggering method, the collected data is processed, and the processed data is stored and output.

[0170] Specifically, after entering the label dotting system, the on-board traffic police first conducts digital identity authentication for the intelligent connected vehicle and the traffic police identity authentication. If the authentication is passed, the login is successful and the visual interface is entered. If it is necessary to adjust the dotting label, trigger conditions and thresholds, enter the background label management module to modify them. After the modification is completed, the actual vehicle test will begin. During the test, if the traffic police encounter an event that involves the judgment of road traffic regulations and the corresponding data needs to be recorded, manual dotting and label selection will be performed. The system will detect the label format. If the format is correct, it will automatically process and save the data collected from 15 seconds before to 5 seconds after the trigger time. In addition, if the preset threshold of the system is triggered, the data will be automatically dotted and saved.

[0171] The dot labeling system can perform self-checking. If the self-checking is normal, the visual interface will be displayed and enter the running state. If there is any problem in the self-checking, the fault will be reported and the alarm prompt will be given.

[0172] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A driving event labeling system for intelligent connected vehicle road traffic regulations compliance testing, characterized in that: include: Identity authentication module, visualization module, data recording trigger module, tag management module, data acquisition module, data processing module and data storage module; The identity authentication module is used to authenticate the user identity type; The identity authentication module includes: an intelligent connected vehicle digital identity unit and a traffic police identity unit; The intelligent connected vehicle digital identity unit is used to perform identity authentication using a digital certificate issued by a certification authority CA and stored in the active carrier certificate area; The traffic police identity includes: a mobile digital certificate using the national secret algorithm SM2, supplemented by biometric identification authentication; The visualization module is used to display the interaction interface between the user and the system; The data recording trigger module is used to trigger data recording under preset conditions or to record data according to user selection on a visual interface; The data recording trigger module includes: a manual dot trigger submodule and a system preset condition trigger submodule; The manual tapping trigger submodule includes: a button tapping trigger unit and a speech recognition unit; The system preset condition trigger submodule is used to trigger data recording according to time, speed, acceleration and distance; The click button trigger unit is used to upload the tag file composed of the tag information and the timestamp of the marking moment to the cloud platform by clicking and selecting the combined event tag; The speech recognition unit is used for performing speech recognition based on sound source localization and deep learning model based on arrival time difference; the speech recognition unit includes: a sound source localization subunit and a speech processing subunit; The sound source collection subunit is used to The sensor array composed of microphones collects sound source signals at the same time, filters and reduces noise on the signals, calculates the delay difference of the sound source signals reaching different microphone array elements, and combines the speed of sound propagation in the air. The geometric relationship between the microphone arrays is used to obtain the coordinate position of the sound source, obtain the speaker's position information, and lock the voice zone of the traffic police on the passenger seat; The speech processing subunit is used to filter, reduce noise, and perform frame preprocessing on the collected speech signals. It can extract acoustic feature information based on the speech to be recognized. It can use the commonly used acoustic feature extraction methods such as Mel cepstral coefficients and linear prediction cepstral coefficients, and use deep learning models such as deep full-sequence convolutional neural networks and feedforward sequence memory networks to learn acoustic features, capture the time sequence information in the speech signal, and automatically recognize the speech content, so as to match it with the standard speech library; Sound source localization is performed by The sensor array composed of microphones simultaneously collects the sound source signal , filter and reduce noise on the signal, and calculate the sound source signal Reaching different microphone array elements The delay difference, combined with the speed of sound in air The geometric relationship between the microphone arrays is used to obtain the coordinate position of the sound source, obtain the speaker's position information, and lock the voice zone of the traffic police on the passenger seat. The sound source signals received by the two microphones are and : , is the sound propagation attenuation factor, is the sound source signal, , is the received noise signal, , is the time difference between the sound source reaching the two microphones, then and The cross-correlation function for: It is usually considered that the sound source signal and and noise and Uncorrelated and noisy and It is also considered that the randomness is unrelated, and the above formula is simplified to: The two microphones receive the same sound source signal , the cross-correlation function can be transformed into an autocorrelation function, let , which is simplified to: in ,when hour is the maximum value, at this time is the time delay difference between the two microphones, that is Due to the external reverberation interference, the cross-correlation function of the microphone array The waveform will have multiple peaks, which will affect the judgment of delay, and a weighting function needs to be introduced , highlighting the true peak of the cross-correlation function waveform and suppressing the false peak. Perform a short-time Fourier transform: is the cross-power spectrum function of the two signals. Finally, the corresponding value of the weighted cross-correlation function is obtained. , which is the time delay between the two microphones: For the far-field model, the waveform of the sound source reaching the microphone array is regarded as a plane wave, and the sound source is obtained The time difference of arrival at the microphone array After that, the sound source position can be solved according to the spatial arrangement relationship between the microphones. is the distance between the two microphones, is the speed of sound, then the sound azimuth is , the sound source can be located; For the near-field model, the waveform of the sound source reaching the microphone array is considered to be a spherical wave, and three microphones are required. and is the time difference between the first, second and third microphones: By calculating the time difference and , the above trigonometric functions can be solved , , and , using the law of sines: Available and , the sound source can be located; The label management module is used to adjust labels and update the voice library according to user selection; The data acquisition module is used to collect data from the test vehicle based on the data recording trigger module and the tag management module; The data processing module is used to process the collected raw data into data in a format required for traffic rule compliance testing; The data storage module is used to store and label the processed data.

2. A driving event labeling system for testing the compliance of intelligent connected vehicles with road traffic regulations according to claim 1, characterized in that: The visualization module includes: electronic map, real-time vehicle positioning, vehicle speedometer, location tracking and dot labeling; The electronic map is used to display the vehicle position and driving path in real time; The real-time positioning of the vehicle is used for combined positioning using a global navigation satellite system and an inertial navigation system; The vehicle speedometer is used to display the vehicle speed in the form of a pointer combined with text; The position tracking is used to track the vehicle on the electronic map; The dotting tag is used to perform dotting through a tag selection page.

3. A driving event labeling system for intelligent connected vehicle road traffic regulations compliance testing according to claim 1, characterized in that: The label management module includes: a standard label system and a standard voice library; The standard labeling system is used to classify situations involved in the road traffic regulations compliance assessment during vehicle testing; The standard speech library is used to construct a standard pronunciation model. According to the standard pronunciation model, the speech signal after syllable segmentation and fundamental frequency marking constitutes the standard speech library.

4. A driving event labeling system for intelligent connected vehicle road traffic regulations compliance testing according to claim 1, characterized in that: The data acquisition module includes: collecting perception data, positioning data, video monitoring data and vehicle-mounted data.

5. A driving event labeling system for testing the compliance of intelligent connected vehicles with road traffic regulations according to claim 1, characterized in that: Processing of the collected data includes: The collected data is subjected to fusion perception, data annotation and structured processing to obtain the vehicle status data, traffic environment data, traffic participant data, road data, traffic sign markings and signal light data during the vehicle's driving process.

6. A method for marking driving event labels for compliance testing of road traffic regulations for intelligent connected vehicles, characterized in that: include: The user performs digital identity authentication for the smart connected car and the traffic police. After successful authentication, the user enters the visual interface. Determine whether to adjust the label and the trigger condition on the visualization interface, and determine the determination result; According to the judgment result, a real vehicle test is carried out. During the test, the system preset and manual marking trigger data recording are carried out according to the actual test vehicle operation situation and the experience of the traffic police. If the system preset marking time and the manual marking time are less than 20 seconds apart, the union data of the two time intervals are collected; Data is collected according to the triggering method, the collected data is processed, and the processed data is stored and output.

Images