A risk detection method, device, medium and equipment for automatic driving of an automobile

Abstract

This specification discloses a risk detection method, apparatus, medium, and device for autonomous driving in automobiles, relating to the field of autonomous driving technology. It includes: acquiring real-time driving video captured by an in-vehicle camera over a preset period of time, and pre-captured reference driving video. Based on each frame of the real-time and reference driving videos, it determines whether the autonomous driving operation corresponding to the real-time driving video conforms to preset standards. If so, it is determined that the autonomous driving operation corresponding to the real-time driving video is risk-free, and real-time driving video acquisition continues until the autonomous driving detection task is completed. If not, it is determined that a risk exists, and the autonomous driving detection task ends. This approach no longer relies solely on the onboard autonomous driving system itself to determine whether a problem has occurred; it can comprehensively assess the risk of autonomous driving through images captured by the in-vehicle camera and detection of driving smoothness and efficiency, providing dual protection for autonomous driving detection.

Description

Technical Field

[0001] This specification relates to the field of autonomous driving technology, and in particular to a risk detection method, device, medium and equipment for autonomous driving. Background Technology

[0002] Currently, with the widespread use of artificial intelligence, autonomous driving technology in automobiles is also developing rapidly. Utilizing autonomous driving technology, cars can automatically perform driving tasks, providing passengers with a superior driving experience. As autonomous vehicles become increasingly equipped with a wider range of sensors and more diverse functions, and as their autonomous driving capabilities mature, people are paying increasing attention to potential problems that may arise during this increasingly relied-upon autonomous driving process.

[0003] In current autonomous driving processes, when a problem arises with the vehicle, the onboard autonomous driving system needs to determine whether to address it based on the severity of the problem. To better assess the severity, a qualitative safety analysis can be performed to classify the impact of different problems and then categorize them according to their severity of influence on autonomous driving. In this approach, the onboard autonomous driving system needs to not only execute the autonomous driving process but also identify and address problems that arise during autonomous driving.

[0004] However, in this approach, when problems arise, the onboard autonomous driving system may not be able to respond quickly enough. In such cases, the autonomous driving system often fails to recognize that its decision has deviated from the normal decision, or may even make an incorrect decision.

[0005] Therefore, this specification provides a method, apparatus, medium, and device for risk detection in autonomous driving of automobiles. Summary of the Invention

[0006] This specification provides a method, apparatus, medium, and device for risk detection in autonomous driving of automobiles, in order to partially solve the aforementioned problems existing in the prior art.

[0007] The following technical solution is adopted in this specification:

[0008] This manual provides a risk detection method for autonomous driving of automobiles, including:

[0009] The system acquires real-time driving video captured by an in-vehicle camera for a preset time, as well as a pre-captured reference driving video. The preset time is the same as the capture duration of the reference driving video. The in-vehicle camera is installed on the top of the vehicle and is used to capture videos including the driver's seat scene and the windshield during autonomous driving.

[0010] Based on each frame of the real-time driving video, a real-time driving image sequence is determined, and based on each frame of the reference driving video, a reference driving image sequence is determined.

[0011] Based on the real-time driving image sequence and the reference driving image sequence, it is determined whether the autonomous driving operation of the car corresponding to the real-time driving video meets the preset standard;

[0012] If so, it is determined that there is no risk in the autonomous driving operation of the car corresponding to the real-time driving video, and the real-time driving video captured by the in-vehicle camera continues to be acquired until the autonomous driving detection task is completed.

[0013] If not, it is determined that there is a risk in the autonomous driving operation of the car corresponding to the real-time driving video, and the autonomous driving detection task is terminated.

[0014] Optionally, based on the real-time driving image sequence and the reference driving image sequence, it is determined whether the vehicle's autonomous driving operation corresponding to the real-time driving video meets a preset standard, specifically including:

[0015] The number of image comparison wheels is determined based on the number of images in the real-time driving image sequence;

[0016] According to the number of image comparison rounds, the shooting order of each frame in the real-time driving image sequence and the reference driving image sequence, the image comparison operation corresponding to the number of image comparison rounds is performed to determine the similarity between the images in the real-time driving image sequence and the reference driving image sequence. Specifically, for the Nth round of image comparison operation, the Nth frame image in the real-time driving image sequence and the Nth frame image in the reference driving image sequence are determined according to the shooting order, and the similarity between the Nth frame image in the real-time driving image sequence and the Nth frame image in the reference driving image sequence is calculated, where N is a positive integer not less than 1.

[0017] Based on the similarity scores, the similarity between the real-time driving image sequence and the reference driving image sequence is determined.

[0018] Determine whether the similarity between the real-time driving image sequence and the reference driving image sequence meets a preset similarity value.

[0019] By determining the number of image comparison rounds and performing corresponding image comparison operations based on the shooting order, it is possible to ensure that corresponding frames in two image sequences are accurately matched. This helps to identify the differences between two sets of images at the same time point or under the same driving conditions. Calculating the similarity of each pair of corresponding images allows for the quantification of the overall similarity between the two image sequences, which helps to determine whether the real-time driving situation is consistent with the preset reference scenario.

[0020] Optionally, each frame of the real-time driving video includes at least the dashboard and steering wheel;

[0021] For the Nth frame image in the real-time driving image sequence and the Nth frame image in the reference driving image sequence, calculate the similarity between the Nth frame image in the real-time driving image sequence and the Nth frame image in the reference driving image sequence, specifically including:

[0022] Identify the target object in the Nth frame of the real-time driving image sequence and the Nth frame of the reference driving image sequence, wherein the target object is either the dashboard or the steering wheel;

[0023] The Nth frame image in the real-time driving image sequence is segmented to determine the first target image, and the Nth frame image in the reference driving image sequence is segmented to determine the second target image.

[0024] Calculate the similarity between the first target image and the second target image, and use it as the similarity between the Nth frame image in the real-time driving image sequence and the Nth frame image in the reference driving image sequence.

[0025] Based on the aforementioned technical methods, by segmenting the Nth frame image in a real-time driving image sequence and a reference driving image sequence, the dashboard image (when the target object is the dashboard) can be accurately extracted. Since the dashboard is a crucial indicator of vehicle operating status (such as speed, turn signals, gear position, etc.), focusing on this area for comparison helps to more accurately assess the consistency between the two driving scenarios. Furthermore, similarity calculations can be performed only on the segmented first steering wheel image (when the target object is the steering wheel) and the second steering wheel image, rather than the entire image. This significantly reduces the impact of background changes (such as road environment, weather conditions, etc.) on the comparison results, thereby improving the accuracy of similarity calculations.

[0026] Optionally, before continuing to acquire the real-time driving video captured by the in-vehicle camera, the method further includes:

[0027] Acquire data from various vibration sensors pre-installed on the seat cushions, seat backs, and floor inside the vehicle;

[0028] Based on the data collected by the vibration sensors, the root mean square value of the total weighted acceleration of the seat cushion, the seat back, and the floor is determined.

[0029] Determine the navigation route recorded by the navigation terminal device pre-set in the vehicle, and determine the driving route corresponding to the reference driving video;

[0030] The distance difference is determined based on the navigation distance and the driving distance;

[0031] The score corresponding to the real-time driving video is determined according to the preset scoring rule text, the root mean square value of the total weighted acceleration, and the distance difference. The scoring rule text records the scoring rules for scoring the real-time driving video.

[0032] If the score corresponding to the real-time driving video does not reach the preset score, and the autonomous driving system of the car corresponding to the real-time driving video is already in the minimum risk strategy state, a prompt message is issued to prompt the user in the car to drive manually.

[0033] Based on the aforementioned technical means, vibration sensors pre-installed in the vehicle's seat cushions, backrests, and floor can comprehensively collect vibration data generated during vehicle operation. This data is crucial for assessing driving comfort and detecting potential mechanical problems or unstable driving behaviors. Calculating the root mean square (RMS) of total weighted acceleration based on the data collected by each vibration sensor provides an objective metric for quantifying the smoothness of the vehicle's ride. A lower RMS generally indicates a smoother driving experience. Combining the navigation route recorded by the terminal device with the corresponding driving route from a reference driving video to determine the distance difference helps identify whether the actual driving route meets expectations. Determining the score for the real-time driving video based on preset scoring rules, the RMS of total weighted acceleration, and the distance difference provides a comprehensive method for evaluating driving quality. This scoring considers not only the accuracy of the driving route but also factors such as driving smoothness. When the score for the real-time driving video fails to meet the preset standard and the autonomous driving system is in a minimum risk strategy state, a prompt message is issued, encouraging the user to take over manual driving. This enhances the system's safety, ensuring that control is promptly transferred to the driver when autonomous driving may be insufficient to handle current road conditions, thereby avoiding potential dangers.

[0034] Optionally, the score corresponding to the real-time driving video is determined according to the preset scoring rule text, the root mean square value of the total weighted acceleration, and the distance difference, specifically including:

[0035] Determine whether there is a risk in the vehicle's autonomous driving operation corresponding to the real-time driving video;

[0036] If so, the vehicle's autonomous driving operation corresponding to the real-time driving video is determined to obtain a safety compliance score that accounts for 50% of the preset total score; the range corresponding to the root mean square value of the total weighted acceleration is determined according to the preset range of each range, and a smoothness score is determined according to the range corresponding to the root mean square value of the total weighted acceleration; an efficiency score is determined according to the preset range of each difference and the distance difference; and a rating corresponding to the real-time driving video is determined according to the safety compliance score, the smoothness score, and the efficiency score.

[0037] If not, no safety compliance score will be obtained, and the score corresponding to the real-time driving video will be determined to be 0.

[0038] Optionally, before concluding the vehicle autonomous driving detection task, the method further includes:

[0039] Determine whether the autonomous driving system of the car corresponding to the real-time driving video is in a minimum risk strategy state;

[0040] If so, then determine that the car corresponding to the real-time driving video triggers the minimum risk strategy within the target time.

[0041] If not, an alarm message will be issued to indicate that the minimum risk strategy has not been triggered within the target time.

[0042] Based on the aforementioned technical means, by monitoring in real time whether the autonomous driving system triggers the minimum risk strategy within the target time and issuing an alarm when necessary, the safety during the driving process is effectively enhanced. This ensures that when the autonomous driving system cannot respond to emergencies in a timely manner, human intervention can be carried out quickly, thereby protecting the safety of passengers in the vehicle and other road users.

[0043] Optionally, the method further includes:

[0044] If there is a risk in the autonomous driving operation of the car corresponding to the real-time driving video, and if the car corresponding to the real-time driving video triggers the minimum risk strategy within the target time, a prompt message is issued to prompt the user in the car to drive manually.

[0045] The system receives an instruction to continue executing the vehicle autonomous driving detection task, and continues to acquire real-time driving videos captured by the in-vehicle camera according to the instruction until the vehicle autonomous driving detection task is completed.

[0046] Based on the aforementioned technical means, through effective risk response mechanisms, user interaction design, and continuous driving monitoring, it is ensured that potential risks can be responded to quickly during autonomous driving while fully respecting the user's decision-making rights, thereby improving the safety and satisfaction of the overall driving experience.

[0047] This manual provides a risk detection device for autonomous driving of automobiles, including:

[0048] The acquisition module is used to acquire real-time driving video captured by the in-vehicle camera at a preset time and a pre-captured reference driving video. The preset time is the same as the capture time of the reference driving video. The in-vehicle camera is installed on the top of the vehicle and is used to capture videos including the driver's seat scene and the windshield during autonomous driving.

[0049] The determining module is used to determine a real-time driving image sequence based on each frame of the real-time driving video, and to determine a reference driving image sequence based on each frame of the reference driving video;

[0050] The judgment module is used to determine whether the autonomous driving operation of the car corresponding to the real-time driving video meets the preset standard based on the real-time driving image sequence and the reference driving image sequence; if yes, it is determined that there is no risk in the autonomous driving operation of the car corresponding to the real-time driving video, and the real-time driving video captured by the in-vehicle camera continues to be acquired until the autonomous driving detection task is completed; if no, it is determined that there is a risk in the autonomous driving operation of the car corresponding to the real-time driving video, and the autonomous driving detection task is terminated.

[0051] This specification provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned risk detection method for autonomous driving of automobiles.

[0052] This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement a risk detection method for autonomous driving of a vehicle.

[0053] The above-mentioned technical solutions adopted in this specification can achieve the following beneficial effects:

[0054] The risk detection method for autonomous driving provided in this manual first acquires real-time driving video captured by an in-vehicle camera for a preset time, as well as a pre-captured reference driving video. The preset time is the same as the capture time of the reference driving video. Based on each frame of the real-time driving video, a real-time driving image sequence is determined, and based on each frame of the reference driving video, a reference driving image sequence is determined. Based on the real-time driving image sequence and the reference driving image sequence, it is determined whether the autonomous driving operation corresponding to the real-time driving video conforms to a preset standard. If yes, it is determined that the autonomous driving operation corresponding to the real-time driving video does not pose a risk, and the acquisition of real-time driving video captured by the in-vehicle camera continues until the autonomous driving detection task is completed. If not, it is determined that the autonomous driving operation corresponding to the real-time driving video poses a risk, and the autonomous driving detection task ends.

[0055] By using an in-vehicle camera independent of the autonomous driving system, real-time driving video is captured during the autonomous driving process. Analysis of this video footage determines if any problems have occurred during autonomous driving. This means that when onboard sensors connected to the autonomous driving system are affected by extreme weather (heavy rain / dense fog) or hardware malfunctions, the system can no longer rely solely on the onboard autonomous driving system to determine if a problem has occurred; instead, it can still compare images captured by the in-vehicle camera, providing dual protection for driving safety and autonomous driving detection. Attached Figure Description

[0056] The accompanying drawings, which are included to provide a further understanding of this specification and form part of this specification, illustrate exemplary embodiments and are used to explain this specification, but do not constitute an undue limitation thereof. In the drawings:

[0057] Figure 1 A flowchart illustrating a risk detection method for autonomous driving of a vehicle provided in an embodiment of this specification;

[0058] Figure 2 This is a schematic diagram of a vehicle autonomous driving detection process provided in this specification;

[0059] Figure 3 This is a schematic diagram of a risk detection device for autonomous driving of a vehicle, as provided in this specification.

[0060] Figure 4 This specification provides a corresponding Figure 1 A schematic diagram of the structure of an electronic device. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this specification will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments in this specification without creative effort are within the scope of protection of this application.

[0062] The technical solutions provided in the various embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0063] Figure 1 A flowchart illustrating a risk detection method for autonomous driving provided in this specification includes the following steps:

[0064] S100: Acquire real-time driving video and pre-shot reference driving video captured by an in-vehicle camera for a preset time. The preset time is the same as the shooting duration of the reference driving video. The in-vehicle camera is installed on the top of the vehicle and is used to capture video including the driver's seat and the windshield during autonomous driving.

[0065] The risk detection process for autonomous driving in this specification typically involves image data processing. In the embodiments described herein, the risk detection process can be performed by an onboard electronic control unit (ECU) or an edge computing device. However, this specification does not limit the type of device or platform used to perform the risk detection process; for example, personal computers and mobile terminals can also be used. For ease of description, the following description uses a terminal device as the execution subject, such as a terminal device installed in the vehicle that can be used for computation and image processing.

[0066] In one or more embodiments of this specification, the object of risk detection for autonomous driving is a car equipped with an autonomous driving system and possessing autonomous driving capabilities. An in-vehicle camera capable of capturing images of the driver's seat and the windshield is installed on the roof of the car. However, this specification does not limit the specific installation location of the in-vehicle camera on the roof; for example, it can be installed above the driver's seat or in the center of the roof. The in-vehicle camera can capture images of the dashboard, steering wheel, etc., in front of the driver's seat, as well as the windshield. Since the windshield does not obstruct the view, the in-vehicle camera can also capture the road scene as seen through the windshield.

[0067] Therefore, before testing the autonomous driving capabilities of a car equipped with an autonomous driving system, a standard reference driving video can be pre-recorded using the in-vehicle camera. This reference driving video is recorded in accordance with road traffic safety regulations and the detailed rules for the third subject of the driver's license test.

[0068] When performing autonomous driving testing tasks, the terminal device can acquire real-time driving video captured by the in-vehicle camera over a preset period of time during the testing process. In this specification, to more efficiently test the vehicle's autonomous driving capabilities, the autonomous driving operations within each time frame captured by the in-vehicle camera are analyzed. Of course, there is no limitation on the time frame for each analysis by the terminal device; it's even possible to detect autonomous driving capabilities by analyzing each frame captured by the in-vehicle camera.

[0069] In one or more embodiments of this specification, since a standard reference driving video has been pre-captured by the in-vehicle camera before detecting the autonomous driving capability, the reference driving video can be used as a reference to compare with the real-time driving video captured by the in-vehicle camera at a preset time during the detection process, so as to facilitate the analysis of the vehicle's autonomous driving capability.

[0070] Furthermore, in order to facilitate frame-by-frame comparison between the reference driving video and the real-time driving video, the terminal device also needs to divide the pre-shot reference driving video into segments for the real-time driving video captured by the in-vehicle camera at a preset time. The duration (i.e., shooting duration) of each segment of the reference driving video is the same as the shooting time (i.e., preset time) of the real-time driving video.

[0071] S102: Determine a real-time driving image sequence based on each frame of the real-time driving video, and determine a reference driving image sequence based on each frame of the reference driving video.

[0072] In one or more embodiments of this specification, the terminal device can assemble a real-time driving image sequence based on each frame of the real-time driving video. Similarly, the terminal device can also assemble a reference driving image sequence based on each frame of the reference driving video.

[0073] S104: Based on the real-time driving image sequence and the reference driving image sequence, determine whether the autonomous driving operation of the car corresponding to the real-time driving video meets the preset standard. If yes, proceed to step S106; otherwise, proceed to step S108.

[0074] In one or more embodiments of this specification, the terminal device can determine whether the autonomous driving operation of the car corresponding to the real-time driving video meets the preset standard based on the real-time driving image sequence and the reference driving image sequence.

[0075] Specifically, the terminal device can determine the number of image comparison rounds based on the number of images in the real-time driving image sequence. Alternatively, it can determine the number of image comparison rounds based on the number of images in the reference driving video.

[0076] Subsequently, the terminal device can perform image comparison operations corresponding to the number of image comparison rounds, the shooting order of each frame in the real-time driving image sequence, and the shooting order of each frame in the reference driving image sequence to determine the similarity between images in the real-time driving image sequence and the reference driving image sequence. Specifically, for the Nth round of image comparison operations, the Nth frame in the real-time driving image sequence is determined according to the shooting order of each frame in the real-time driving image sequence, and the Nth frame in the reference driving image sequence is determined according to the shooting order of each frame in the reference driving image sequence. The similarity between the Nth frame in the real-time driving image sequence and the Nth frame in the reference driving image sequence is then calculated, where N is a positive integer not less than 1. Of course, the method for calculating image similarity is not limited in this specification and can be set according to the actual situation, such as comparing based on image pixel values ​​and using Mean Square Error (MSE) to calculate similarity.

[0077] It is worth noting that after the terminal device calculates the pairwise similarity between the real-time driving image sequence and the reference driving image sequence, it can determine the similarity between the real-time driving image sequence and the reference driving image sequence based on the calculated similarities. This specification does not limit the method used to determine the similarity between the real-time driving image sequence and the reference driving image sequence; for example, the sum or average of the similarities can be used as the similarity between the real-time driving image sequence and the reference driving image sequence. Finally, the terminal device can determine whether the similarity between the real-time driving image sequence and the reference driving image sequence meets the preset similarity value.

[0078] Of course, in this specification, the method by which the terminal device determines whether the autonomous driving operation of the car corresponding to the real-time driving video meets the preset standard based on the real-time driving image sequence and the reference driving image sequence can also be to extract the image features of each frame of the real-time driving image sequence and the reference driving image sequence, and match the features of each frame of the real-time driving image sequence and the reference driving image sequence. If the number of matched images reaches the preset number, it can be determined that the autonomous driving operation of the car corresponding to the real-time driving video meets the preset standard.

[0079] S106: Determine that there is no risk in the autonomous driving operation of the car corresponding to the real-time driving video, and continue to acquire the real-time driving video captured by the in-vehicle camera until the autonomous driving detection task is completed.

[0080] In one or more embodiments of this specification, when the autonomous driving operation of the car corresponding to the real-time driving video meets the preset standard, the terminal device can determine that there is no risk in the autonomous driving operation of the car corresponding to the real-time driving video. Then the terminal device can continue to acquire the real-time driving video captured by the in-vehicle camera until the autonomous driving detection task is completed.

[0081] S108: Determine that there is a risk in the autonomous driving operation of the car corresponding to the real-time driving video, and terminate the autonomous driving detection task.

[0082] In one or more embodiments of this specification, when the autonomous driving operation of the car corresponding to the real-time driving video does not meet the preset standard, the terminal device can determine that there is a risk in the autonomous driving operation of the car corresponding to the real-time driving video. Based on the principle that safety is the highest priority, the terminal device can end the autonomous driving detection task at this time. Of course, after the autonomous driving detection task is ended, the terminal device can issue an alarm through the connected microphone device, etc., to indicate that the autonomous driving detection task has ended.

[0083] based on Figure 1 The risk detection method for autonomous driving shown here uses an in-vehicle camera, independent of the autonomous driving system, to capture real-time driving video during the autonomous driving process. By analyzing this real-time video, it determines whether any problems have occurred during autonomous driving. This means that when onboard sensors connected to the autonomous driving system are affected by extreme weather (heavy rain / dense fog) or hardware malfunctions, the method no longer relies solely on the onboard autonomous driving system to determine if a problem has occurred; it can still compare images captured by the in-vehicle camera, providing dual protection for driving safety and autonomous driving detection.

[0084] Furthermore, in practical applications, people inevitably rely on the vehicle's intelligent driving system when it is in autonomous driving mode, which can lead to inattention (such as playing on their phones or having deep conversations) and an inability to fully concentrate on the autonomous driving situation at all times. Therefore, the risk detection method for autonomous driving provided in this manual can assist the driver in paying attention to the autonomous driving situation anytime and anywhere through the in-vehicle camera, thereby increasing safety.

[0085] Furthermore, in one or more embodiments of this specification, each frame of the real-time driving video includes at least the dashboard, i.e., the scene captured by the in-vehicle camera mounted on the top of the vehicle during autonomous driving, which includes the dashboard and steering wheel in the driver's seat.

[0086] Therefore, when the terminal device calculates the similarity between the Nth frame of the real-time driving image sequence and the Nth frame of the reference driving image sequence, it can determine the target object in the Nth frame of the real-time driving image sequence and the Nth frame of the reference driving image sequence. The target object is either the dashboard or the steering wheel.

[0087] When the target object is the dashboard, the Nth frame of the real-time driving image sequence is segmented to determine the first dashboard image, and the Nth frame of the reference driving image sequence is segmented to determine the second dashboard image. The similarity between the first and second dashboard images is then calculated and used as the similarity between the Nth frame of the real-time driving image sequence and the Nth frame of the reference driving image sequence. The terminal device can employ segmentation algorithms such as region growing to perform image segmentation.

[0088] In one or more embodiments of this specification, when the target object is a steering wheel, the terminal device may further perform image segmentation on the Nth frame image in the real-time driving image sequence to determine a first steering wheel image, and perform image segmentation on the Nth frame image in the reference driving image sequence to determine a second steering wheel image. The similarity between the first steering wheel image and the second steering wheel image is then calculated as the similarity between the Nth frame image in the real-time driving image sequence and the Nth frame image in the reference driving image sequence.

[0089] In one or more embodiments of this specification, the terminal device may also acquire data collected by various vibration sensors pre-installed on the seat cushion, seat back, and floor of the vehicle. Based on the data collected by each vibration sensor, the root mean square value of the total weighted acceleration of the seat cushion, seat back, and floor is determined. The method for calculating the root mean square value of the total weighted acceleration can be referenced in the standard document "Automotive Ride Comfort Test Method" (GB / T 4970-2009).

[0090] Simultaneously, the terminal device can also determine the navigation route recorded by a pre-installed navigation terminal device in the vehicle, as well as the driving route corresponding to the reference driving video. Then, based on the navigation route and the driving route, the distance difference is determined. Afterwards, according to the preset scoring rule text, the root mean square value of the total weighted acceleration, and the distance difference, the score corresponding to the real-time driving video is determined. The scoring rule text records the scoring rules for evaluating the real-time driving video.

[0091] Furthermore, if the score corresponding to the real-time driving video does not reach the preset score, or if the autonomous driving system of the car corresponding to the real-time driving video is already in the Minimal Risk Maneuver (MRM) state, a prompt message will be issued to remind the user in the car to drive manually.

[0092] The process of determining the score for a real-time driving video based on preset scoring rules, the root mean square value of total weighted acceleration, and the distance difference can be as follows: The terminal device determines whether the autonomous driving operation corresponding to the real-time driving video poses a risk. If so, the autonomous driving operation corresponding to the real-time driving video receives a safety compliance score, accounting for 50% of the preset total score. Based on preset interval ranges, the interval range corresponding to the root mean square value of total weighted acceleration is determined, and a smoothness score is determined based on this interval range. An efficiency score is determined based on preset difference ranges and the distance difference. The score corresponding to the real-time driving video is determined based on the safety compliance score, smoothness score, and efficiency score. If not, no safety compliance score is obtained, and the score corresponding to the real-time driving video is determined to be 0. In this case, safety compliance is considered unqualified, and smoothness and efficiency are no longer considered; safety compliance is the first priority.

[0093] The following is an example of a preset scoring rule text:

[0094] I. Safety compliance score accounts for 50% of the total score. The existence of risks in the autonomous driving operation corresponding to the real-time driving video will be used as the standard for safety compliance scoring. If there are no risks in the autonomous driving operation, the vehicle will be considered to have obtained safety compliance score; otherwise, it will be considered to have not obtained safety compliance score.

[0095] 2. Ride comfort score accounts for 30% of the total score. The root mean square value of total weighted acceleration is used as the standard for ride comfort scoring. If the calculated root mean square value of total weighted acceleration is less than 0.3, full marks are given; 0.3-0.4 (excluding 0.4) accounts for 60% of the ride comfort score; (3) 0.4-0.5 (excluding 0.5) accounts for 30% of the ride comfort score; (4) greater than or equal to 0.5, no marks are given. Among them, the root mean square value of total weighted acceleration less than 0.3, 0.3-0.4, 0.4-0.5, and greater than or equal to 0.5 are the preset interval ranges. The premise of the ride comfort test in this scheme is that the vehicle hardware has met the ride comfort requirements, that is, the factors affecting ride comfort of non-autonomous driving systems such as the active control suspension system have been eliminated.

[0096] III. Efficiency accounts for 20% of the total score. If the preset time for recording real-time driving video is 15 minutes, and the navigation distance recorded by the in-vehicle navigation device is one-third less than the driving distance corresponding to the reference driving video, no efficiency score will be given. If the navigation distance recorded by the in-vehicle navigation device is one-quarter less than the driving distance corresponding to the reference driving video, 30% of the efficiency score will be given. If the navigation distance recorded by the in-vehicle navigation device is one-fifth less than the driving distance corresponding to the reference driving video, 60% of the efficiency score will be given. If the navigation distance recorded by the in-vehicle navigation device is the same as the driving distance corresponding to the reference driving video, full marks will be awarded. Among these, the difference of one-third, one-quarter, one-fifth, and the difference when the distance is the same is 0, which is the preset range of each difference.

[0097] IV. The total score can be based on a 100-point scale. If the sum of the safety compliance score, efficiency score, and smoothness score is less than 80 points, it is considered that the minimum risk strategy of the on-board autonomous driving system should be triggered.

[0098] By comprehensively covering evaluation dimensions with scores in safety compliance, efficiency, and smoothness, the system provides a holistic perspective for assessing the performance of autonomous driving systems. Safety compliance ensures that vehicle operation complies with traffic rules and legal requirements; efficiency focuses on the system's ability to complete tasks or reach destinations quickly while optimizing resource utilization; and smoothness examines passenger comfort, avoiding situations such as sudden braking or acceleration. Furthermore, the safety compliance score directly measures the autonomous driving system's ability to adhere to traffic rules and safe driving guidelines, thereby reducing the likelihood of accidents and improving the safety of other users on public roads. It also optimizes the user experience; a smoothness score helps ensure a comfortable riding experience and reduces passenger discomfort caused by uneven driving. This is crucial for increasing user trust and acceptance of autonomous driving technology.

[0099] It is worth noting that the core steps for calculating the root mean square value of total weighted acceleration, referring to the standard document "Test Method for Ride Comfort of Vehicles" (GB / T 4970-2009), are exemplarily listed here. The data collected by vibration sensors installed on the seat cushions, seat backs, and floor of the vehicle can be summarized into data in three directions: vibration sensors are installed along the X-axis (seat back), Y-axis (seat cushion), and Z-axis (floor), and data is collected in these three directions. Taking the calculation of the root mean square value of weighted acceleration in one direction as an example, formula A.3 in A.2.1 of the standard document "Test Method for Ride Comfort of Vehicles" (GB / T 4970-2009) is used:

[0100]

[0101] in, The root mean square value of acceleration in the 1 / 3 octave band. —The center frequency is f j The root mean square value of the j-th (j=1,2,3……23) 1 / 3 octave band acceleration, in meters per second squared (m / s²). 2 ). , —The center frequency of each of the 1 / 3 octave bands is f j The upper and lower frequency limits (details are in Table A.2 of the standard document "Test Method for Ride Comfort of Automobiles (GB / T 4970-2009)") are in Hertz (Hz). —Automatic rate spectral density function of acceleration, in units of square meters per cubic second (m²) 2 / s 3 ).

[0102] Then, calculate the weighted root mean square value of acceleration according to the following formula (Formula A.4 in A.2.1 of the standard document "Test Method for Ride Comfort of Automobiles (GB / T 4970-2009)"):

[0103]

[0104] in, —Root mean square value of unidirectional weighted acceleration, in meters per second squared (m / s²) 2 ). —The weighting coefficient of the j-th 1 / 3 octave band is determined according to the direction and position of the measurement point to which the unidirectional band belongs. For specific values, please refer to Table A.4 in the standard document "Test Method for Ride Comfort of Automobiles" (GB / T 4970-2009).

[0105] Therefore, the method described above for calculating the root mean square value of unidirectional weighted acceleration can be used to calculate the root mean square value of unidirectional weighted acceleration in three directions. Then, the root mean square value of total weighted acceleration can be calculated according to the preset weighting coefficients and the formula recorded in A.6 of the standard document "Test Method for Ride Comfort of Automobiles" (GB / T 4970-2009).

[0106] In one or more embodiments of this specification, when the terminal device determines that there is a risk in the autonomous driving operation of the car corresponding to the real-time driving video, the terminal device can determine whether the autonomous driving system of the car corresponding to the real-time driving video is already in a minimum risk strategy state. If so, it determines that the car corresponding to the real-time driving video triggers the minimum risk strategy within a target time. The target time can be the difference between the time when the minimum risk strategy is triggered and the time when the terminal device determines that the autonomous driving operation is risky. The target time can be set to 1 minute to determine whether the triggering of the minimum risk strategy is timely. If not, an alarm message is issued to indicate that the minimum risk strategy was not triggered within the target time.

[0107] In one or more embodiments of this specification, if there is a risk in the autonomous driving operation of the vehicle corresponding to the real-time driving video, or if the vehicle corresponding to the real-time driving video triggers the minimum risk strategy within a target time, the terminal device issues a prompt message to remind the user inside the vehicle to perform manual driving. After a period of manual driving, the terminal device receives an instruction to continue performing the vehicle autonomous driving detection task, and continues to acquire real-time driving videos captured by the in-vehicle camera according to the instruction until the vehicle autonomous driving detection task is completed.

[0108] Figure 2 This is a schematic diagram illustrating a process for detecting autonomous driving in a vehicle, as provided in this specification. Figure 2 As shown, when autonomous driving detection begins, the terminal device acquires real-time driving video captured by the in-vehicle camera and compares it with pre-recorded reference driving video to determine whether the vehicle's autonomous driving capability meets the preset standard. If so, the autonomous driving detection continues. Otherwise, the terminal device also needs to determine whether the autonomous driving system triggers MRM within the target time. If so, the vehicle is switched to manual driving, and the autonomous driving detection continues after the terminal device receives an instruction to continue. If not, the autonomous driving detection ends directly.

[0109] The above describes a risk detection method for autonomous driving of automobiles, provided by one or more embodiments of this specification. Based on the same idea, this specification also provides a corresponding risk detection device for autonomous driving of automobiles, such as... Figure 3 As shown.

[0110] Figure 3 This specification provides a schematic diagram of a risk detection device for autonomous driving in automobiles, specifically including:

[0111] The acquisition module 300 is used to acquire real-time driving video captured by an in-vehicle camera for a preset time and a pre-captured reference driving video. The preset time is the same as the shooting duration of the reference driving video. The in-vehicle camera is installed on the top of the vehicle and is used to capture videos including the driver's seat scene and the windshield during autonomous driving.

[0112] The determining module 302 is used to determine a real-time driving image sequence based on each frame of the real-time driving video, and to determine a reference driving image sequence based on each frame of the reference driving video;

[0113] The judgment module 304 is used to determine whether the autonomous driving operation of the car corresponding to the real-time driving video meets the preset standard based on the real-time driving image sequence and the reference driving image sequence; if yes, it is determined that there is no risk in the autonomous driving operation of the car corresponding to the real-time driving video, and the real-time driving video captured by the in-vehicle camera continues to be acquired until the autonomous driving detection task is completed; if no, it is determined that there is a risk in the autonomous driving operation of the car corresponding to the real-time driving video, and the autonomous driving detection task is terminated.

[0114] Optionally, the judgment module 304 is used to determine the number of image comparison rounds based on the number of images in the real-time driving image sequence, and to perform image comparison operations corresponding to the number of image comparison rounds according to the shooting order of each frame in the real-time driving image sequence and the reference driving image sequence, to determine the similarity between the images in the real-time driving image sequence and the reference driving image sequence. Specifically, for the Nth round of image comparison operations, the Nth frame image in the real-time driving image sequence and the Nth frame image in the reference driving image sequence are determined according to the shooting order, and the similarity between the Nth frame image in the real-time driving image sequence and the Nth frame image in the reference driving image sequence is calculated, where N is a positive integer not less than 1. Based on each similarity, the similarity between the real-time driving image sequence and the reference driving image sequence is determined, and it is determined whether the similarity between the real-time driving image sequence and the reference driving image sequence meets a preset similarity value.

[0115] Optionally, each frame of the real-time driving video includes at least the dashboard and steering wheel;

[0116] The judgment module 304 is used to determine the target object in the Nth frame image of the real-time driving image sequence and the Nth frame image of the reference driving image sequence. The target object is either the dashboard or the steering wheel. The module performs image segmentation on the Nth frame image of the real-time driving image sequence to determine a first target object image, and performs image segmentation on the Nth frame image of the reference driving image sequence to determine a second target object image. The module calculates the similarity between the first target object image and the second target object image as the similarity between the Nth frame image of the real-time driving image sequence and the Nth frame image of the reference driving image sequence.

[0117] Optionally, the judgment module 304 is used to acquire data collected by various vibration sensors pre-set on the seat cushion, seat back, and floor of the vehicle, determine the root mean square value of the total weighted acceleration of the seat cushion, seat back, and floor based on the data collected by the vibration sensors, determine the navigation route recorded by the navigation terminal device pre-set in the vehicle, and determine the driving route corresponding to the reference driving video, determine the route difference based on the navigation route and the driving route, and determine the score corresponding to the real-time driving video according to the preset scoring rule text, the root mean square value of the total weighted acceleration, and the route difference. The scoring rule text records the scoring rules for scoring the real-time driving video. If the score corresponding to the real-time driving video does not reach the preset score, and the autonomous driving system of the vehicle corresponding to the real-time driving video is in the minimum risk strategy state, a prompt message is issued. The prompt message is used to prompt the user in the vehicle to drive manually.

[0118] Optionally, the judgment module 304 is used to determine whether there is a risk in the autonomous driving operation of the car corresponding to the real-time driving video. If so, it determines that the autonomous driving operation of the car corresponding to the real-time driving video obtains a safety compliance score accounting for 50% of the preset total score; it determines the interval range corresponding to the root mean square value of the total weighted acceleration according to the preset interval ranges, and determines the smoothness score according to the interval range corresponding to the root mean square value of the total weighted acceleration; it determines the efficiency score according to the preset difference ranges and the distance difference; it determines the score corresponding to the real-time driving video according to the safety compliance score, the smoothness score, and the efficiency score; if not, it does not obtain a safety compliance score, and determines that the score corresponding to the real-time driving video is 0.

[0119] Optionally, the judgment module 304 is used to determine whether the autonomous driving system of the car corresponding to the real-time driving video is in the minimum risk strategy state. If so, it is determined that the car corresponding to the real-time driving video has triggered the minimum risk strategy within the target time. If not, an alarm message is issued, which is used to indicate that the minimum risk strategy has not been triggered within the target time.

[0120] Optionally, the judgment module 304 is used to issue a prompt message when there is a risk in the autonomous driving operation of the car corresponding to the real-time driving video, and when the car corresponding to the real-time driving video triggers the minimum risk strategy within a target time. The prompt message is used to prompt the user in the car to drive manually, receive the instruction information to continue to execute the autonomous driving detection task, and continue to acquire the real-time driving video captured by the in-vehicle camera according to the instruction information until the autonomous driving detection task is completed.

[0121] This specification also provides a computer-readable storage medium storing a computer program that can be used to execute the above-described... Figure 1 This paper presents a risk detection method for autonomous driving of automobiles.

[0122] This instruction manual also provides Figure 4 The diagram shows a schematic structural representation of the electronic device. Figure 4 As shown, at the hardware level, this electronic device includes a processor, internal bus, network interface, memory, and non-volatile memory, and may also include other hardware required for business operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to achieve the above. Figure 1 The aforementioned risk detection method for autonomous driving of automobiles.

[0123] Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of hardware and software. In other words, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0124] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must also be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0125] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0126] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0127] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware components.

[0128] Those skilled in the art will understand that embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0129] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0130] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0131] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0132] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0133] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0134] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0135] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0136] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0137] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0138] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0139] The above description is merely an embodiment of this specification and is not intended to limit this specification. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this specification.

Claims

1. A risk detection method for autonomous driving of automobiles, characterized in that, include: The system acquires real-time driving video captured by an in-vehicle camera for a preset time, as well as a pre-captured reference driving video. The preset time is the same as the capture duration of the reference driving video. The in-vehicle camera is installed on the top of the vehicle and is used to capture videos including the driver's seat scene and the windshield during autonomous driving. Based on each frame of the real-time driving video, a real-time driving image sequence is determined, and based on each frame of the reference driving video, a reference driving image sequence is determined. Based on the real-time driving image sequence and the reference driving image sequence, it is determined whether the autonomous driving operation of the car corresponding to the real-time driving video meets the preset standard; If so, it is determined that there is no risk in the autonomous driving operation of the car corresponding to the real-time driving video, and the real-time driving video captured by the in-vehicle camera continues to be acquired until the autonomous driving detection task is completed. If not, it is determined that there is a risk in the autonomous driving operation of the car corresponding to the real-time driving video, and the autonomous driving detection task is terminated. Before acquiring the real-time driving video captured by the in-vehicle camera, the method further includes: acquiring data collected by various vibration sensors pre-set on the seat cushion, seat back, and floor of the vehicle. Based on the data collected by the vibration sensors, the root mean square value of the total weighted acceleration of the seat cushion, the seat back, and the floor is determined; the navigation route recorded by the navigation terminal device pre-set in the vehicle is determined, as well as the driving route corresponding to the reference driving video is determined; the distance difference is determined based on the navigation route and the driving route; a score corresponding to the real-time driving video is determined according to a preset scoring rule text, the root mean square value of the total weighted acceleration, and the distance difference, wherein the scoring rule text records the scoring rules for scoring the real-time driving video; if the score corresponding to the real-time driving video does not reach the preset score, and the autonomous driving system of the vehicle corresponding to the real-time driving video is in a minimum risk strategy state, a prompt message is issued, which is used to prompt the user in the vehicle to drive manually.

2. The risk detection method for autonomous driving of automobiles as described in claim 1, characterized in that, Based on the real-time driving image sequence and the reference driving image sequence, it is determined whether the autonomous driving operation of the vehicle corresponding to the real-time driving video meets the preset standard, specifically including: The number of image comparison wheels is determined based on the number of images in the real-time driving image sequence; According to the number of image comparison rounds, the shooting order of each frame in the real-time driving image sequence and the reference driving image sequence, the image comparison operation corresponding to the number of image comparison rounds is performed to determine the similarity between the images in the real-time driving image sequence and the reference driving image sequence. Specifically, for the Nth round of image comparison operation, the Nth frame image in the real-time driving image sequence and the Nth frame image in the reference driving image sequence are determined according to the shooting order, and the similarity between the Nth frame image in the real-time driving image sequence and the Nth frame image in the reference driving image sequence is calculated, where N is a positive integer not less than 1. Based on the similarity scores, the similarity between the real-time driving image sequence and the reference driving image sequence is determined. Determine whether the similarity between the real-time driving image sequence and the reference driving image sequence meets a preset similarity value.

3. The risk detection method for autonomous driving of automobiles as described in claim 2, characterized in that, Each frame of the real-time driving video includes at least the dashboard and the steering wheel; For the Nth frame image in the real-time driving image sequence and the Nth frame image in the reference driving image sequence, calculate the similarity between the Nth frame image in the real-time driving image sequence and the Nth frame image in the reference driving image sequence, specifically including: Identify the target object in the Nth frame of the real-time driving image sequence and the Nth frame of the reference driving image sequence, wherein the target object is either the dashboard or the steering wheel; The Nth frame image in the real-time driving image sequence is segmented to determine the first target image, and the Nth frame image in the reference driving image sequence is segmented to determine the second target image. Calculate the similarity between the first target image and the second target image, and use it as the similarity between the Nth frame image in the real-time driving image sequence and the Nth frame image in the reference driving image sequence.

4. The risk detection method for autonomous driving of automobiles as described in claim 1, characterized in that, The score corresponding to the real-time driving video is determined according to the preset scoring rule text, the root mean square value of the total weighted acceleration, and the distance difference, specifically including: Determine whether there is a risk in the vehicle's autonomous driving operation corresponding to the real-time driving video; If so, the vehicle's autonomous driving operation corresponding to the real-time driving video is determined to obtain a safety compliance score that accounts for 50% of the preset total score; the range corresponding to the root mean square value of the total weighted acceleration is determined according to the preset range of each range, and a smoothness score is determined according to the range corresponding to the root mean square value of the total weighted acceleration; an efficiency score is determined according to the preset range of each difference and the distance difference; and a rating corresponding to the real-time driving video is determined according to the safety compliance score, the smoothness score, and the efficiency score. If not, no safety compliance score will be obtained, and the score corresponding to the real-time driving video will be determined to be 0.

5. The risk detection method for autonomous driving of automobiles as described in claim 1, characterized in that, Before concluding the vehicle autonomous driving detection task, the method further includes: Determine whether the autonomous driving system of the car corresponding to the real-time driving video is in a minimum risk strategy state; If so, then determine that the car corresponding to the real-time driving video triggers the minimum risk strategy within the target time. If not, an alarm message will be issued to indicate that the minimum risk strategy has not been triggered within the target time.

6. The risk detection method for autonomous driving of automobiles as described in claim 5, characterized in that, The method further includes: If there is a risk in the autonomous driving operation of the car corresponding to the real-time driving video, and if the car corresponding to the real-time driving video triggers the minimum risk strategy within the target time, a prompt message is issued to prompt the user in the car to drive manually. The system receives an instruction to continue executing the vehicle autonomous driving detection task, and continues to acquire real-time driving videos captured by the in-vehicle camera according to the instruction until the vehicle autonomous driving detection task is completed.

7. A risk detection device for autonomous driving of automobiles, characterized in that, include: The acquisition module is used to acquire real-time driving video captured by the in-vehicle camera for a preset time and a pre-captured reference driving video. The preset time is the same as the shooting duration of the reference driving video. The in-vehicle camera is installed on the top of the vehicle and is used to capture videos including the driver's seat scene and the windshield during autonomous driving. The determining module is used to determine a real-time driving image sequence based on each frame of the real-time driving video, and to determine a reference driving image sequence based on each frame of the reference driving video; The judgment module is used to determine whether the autonomous driving operation of the car corresponding to the real-time driving video meets the preset standard based on the real-time driving image sequence and the reference driving image sequence; if yes, it is determined that there is no risk in the autonomous driving operation of the car corresponding to the real-time driving video, and the real-time driving video captured by the in-vehicle camera continues to be acquired until the autonomous driving detection task is completed; if no, it is determined that there is a risk in the autonomous driving operation of the car corresponding to the real-time driving video, and the autonomous driving detection task is terminated. The judgment module is also used to acquire data collected by various vibration sensors pre-set on the seat cushion, seat back and floor of the vehicle before continuing to acquire the real-time driving video captured by the in-vehicle camera. Based on the data collected by the vibration sensors, the root mean square value of the total weighted acceleration of the seat cushion, the seat back, and the floor is determined; the navigation route recorded by the navigation terminal device pre-set in the vehicle is determined, as well as the driving route corresponding to the reference driving video is determined; the distance difference is determined based on the navigation route and the driving route; a score corresponding to the real-time driving video is determined according to a preset scoring rule text, the root mean square value of the total weighted acceleration, and the distance difference, wherein the scoring rule text records the scoring rules for scoring the real-time driving video; if the score corresponding to the real-time driving video does not reach the preset score, and the autonomous driving system of the vehicle corresponding to the real-time driving video is in a minimum risk strategy state, a prompt message is issued, which is used to prompt the user in the vehicle to drive manually.

8. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the method described in any one of claims 1 to 6.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in any one of claims 1 to 6.

Images