Artificial intelligence-based intelligent vehicle assisted teaching method, device, medium and equipment

Abstract

The embodiments of the present application disclose an artificial intelligence-based intelligent vehicle assisted teaching method, apparatus, medium, and device, wherein the method includes: when determining that an intelligent vehicle has deviated from a track, retrieving a running video of the intelligent vehicle from entering the track to deviating from the track; based on the running video, determining at least two frames of images of a track element in the track before the intelligent vehicle deviated from the track; based on the at least two frames of images, determining the cause of the abnormality of the intelligent vehicle in the track element; and performing assisted teaching based on the abnormality. Using the embodiments of the present application, assisted teaching can enable students to promptly discover the cause of the fault, thereby performing debugging based on the fault cause, thereby improving the debugging efficiency of the intelligent vehicle.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and specifically to an artificial intelligence-based intelligent vehicle assisted teaching method, device, medium and equipment. Background Art

[0002] With the increasing popularity of semiconductor applications in automobiles, automotive electronic applications have covered all systems from automotive electronic control devices to on-board automotive electronic devices. The electrification of automobiles has become an inevitable trend in the development of the industry. In such an industry context, in order to cultivate cutting-edge scientific and technological talents for the future, smart car teaching has become one of the important experimental courses for colleges and universities to cultivate talents.

[0003] Smart cars are a product of the integration of the latest scientific and technological achievements, such as computers, with the modern automotive industry. Existing smart car instruction typically involves students repeatedly debugging the smart car program to enable it to automatically shift gears, identify roads, and navigate various racetrack elements. However, in actual experimental teaching, due to the difficulty of theoretical and practical knowledge of smart cars, the high degree of randomness in experiments, and the high speed of smart cars, when smart car experiments fail, students are unable to identify the cause of the failure in a timely manner and are unable to debug it accordingly, resulting in low debugging efficiency. Summary of the Invention

[0004] The present application provides an artificial intelligence-based intelligent vehicle assisted teaching method, device, medium and equipment, which can improve the debugging efficiency of intelligent vehicles.

[0005] In a first aspect of the present application, a method for teaching an intelligent vehicle based on artificial intelligence is provided, comprising:

[0006] When it is determined that the smart car has deviated from the track, a running video of the smart car from entering the track to deviating from the track is retrieved, and at least two frames of images of track elements of the track that the smart car entered before deviating from the track are determined based on the running video;

[0007] determining, based on the at least two frames of images, a cause of an abnormality of the smart car in the track element;

[0008] Auxiliary teaching is carried out according to the cause of the abnormality.

[0009] Through the above technical solution, video information of the smart car before it deviates from the track is retrieved, and two or more frames of images of the track element that the smart car entered before deviating from the track are determined. The abnormal cause of the smart car in the track element is determined based on the images, and auxiliary teaching is carried out based on the abnormal cause, so that students can discover the cause of the fault in time and debug according to the fault cause, thereby improving the debugging efficiency of the smart car.

[0010] Optionally, the track elements include curve elements, roundabout elements, crossroad elements, and fork road elements, and determining, based on the at least two frames of images, a cause of an abnormality of the smart car in the track elements includes:

[0011] Before determining that the smart vehicle enters any one of the curve element, the roundabout element, the transverse element, and the fork in the road element, calculating a first running speed of the smart vehicle based on the at least two frames of images;

[0012] If the first running speed exceeds a first speed threshold, it is determined that the smart car has a speeding abnormality in the track element.

[0013] By adopting the above technical solution, when the smart car enters a curve element, a roundabout element, a cross-section element, or a fork in the road, it needs to slow down. Therefore, there is a high possibility that the smart car will deviate from the track due to speeding. The running speed of the smart car is analyzed by using more than two frames of images to determine whether the smart car has a speeding abnormality.

[0014] Optionally, determining a cause of an abnormality of the smart car in the track element based on the at least two frames of images includes:

[0015] Before determining that the smart vehicle enters any one of the curve element, the roundabout element, the transverse element, and the fork in the road element, determining a deflection direction of the smart vehicle based on the at least two frames of images;

[0016] If the deflection direction is different from a preset deflection direction, it is determined that the smart car has an abnormal direction in the track element.

[0017] By adopting the above technical solution, before the smart car enters a curve element, a roundabout element, a cross-section element, or a fork in the road, it is necessary to perform image recognition through the camera installed on the smart car to determine the deflection direction. Since the smart car is likely to deviate from the track due to an incorrect deflection direction, the running speed of the smart car is analyzed by using more than two frames of images to determine whether the smart car has an abnormal direction.

[0018] Optionally, determining the cause of the abnormality of the smart car in the track element based on the at least two frames of images includes:

[0019] After determining that the smart vehicle enters any one of the curve element, the roundabout element, the transverse element, and the fork in the road element, calculating a second running speed of the smart vehicle based on the at least two frames of images;

[0020] If the deviation between the second running speed and the preset speed is greater than a speed threshold, it is determined that a speed abnormality occurs for the smart car in the track element.

[0021] By adopting the above technical solution, when the smart car enters a curve element, a roundabout element, a cross-section element, or a fork in the road, PID control of the speed is required to determine the speed when turning. Therefore, there is a high possibility that the smart car will deviate from the track due to speed error. By analyzing the deviation between the running speed of the smart car and the preset speed through more than two frames of images, it is determined whether the smart car has a speed abnormality.

[0022] Optionally, the track element includes a curve element and a roundabout element, and determining, based on the at least two frames of images, a cause of an abnormality of the smart car in the track element includes:

[0023] Before determining that the smart car enters the curve element or the roundabout element, calculating a deflection angle of the smart car based on the at least two frames of images;

[0024] If the deviation between the deflection angle and the preset angle is greater than an angle threshold, it is determined that the smart car has a line filling anomaly in the track element.

[0025] By adopting the above technical solution, before the smart car enters a curve element or a roundabout element, it is necessary to identify the track through the camera installed on the smart car, perform line-filling operations on the track, and thus calculate the fitted center line. The deflection angle of the smart car's servo is controlled according to the fitted center line. Therefore, there is a high possibility that the smart car will deviate from the track due to the deflection angle. The deflection angle of the smart car is analyzed by using more than two frames of images to determine whether the smart car has a line-filling abnormality.

[0026] Optionally, the track element includes a transverse element, and determining, based on the at least two frames of images, a cause of an abnormality of the smart car in the track element includes:

[0027] When it is determined that the smart car enters the transverse element, calculating, based on the at least two frames of images, a first distance between the smart car and the transverse obstacle when turning, and a second distance between the smart car and the transverse obstacle after returning to the track;

[0028] If the first distance or the second distance exceeds a distance threshold, it is determined that the smart car has made an abnormal turn in the track element.

[0029] By adopting the above technical solution, when the smart car enters a transverse element, it needs to turn before reaching a certain distance on one side of the transverse obstacle and return to the track after reaching a certain distance on the other side of the transverse obstacle. Therefore, there is a high possibility that the smart car will deviate from the track due to turning. The distance between the smart car and the transverse obstacle is analyzed by using more than two frames of images to determine whether the smart car has made an abnormal turn.

[0030] Optionally, the track element includes a cross element, and determining, based on the at least two frames of images, a cause of an abnormality of the smart car in the track element includes:

[0031] When it is determined that the smart car enters the cross element, judging whether the smart car has deflected according to the at least two frames of images;

[0032] If the smart car deflects, it is determined that the smart car has a feature point recognition anomaly in the track element.

[0033] By adopting the above technical solution, when the smart car enters the cross element, it needs to pass through the cross element directly. Since the smart car may misjudge the feature points in the cross element as curve elements and turn off the track, more than two frames of images are used to analyze whether the smart car has deflection, and then determine whether the smart car has feature point abnormalities.

[0034] In a second aspect of the present application, an artificial intelligence-based intelligent vehicle auxiliary teaching device is provided, the device comprising:

[0035] an image capture module, configured to retrieve, when determining that the smart car has deviated from the track, a running video of the smart car from the time it entered the track to the time it deviated from the track, and determine, based on the running video, at least two frames of images of track elements of the track that the smart car entered before deviating from the track;

[0036] an abnormality cause determination module, configured to determine an abnormality cause of the smart car in the track element based on the at least two frames of images;

[0037] The auxiliary teaching generation module is used to perform auxiliary teaching according to the abnormal cause.

[0038] In a third aspect of the present application, a computer-readable storage medium is provided, wherein the computer storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded by a processor and executing the above-mentioned method steps.

[0039] In a fourth aspect of the present application, an electronic device is provided, comprising: a processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the above-mentioned method steps.

[0040] In summary, this application has at least one of the following beneficial effects:

[0041] Through the technical solution of the present application, video information before the smart car deviates from the track is retrieved, and more than two frames of images of the track element in the track that the smart car entered before deviating from the track are determined. The abnormal cause of the smart car in the track element is determined based on the images, and auxiliary teaching is carried out based on the abnormal cause, so that students can discover the cause of the fault in time and debug according to the cause of the fault, thereby improving the debugging efficiency of the smart car. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0043] Figure 1 This is a flow chart of an artificial intelligence-based intelligent vehicle assisted teaching method provided in an embodiment of the present application;

[0044] Figure 2 Schematic diagram of an artificial intelligence-based smart car assisted teaching device provided in an embodiment of the present application;

[0045] Figure 3 This is a structural diagram of an electronic device provided in an embodiment of the present application.

[0046] Explanation of the accompanying symbols: 1. Device for assisted teaching of intelligent vehicles based on artificial intelligence; 11. Image capture module; 12. Abnormal cause determination module; 13. Assisted teaching generation module; 1000. Electronic device; 1001. Processor; 1002. Communication bus; 1003. User interface; 1004. Network interface; 1005. Memory. DETAILED DESCRIPTION

[0047] In order to enable people skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments.

[0048] In the description of the embodiments of this application, words such as "exemplary," "for example," or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary," "for example," or "for example" in the embodiments of this application should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary," "for example," or "for example" is intended to present the relevant concepts in a concrete manner.

[0049] In the description of the embodiments of the present application, the term "and / or" is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, B exists alone, and A and B exist at the same time. In addition, unless otherwise specified, the term "multiple" means two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals. In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "include", "comprise", "have" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0050] The present application is described in detail below with reference to specific embodiments.

[0051] In one embodiment, please refer to Figure 1 In this paper, we propose an AI-based intelligent vehicle-assisted teaching method. This method can be implemented using a computer program, a single-chip microcontroller, or run on an AI-based intelligent vehicle-assisted teaching device based on the von Neumann architecture. The computer program can be integrated into an application or run as a standalone tool.

[0052] Step 101: When it is determined that the smart car has deviated from the track, a running video of the smart car from entering the track to deviating from the track is retrieved, and at least two frames of track elements of the track entered by the smart car before deviating from the track are determined based on the running video.

[0053] Among them, the track refers to the runway of the smart car. Students download the debugging program to the controller of the smart car, so that the smart car can automatically run the entire track according to the program. In the embodiment of the present application, the track can be understood as being composed of multiple track elements, and the track elements can be further understood as being composed of partial tracks of different shapes, which can be straight elements, curve elements, roundabout elements, cross elements, fork elements, cross elements, etc.

[0054] Furthermore, after downloading the program to the smart car, the students placed it at the starting point of the track and pressed a switch to start the car's movement. During its movement, the smart car captured real-time images of the track through its own camera and processed them, controlling its servo deflection and speed, thus achieving automatic control. However, during the actual experiment, due to the smart car's high speed, when the smart car deviated from the track, the students were unable to accurately observe the cause of the malfunction and, therefore, were unable to modify the program accordingly, resulting in low debugging efficiency.

[0055] For example, in the smart car experimental classroom in the embodiment of the present application, an artificial intelligence-based smart car assisted teaching device is provided. The artificial intelligence-based smart car assisted teaching device is mainly composed of a camera, a controller and a display screen. The controller stores a program of the artificial intelligence-based smart car assisted teaching method. The camera can obtain and store the student's smart car experimental process in real time. When it is determined that the smart car deviates from the track, the controller calls up the running video of the smart car from entering the track to deviating from the track, and splits the running video into videos of the smart car driving in each track element. It determines that more than two frames of images of the smart car entering the track element before deviating from the track are analyzed to obtain the analysis results and display them through the display screen.

[0056] Step 102: Determine the cause of the abnormality of the smart car in the track element based on at least two frames of images.

[0057] Specifically, the smart car needs to have different driving strategies for different track elements. Since different track elements may cause different reasons for the failure of the smart car experiment, image analysis can be performed on the track element before the smart car deviates from the track based on more than two frames of images to determine the abnormal reason that caused the smart car to fail the experiment in this track element.

[0058] Based on the above embodiment, as an optional embodiment, determining the cause of the abnormality of the smart car in the track element based on at least two frames of images may further include the following steps:

[0059] Step 201: before determining that the smart vehicle enters any one of a curve element, a roundabout element, a transverse element, and a fork in the road element, a first running speed of the smart vehicle is calculated based on at least two frames of images.

[0060] Among them, the curve element in the embodiment of the present application refers to an arc-shaped track, the roundabout element in the embodiment of the present application refers to a circular track, the cross-section element in the embodiment of the present application refers to a straight track with obstacles, and the fork element in the embodiment of the present application refers to two arc-shaped tracks that branch off from the straight track.

[0061] For example, since the smart car needs to slow down and turn before entering the above-mentioned track element, there is a high possibility that the smart car will deviate from the track due to excessive speed. The controller can obtain more than two frames of images before the smart car is about to enter the above-mentioned track element in the running video, calculate the running speed of the smart car before entering the above-mentioned track, and define it as the first running speed.

[0062] Step 202: If the first running speed exceeds a first speed threshold, it is determined that the smart car has a speeding anomaly in the track element.

[0063] Among them, the first speed threshold in the embodiment of the present application can be understood as the maximum speed limit for passing the above-mentioned track element. Before entering the above-mentioned track element, if the running speed of the smart car exceeds the first speed threshold, it may cause the smart car to lose control when turning and thus deviate from the track.

[0064] For example, the calculated first operating speed is compared with a first speed threshold. If the first operating speed of the smart car exceeds the maximum speed limit of the track element being passed, it is determined that the smart car has experienced a speeding problem in the track element, and a display screen indicates that the smart car has experienced a speeding abnormality in the track element.

[0065] In another feasible implementation manner, as an optional embodiment, determining the cause of the abnormality of the smart car in the track element based on at least two frames of images may further include the following steps:

[0066] Step 301: before determining that the smart car enters any one of a curve element, a roundabout element, a crossroad element, and a fork in the road element, determining a turning direction of the smart car based on at least two frames of images.

[0067] For example, since the smart car needs to perform image recognition of the track after entering a curve element, a roundabout element, a cross-section element, and a fork in the road element to determine the deflection direction, when the image processing of the controller in the smart car is wrong, it may cause the smart car to control the deflection direction of the servo wrong, and thus there is a high possibility that the smart car will deviate from the track. The controller obtains more than two frames of images in the running video before the smart car enters the above-mentioned track elements to identify the deflection direction of the smart car.

[0068] Step 302: If the deflection direction is different from the preset deflection direction, it is determined that the direction of the smart car in the track element is abnormal.

[0069] Among them, the preset direction in the embodiment of the present application can be understood as the direction to be deflected through the above-mentioned track element. Before entering the above-mentioned track element, if the deflection direction of the smart car is inconsistent with the preset deflection direction, it may cause the smart car to directly deviate from the track.

[0070] Exemplarily, the obtained deflection direction is compared with a preset deflection direction. If it is determined that the two deflection directions are inconsistent, it is determined that the smart car has an incorrect deflection direction in the track element, and the display screen displays that the smart car has an abnormal direction in the track element.

[0071] In another feasible implementation manner, as an optional embodiment, determining the cause of the abnormality of the smart car in the track element based on at least two frames of images may further include the following steps:

[0072] Step 401: After determining that the smart vehicle enters any one of a curve element, a roundabout element, a transverse element, and a fork in the road element, a second running speed of the smart vehicle is calculated based on at least two frames of images.

[0073] For example, since the smart car needs to perform speed control after entering a curve element, a roundabout element, a cross-section element, or a fork in the road, the controller compares the actual speed of the smart car with the desired speed. When there is a deviation between the actual speed and the desired speed, the speed is adjusted using the PID algorithm. During adjustment, the angular velocity loop is first adjusted using coefficient P. When the car oscillates, coefficient I is adjusted to stabilize the smart car. However, if the smart car is disturbed, the system will oscillate again. At this time, coefficient D is added to stabilize the smart car. In order to adapt to the speed control of various track elements, students usually have great difficulty in actual debugging of PID parameters. Therefore, when the smart car enters the above track elements, there is a high possibility that the smart car will deviate from the track due to unstable operating speed control. The controller can obtain two or more frames of images in the running video after the smart car enters the above track element, calculate the running speed of the smart car after entering the above track, and define it as the second running speed.

[0074] Step 402: If the deviation between the second running speed and the preset speed is greater than the speed threshold, it is determined that the smart car has a speed anomaly in the track element.

[0075] Among them, the preset speed in the embodiment of the present application may refer to a suitable speed value for passing through the above-mentioned track elements, or may refer to a suitable speed range.

[0076] Exemplarily, the calculated second operating speed is subtracted from the preset speed to obtain a deviation value between the second operating speed and the preset speed, and the deviation value is compared with a speed threshold. If the deviation value is greater than the speed threshold, it is determined that the smart car has a speed control problem in the track element, and the display screen displays that the smart car has a speed abnormality in the track element.

[0077] In another feasible implementation manner, as an optional embodiment, determining the cause of the abnormality of the smart car in the track element based on at least two frames of images may further include the following steps:

[0078] Step 501: before determining that the smart car enters a curve element or a roundabout element, calculate the deflection angle of the smart car based on at least two frames of images.

[0079] For example, because curves and roundabouts have greater curvature than other curved tracks, the smart car's steering angle, or the control of the vehicle's deflection angle, requires higher control. The smart car must capture the track shape using its own camera, infill any track lines not captured, and calculate a fitted centerline based on the infilled lines. This infilled centerline is then used to control the steering angle. Therefore, there is a high probability that the smart car will deviate from the track due to the infilling operation. The controller can capture two or more frames of video footage after the smart car enters the aforementioned track element and calculate the deflection angle after the smart car enters the track.

[0080] Step 502: If the deviation between the deflection angle and the preset angle is greater than the angle threshold, it is determined that the smart car has a line filling anomaly in the track element.

[0081] Among them, the preset angle in the embodiment of the present application may refer to a suitable deflection angle value passing through the above-mentioned track elements, or may refer to a suitable deflection angle range.

[0082] Exemplarily, the calculated deflection angle is subtracted from the preset angle to obtain a deviation value between the deflection angle and the preset angle, and the deviation value is compared with an angle threshold. If the deviation value is greater than the angle threshold, it is determined that the smart car has an angle control problem in the track element, and the display screen displays that the smart car has a line filling abnormality in the track element.

[0083] In another feasible implementation manner, as an optional embodiment, determining the cause of the abnormality of the smart car in the track element based on at least two frames of images may further include the following steps:

[0084] Step 601: When it is determined that the smart car enters a transverse element, a first distance between the smart car and the transverse obstacle when turning and a second distance between the smart car and the transverse obstacle after returning to the track are calculated based on at least two frames of images.

[0085] For example, when a smart car enters a transverse element and identifies a transverse obstacle in the transverse element, it needs to deflect outward at a certain angle to leave the track, travel a certain distance, and then deflect back toward the track at a certain angle to return to the track, without colliding with the obstacle during the entire process. Therefore, in this embodiment of the application, the distance between the smart car and the transverse obstacle when turning is defined as the first distance, and the distance between the smart car and the transverse obstacle after returning to the track is defined as the second distance. If the first and second distances are too large or too small, the smart car will deviate from the track. The controller can obtain the first and second distances between the smart car and the transverse obstacle in the running video.

[0086] Step 602: If the first distance or the second distance exceeds the distance threshold, it is determined that the smart car has made an abnormal turn in the track element.

[0087] Exemplarily, the calculated first distance and second distance are compared with a distance threshold. If the first distance or the second distance of the smart car exceeds the distance threshold, it is determined that the smart car has a problem of premature deflection or late deflection in the track element, and the display screen displays that the smart car has a turning abnormality in the track element.

[0088] In another feasible implementation manner, as an optional embodiment, determining the cause of the abnormality of the smart car in the track element based on at least two frames of images may further include the following steps:

[0089] Step 701: When it is determined that the smart car enters a cross element, determine whether the smart car has deflected based on at least two frames of images.

[0090] For example, in the embodiments of this application, a cross element refers to a cross-shaped track. When a smart car passes through a cross track, it needs to use a camera installed on its own to obtain the track shape and identify feature points on the track shape based on the track shape. Based on the feature points, it further determines that it is a cross-element track and passes through it directly. If the smart car does not accurately identify the feature points, it may mistakenly identify the cross element as a track element requiring a turn, such as a curve element, and turn, causing the smart car to deviate from the track. The controller can obtain two or more frames of images in the running video before the smart car enters the cross track element to determine whether the smart car has deviated.

[0091] Step 702: If the smart car deflects, it is determined that a feature point recognition anomaly occurs in the track element of the smart car.

[0092] For example, if it is determined that the smart car has a deflection phenomenon, it is determined that the smart car has a steering gear angle problem in the cross element, and the display screen displays that the smart car has a feature point recognition abnormality in the track element.

[0093] Step 103: Perform auxiliary teaching according to the cause of the abnormality.

[0094] For example, in the embodiment of the present application, assisted teaching refers to retrieving the running video of the smart car before it deviates from the track when the smart car deviates from the track, further determining the abnormal reason for the smart car to deviate from the track based on the running video, and analyzing the abnormal reason by obtaining the running data of the smart car through image recognition, and displaying the analysis on the display screen, so that students can discover the abnormal reason in time and debug according to the abnormal reason, thereby improving the debugging efficiency of the smart car.

[0095] The following are device embodiments of the present application, which can be used to implement the method embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments of the present application.

[0096] Please refer to Figure 2, an artificial intelligence-based intelligent vehicle auxiliary teaching device provided in an embodiment of the present application, the artificial intelligence-based intelligent vehicle auxiliary teaching device 1 may include: an image capture module 11, an abnormality cause determination module 12 and an auxiliary teaching generation module 13, wherein:

[0097] The image capture module 11 is configured to retrieve a running video of the smart car from the time it enters the track to the time it deviates from the track when it is determined that the smart car deviates from the track, and determine, based on the running video, at least two frames of images of track elements of the track that the smart car entered before deviating from the track;

[0098] an abnormality cause determination module 12, configured to determine an abnormality cause of the smart car in the track element based on the at least two frames of images;

[0099] The auxiliary teaching generation module 13 is used to perform auxiliary teaching according to the abnormal cause.

[0100] Based on the above embodiment, as an optional embodiment, the abnormality cause determination module 12 includes: a first operating speed determination unit and an overspeed abnormality cause determination unit, wherein:

[0101] a first running speed determining unit, configured to calculate a first running speed of the smart vehicle based on the at least two frames of images before determining that the smart vehicle enters any one of the curve element, the roundabout element, the transverse element, and the fork in the road element;

[0102] The speeding abnormality cause determining unit is configured to determine that the smart car has a speeding abnormality in the track element if the first operating speed exceeds a first speed threshold.

[0103] Based on the above embodiment, as an optional embodiment, the abnormality cause determination module 12 further includes: a deflection direction determination unit and a direction abnormality cause determination unit, wherein:

[0104] a deflection direction determining unit, configured to determine a deflection direction of the smart vehicle based on the at least two frames of images before determining that the smart vehicle enters any one of the curve element, the roundabout element, the transverse element, and the fork in the road element;

[0105] The direction abnormality cause determining unit is configured to determine that the smart car has a direction abnormality in the track element if the deflection direction is different from a preset deflection direction.

[0106] Based on the above embodiment, as an optional embodiment, the abnormality cause determination module 12 further includes: a second operating speed determination unit and a speed abnormality cause unit, wherein:

[0107] a second running speed determining unit, configured to calculate a second running speed of the smart vehicle based on the at least two frames of images after determining that the smart vehicle has entered any one of the curve element, the roundabout element, the transverse element, and the fork in the road element;

[0108] The speed abnormality cause unit is configured to determine that a speed abnormality occurs in the smart car in the track element if a deviation between the second running speed and a preset speed is greater than a speed threshold.

[0109] Based on the above embodiment, as an optional embodiment, the abnormality cause determination module 12 further includes: a deflection angle determination unit and a line-patch abnormality cause determination unit, wherein:

[0110] a deflection angle determination unit, configured to calculate a deflection angle of the smart vehicle based on the at least two frames of images before determining that the smart vehicle enters the curve element or the roundabout element;

[0111] The line-filling abnormality cause determining unit is configured to determine that a line-filling abnormality occurs in the track element if a deviation between the deflection angle and a preset angle is greater than an angle threshold.

[0112] Based on the above embodiment, as an optional embodiment, the abnormality cause determination module 12 further includes: a running distance determination unit and a turning abnormality cause determination unit, wherein:

[0113] a running distance determining unit, configured to calculate, when it is determined that the smart car has entered the traversing element, a first distance between the smart car and the traversing obstacle when turning, and a second distance between the smart car and the traversing obstacle after returning to the track, based on the at least two frames of images;

[0114] The turning abnormality cause determining unit is configured to determine that the smart car has a turning abnormality in the track element if the first distance or the second distance exceeds a distance threshold.

[0115] Based on the above embodiment, as an optional embodiment, the abnormality cause determination module 12 further includes: a deflection phenomenon determination unit and a feature point recognition abnormality determination unit, wherein:

[0116] a deflection phenomenon determining unit, configured to determine whether a deflection phenomenon occurs on the smart car based on the at least two frames of images when it is determined that the smart car has entered the cross element;

[0117] The feature point recognition anomaly determination unit is configured to determine that a feature point recognition anomaly occurs in the track element if the smart car deflects.

[0118] The present application also provides a computer storage medium that can store multiple instructions, which are suitable for being loaded and executed by a processor as described above. Figure 1 The specific implementation process of the intelligent vehicle auxiliary teaching method based on artificial intelligence in the embodiment shown can be seen in Figure 1 The detailed description of the illustrated embodiment will not be repeated here.

[0119] See Figure 3 , is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 3 As shown, the electronic device 1000 may include: at least one processor 1001 , at least one network interface 1004 , a user interface 1003 , a memory 1005 , and at least one communication bus 1002 .

[0120] The communication bus 1002 is used to implement the connection and communication between these components.

[0121] The user interface 1003 may include a display screen (Display) and a camera (Camera). Optionally, the user interface 1003 may also include a standard wired interface and a wireless interface.

[0122] The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0123] The processor 1001 may include one or more processing cores. The processor 1001 utilizes various interfaces and circuits to connect various components within the electronic device 1000. It executes instructions, programs, code sets, or instruction sets stored in the memory 1005, and accesses data stored in the memory 1005 to perform various functions and process data within the electronic device 1000. Optionally, the processor 1001 may be implemented in hardware using at least one of a digital signal processing (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA). The processor 1001 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. The CPU primarily processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing content displayed on the display; and the modem handles wireless communications. It is understood that the modem may also be implemented independently of the processor 1001 and implemented on a separate chip.

[0124] Among them, the memory 1005 may include a random access memory (RAM) or a read-only memory (Read-Only Memory). Optionally, the memory 1005 includes a non-transitory computer-readable storage medium. The memory 1005 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 1005 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 1005 may optionally be at least one storage device located away from the aforementioned processor 1001. As Figure 3 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and an application program for an artificial intelligence-based smart car assisted teaching method.

[0125] It should be noted that the above embodiments provide devices that implement their functions using only the division of the above functional modules as examples. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiments are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0126] exist Figure 3 In the electronic device 1000 shown, the user interface 1003 is mainly used to provide an input interface for the user and obtain data input by the user; and the processor 1001 can be used to call an application program stored in the memory 1005 for a method of intelligent vehicle assisted teaching based on artificial intelligence. When executed by one or more processors, the electronic device executes one or more methods described in the above embodiments.

[0127] An electronic device readable storage medium stores instructions, which, when executed by one or more processors, enable the electronic device to execute one or more of the methods described in the above embodiments.

[0128] Those skilled in the art will clearly understand that the technical solution of this application can be implemented with the help of software and / or hardware. "Unit" and "module" in this specification refer to software and / or hardware that can independently complete or cooperate with other components to complete specific functions, where the hardware can be, for example, a field programmable gate array (FPGA) or an integrated circuit (IC).

[0129] It should be noted that for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0130] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0131] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interface, and the indirect coupling or communication connection of the device or unit can be electrical or other forms.

[0132] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0133] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0134] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a memory and includes a number of instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned memory includes: U disk, read-only memory (ROM), random access memory (RAM), mobile hard disk, magnetic disk or optical disk, and other media that can store program code.

[0135] A person skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be completed by a program instructing related hardware. The program can be stored in a computer-readable memory, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0136] The above is only an exemplary embodiment of the present disclosure and cannot be used to limit the scope of the present disclosure. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the disclosure of the specification and the truth of practice, those skilled in the art will easily think of other embodiments of the present disclosure. This application is intended to cover any variation, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the art that are not recorded in the present disclosure.

Claims

1. An artificial intelligence-based intelligent vehicle assisted teaching method, characterized in that: include: When it is determined that the smart car has deviated from the track, a running video of the smart car from entering the track to deviating from the track is retrieved, and at least two frames of images of track elements of the track that the smart car entered before deviating from the track are determined based on the running video; determining, based on the at least two frames of images, a cause of an abnormality of the smart car in the track element; The track elements include curve elements, roundabout elements, cross-section elements, and fork-road elements. Determining, based on the at least two frames of images, causes of abnormalities of the smart car in the track elements includes: Before determining that the smart vehicle enters any one of the curve element, the roundabout element, the transverse element, and the fork in the road element, calculating a first running speed of the smart vehicle based on the at least two frames of images; If the first running speed exceeds a first speed threshold, determining that the smart car has a speeding anomaly in the track element; Auxiliary teaching is carried out according to the cause of the abnormality.

2. The intelligent vehicle assisted teaching method based on artificial intelligence according to claim 1 is characterized in that: The determining, based on the at least two frames of images, a cause of an abnormality of the smart car in the track element includes: Before determining that the smart vehicle enters any one of the curve element, the roundabout element, the transverse element, and the fork in the road element, determining a deflection direction of the smart vehicle based on the at least two frames of images; If the deflection direction is different from a preset deflection direction, it is determined that the smart car has an abnormal direction in the track element.

3. The intelligent vehicle assisted teaching method based on artificial intelligence according to claim 1 is characterized in that: The determining, based on the at least two frames of images, a cause of an abnormality of the smart car in the track element includes: After determining that the smart vehicle enters any one of the curve element, the roundabout element, the transverse element, and the fork in the road element, calculating a second running speed of the smart vehicle based on the at least two frames of images; If the deviation between the second running speed and the preset speed is greater than a speed threshold, it is determined that a speed abnormality occurs for the smart car in the track element.

4. The intelligent vehicle assisted teaching method based on artificial intelligence according to claim 1 is characterized in that: The track elements include curve elements and roundabout elements, and determining, based on the at least two frames of images, a cause of an abnormality of the smart car in the track elements includes: Before determining that the smart car enters the curve element or the roundabout element, calculating a deflection angle of the smart car based on the at least two frames of images; If the deviation between the deflection angle and the preset angle is greater than an angle threshold, it is determined that the smart car has a line filling anomaly in the track element.

5. The intelligent vehicle assisted teaching method based on artificial intelligence according to claim 1 is characterized in that: The track element includes a transverse element, and determining, based on the at least two frames of images, a cause of an abnormality of the smart car in the track element includes: When it is determined that the smart car enters the transverse element, calculating, based on the at least two frames of images, a first distance between the smart car and the transverse obstacle when turning, and a second distance between the smart car and the transverse obstacle after returning to the track; If the first distance or the second distance exceeds a distance threshold, it is determined that the smart car has made an abnormal turn in the track element.

6. The intelligent vehicle assisted teaching method based on artificial intelligence according to claim 1 is characterized in that: The track element includes a cross element, and determining, based on the at least two frames of images, a cause of an abnormality of the smart car in the track element includes: When it is determined that the smart car enters the cross element, judging whether the smart car has deflected according to the at least two frames of images; If the smart car deflects, it is determined that the smart car has a feature point recognition anomaly in the track element.

7. An intelligent vehicle-assisted teaching device based on artificial intelligence, characterized in that: include: An image capture module (11) is used to retrieve a running video of the smart car from entering the track to deviating from the track when it is determined that the smart car has deviated from the track, and to determine, based on the running video, at least two frames of images of track elements in the track that the smart car entered before deviating from the track; An abnormality cause determination module (12) is used to determine an abnormality cause of the smart car in the track element based on the at least two frames of images; the track element includes a curve element, a roundabout element, a transverse element, and a fork in the road element, and the determining of the abnormality cause of the smart car in the track element based on the at least two frames of images includes: before determining that the smart car enters any one of the curve element, the roundabout element, the transverse element, and the fork in the road element, calculating a first running speed of the smart car based on the at least two frames of images; if the first running speed exceeds a first speed threshold, determining that the smart car has a speeding abnormality in the track element; An auxiliary teaching generation module (13) is used to perform auxiliary teaching according to the abnormal cause.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executed by a method according to any one of claims 1 to 6.

9. An electronic device, characterized in that: The electronic device comprises a processor, a memory and a transceiver, wherein the memory is used to store instructions, the transceiver is used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device executes the method according to any one of claims 1 to 6.

Images