An AI large-model vehicle external hazard source detection system

Abstract

The present invention relates to the technical field of automobile off-vehicle hazard source detection, and discloses an Ai large-model automobile off-vehicle hazard source detection system, comprising a main controller module, an image acquisition module, a wireless communication module, a six-axis attitude sensing module, an analysis and processing module, and a result output module; wherein the main controller module controls each module so that the hazard source detection system can work together to accurately detect off-vehicle hazard sources, and the image acquisition module detects real-time images and imports them into the analysis and processing module through the wireless communication module. The Ai large-model automobile off-vehicle hazard source detection system first detects the state of the hazard source and vehicle information through a dual-channel low-light night vision high-definition camera and a six-axis sensor, imports the information into a chip, extracts features and tabulates the hazard source through a YOLOv5 model, and feeds back the output information to a display screen for classification display through the main control module, while the hazard source type is broadcasted in a loudspeaker.

Description

Technical Field

[0001] The present invention relates to the technical field of automobile external hazard source detection, and in particular to an Ai large-model automobile external hazard source detection system. Background Art

[0002] There are many accidents of "door-opening killing" in motor vehicles. Related news reports frequently appear in the media, triggering public opinion and arousing high social concern, highlighting the urgency and necessity of preventing such accidents. "Door-opening killing" means that when the people in the car open the door, they do not carefully observe whether there are pedestrians or oncoming vehicles behind them, and rashly use the lane change assistance, resulting in pedestrians or vehicles passing by without time to react, causing a collision, and even casualties in severe cases. "Door-opening killing" accidents have a certain degree of predictability. As long as the driver and passengers fully observe the surrounding traffic conditions and confirm safety before opening the door and getting off the car, they can effectively avoid accidents.

[0003] After searching, according to the invention patent with Chinese patent number CN107472131A, which discloses a car door opening warning device, a car door opening warning system, and a car door opening warning method, the invention patent uses millimeter wave radar to detect obstacles behind the car and provide technical means for reminders. However, the cost of the device is relatively high during use, and the cost of a single device is as high as one thousand yuan. The function is single and cannot meet customer requirements. Only through indicator lights, rear passengers cannot directly observe the outside environment of the car. In addition, the current solution has electromagnetic radiation. The transmission power of radar and other equipment reaches 12dBm, and the power consumption is 12W, which causes partial electromagnetic radiation to the driving environment. Some people are worried about radiation to babies and are unwilling to install it. In addition, the installation is complicated. The wiring harness of the traditional solution is complicated. The radar installation requires the removal of the rear bumper, which requires professionals to complete the installation. Therefore, an Ai large model car external hazard source detection system is proposed to solve the above problems. Summary of the invention

[0004] 1. Technical issues to be resolved In view of the shortcomings of the prior art, the present invention provides an Ai large-model automobile external hazard source detection system, which has the advantages of being smarter, easier to install, more environmentally friendly and low-cost. It solves the problem that the cost of the equipment is relatively high during use, the cost of a single device is as high as one thousand yuan, the function is single and cannot meet customer requirements, and the rear passengers cannot intuitively observe the external environment of the car through only indicator light prompts. In addition, the current solution has electromagnetic radiation, and the transmission power of radar and other equipment reaches 12dBm, and the power consumption is 12W, which causes partial electromagnetic radiation to the driving environment. Some people are worried about the radiation to babies and are unwilling to install it. In addition, the installation is complicated, the wiring harness of the traditional solution is complicated, and the radar installation requires the removal of the rear bumper, which requires professionals to complete the installation.

[0005] (II) Technical solution In order to achieve the above-mentioned purpose of being more intelligent, easy to install, more environmentally friendly and low-cost, the present invention provides the following technical solutions: an Ai large-model vehicle external hazard source detection system, including a main controller module, an image acquisition module, a wireless communication module, a six-axis attitude sensing module, an analysis and processing module and a result output module; The main controller module controls each module so that the hazard source detection system can work together to accurately detect the hazard sources outside the vehicle; The image acquisition module detects the real-time image and imports it into the analysis and processing module through the wireless communication module; The six-axis attitude sensing module can detect vehicle attitude and motion information, providing key perception capabilities for this product; The analysis and processing module includes a high-performance AI chip and a YOLOv5 model. The high-performance AI chip has a computing power of 3.6tops to provide computing support for image analysis. The YOLOv5 model can efficiently and accurately detect dangerous sources outside the vehicle. The result output module includes an image display module and a voice reminder module, which presents the rear video of the vehicle and the approaching moving target through the display screen to improve the user experience, and the voice reminder module plays the danger sources of different scenes in a classified manner.

[0006] Preferably, the high-performance AI chip adopts the Aixin Yuanzhi AX620A developed by the domestic Aixin Yuanzhi. The chip has a computing power of 3.6tops and is used for processing and analysis of images and videos, providing this product with environmental perception and decision-making support capabilities.

[0007] Preferably, the image acquisition module adopts a GC4653 low-light night vision high-definition camera, which can clearly detect approaching moving targets from the side and rear even in a dimly lit night environment.

[0008] Preferably, the image display module is a 4.5-inch MIPI ultra-high-definition display screen, which can clearly present the video behind the vehicle and nearby moving targets.

[0009] Preferably, the YOLOv5 model target detection module has a Precision-Recall Curve embedded inside to accurately detect pedestrians, bicycles and motorcycles.

[0010] Preferably, the YOLOv5 model includes an input, a backbone network, a neck network, and a prediction head, wherein the backbone is mainly composed of a CBS module, a fast spatial pyramid pooling (Spatial Pyramid Pooling Fast, SPPF) module, and a C3 module.

[0011] Preferably, the design of the C3 module makes use of the idea of ​​cross-stage local network gradient diversion and residual structure, and is composed of three convolution structures and N bottleneck structures.

[0012] Preferably, the voice reminder module is a broadcasting speaker.

[0013] Preferably, the Aixin Yuanzhi AX620A runs the YOLOv5s model for inference with a low power consumption of 3.2W, and only takes 23ms for a single 640 x 640 image.

[0014] (III) Beneficial effects Compared with the prior art, the present invention provides an Ai large-scale model vehicle external hazard source detection system, which has the following beneficial effects: The Ai large-model automobile external hazard source detection system can detect external hazards more accurately and efficiently through the coordinated work of the main controller module, image acquisition module, wireless communication module, six-axis attitude sensing module, analysis and processing module and result output module, thereby avoiding door-opening kills. The device is more intelligent, easy to install, more environmentally friendly and low-cost.

[0015] The AI ​​large-model vehicle external hazard detection system displays videos of approaching moving targets and labels and classifies them with rectangular frames. When a hazard approaches, video annotation + voice classification broadcast of "motorcycle", "bicycle", "car", etc. are used to assist the driver and rear passengers in making judgments. The cost is low, with the product's estimated material cost being only 200 yuan, which is about 400% less than similar products. The system has high reliability and high AI detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a schematic diagram of a vehicle external hazard source detection system of the present invention; Figure 2 It is a schematic diagram of the product principle of the present invention; Figure 3 Schematic diagram of the detection accuracy of the present invention. DETAILED DESCRIPTION

[0017] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] See also Figure 1-3 , an Ai large-model vehicle external hazard source detection system, including a main controller module, an image acquisition module, a wireless communication module, a six-axis attitude sensing module, an analysis and processing module and a result output module; Among them, the main controller module controls each module so that the hazard source detection system can work together to accurately detect the hazard sources outside the vehicle; The image acquisition module detects the real-time image and imports it into the analysis and processing module through the wireless communication module; The six-axis attitude sensor module can detect vehicle attitude and motion information, providing key perception capabilities for this product; The analysis and processing module includes a high-performance AI chip and a YOLOv5 model. The high-performance AI chip has a computing power of 3.6tops to provide computing support for image analysis, and the YOLOv5 model can efficiently and accurately detect dangerous sources outside the vehicle. The result output module includes an image display module and a voice reminder module. The display screen presents the video behind the vehicle and nearby moving targets to improve the user experience. The voice reminder module plays the dangerous sources of different scenarios in a classified manner.

[0019] exist Figure 1 and Figure 2 The high-performance AI chip uses the Aixin Yuanzhi AX620A developed by the domestic Aixin Yuanzhi. The chip has a computing power of 3.6tops and is used for image and video processing and analysis, providing this product with environmental perception and decision-making support capabilities.

[0020] Specifically, Aixin Yuanzhi AX620A runs the YOLOv5s model for inference with a low power consumption of 3.2W. A single 640x640 image only takes 23ms, making the inference faster and less energy consuming.

[0021] exist Figure 1 and Figure 2 The image acquisition module uses the GC4653 low-light night vision high-definition camera, which can clearly detect approaching moving targets from the side and rear even in a dimly lit night environment.

[0022] Specifically, the GC4653 low-light-level night vision high-definition camera can detect approaching targets in a variety of harsh conditions such as rainy days, snowy nights, etc., making the detection of dangerous sources more accurate.

[0023] exist Figure 1 and Figure 2 The image display module is a 4.5-inch MIPI ultra-high-definition display, which can clearly display the video behind the car and nearby moving targets.

[0024] Specifically, the targets detected by the GC4653 low-light night vision HD camera are processed by the analysis and processing module and then imported into the image display module, allowing the user to observe approaching targets through the MIPI ultra-high-definition display. At the same time, the main controller module will broadcast through the speaker whether the current dangerous object is a pedestrian, bicycle or motorcycle.

[0025] exist Figure 1 and Figure 2 In the target detection module of the YOLOv5 model, the Precision-Recall Curve is embedded inside to accurately detect pedestrians, bicycles and motorcycles. The YOLOv5 model includes the input, backbone network, neck network and prediction head. The backbone is mainly composed of the CBS module, the Spatial Pyramid Pooling Fast (SPPF) module and the C3 module. The design of the C3 module uses the idea of ​​cross-stage local network gradient diversion and residual structure, and consists of three convolution structures and N bottleneck structures.

[0026] Input: responsible for preprocessing the input image. It mainly includes technologies such as Mosaic image enhancement, adaptive anchor box calculation, and adaptive image scaling. Mosaic image enhancement is a data enhancement method that generates new training images by combining multiple different images to improve the generalization ability of the model.

[0027] Backbone: Responsible for extracting multi-level features of the input image. The backbone network of YOLOv5 adopts CSPDarknet53, which is constructed by stacking multiple CBS modules and C3 modules and finally connecting an SPPF module. This structure reduces redundant calculations, improves model efficiency, and optimizes the feature extraction process; Neck network: Improve detection performance through multi-scale fusion; Prediction head: Target prediction is performed based on the fused multi-scale feature map. The prediction head receives the feature map generated by the neck network and uses the preset prior box to perform confidence calculation and bounding box regression on each pixel in the feature map to obtain a multidimensional array including object category, category confidence, box coordinates, width and height information. ‌‌

[0028] In addition, YOLOv5 model training also includes the following steps: 1) Build a database, import more than 30,000 collected road condition images into the database, and divide the images in the database into training sets and test sets; 2) Import the image into the YOLOv5 model, use the labelImg annotation tool to annotate the image, and generate the corresponding .xml annotation files, which contain information such as the location and category of the dangerous source close to the vehicle; 3) Convert the files to a file type that is compatible with the YOLOv5 model, train the YOLOv5 model for automobile hazard source detection using the training set, and verify the accuracy of hazard source detection using the test set; 4) Output a multidimensional array including object category, category confidence, box coordinates, width, distance, speed, and height information; 5) After training is completed, the YOLOv5 model is developed into software and embedded in the vehicle computer development.

[0029] exist Figure 3 The accuracy of the model's Precision-Recall Curve reached: people (pedestrians) 96.7%, car 98.1%, bike (bicycle) 97.3%, motorcycle (motorcycle) 97.4%, which is higher than the standard of "Performance Requirements and Test Methods for Passenger Car Door Opening Warning System" (Draft) issued by the National Automobile Committee.

[0030] Obvious product advantages: (1) Display the video of the moving target and mark it with a rectangular frame; (2) When a dangerous source approaches, video annotation and voice classification broadcast of "motorcycle", "bicycle", "car", etc. are used to assist the driver and rear passengers in making judgments; (3) Low cost. The estimated material cost of the product is only RMB 200, which is about 400% lower than that of similar products. (4) High reliability.

[0031] The specific parameters are shown in Table 1.

[0032] Table 1 Overview of the reliability test structure of this product To sum up, the Ai large-model automobile external hazard source detection system can detect external hazards more accurately and efficiently through the coordinated work of the main controller module, image acquisition module, wireless communication module, six-axis attitude sensing module, analysis and processing module and result output module, thereby avoiding door opening kills. The device has the advantages of being smarter, easier to install, more environmentally friendly and low-cost.

[0033] In addition, the same effects achieved in this patent by replacing chips or AI algorithms are all within the scope of protection of this patent.

[0034] In addition, the target is detected by the YOLOv5 model, and the steps include: Step 1: Preprocess the format and normalization of more than 30,000 input traffic condition images; Step 2: Input the preprocessed image into the YOLOv5 model to obtain the prediction results, including bounding box coordinates and object category probabilities; Step 3: Post-processing using non-maximum suppression; Step 4: The output results are bicycles, motorcycles and pedestrians, and anchor frames are drawn for them; Step 5: Develop the trained YOLOv5 model into software and embed it into the vehicle computer development; Step 6: When the car is driving, the GC4653 low-light night vision HD camera is used to monitor the road conditions and import the information into the YOLOv5 model; Step 7: The detection results are imported into the output module. At this time, the display screen displays the current image and marks the borders of the approaching danger sources. At the same time, the loudspeaker classifies and broadcasts the approaching bicycles, motorcycles, pedestrians, etc.

[0035] In addition, first of all, the status of the hazard source and vehicle information are detected through a dual-channel low-light night vision high-definition camera and a six-axis sensor, and the information is imported into the chip. The YOLOv5 model extracts features and frames the hazard source, and the output information is fed back to the display screen through the main control module for classification display. At the same time, the type of hazard source is announced on the loudspeaker, which solves the high cost of the equipment during use. The cost of a single device is as high as one thousand yuan, the function is single, and it cannot meet customer requirements. Only through the indicator light prompt, the rear passengers cannot intuitively observe the outside environment of the car. In addition, the current solution has electromagnetic radiation. The transmission power of radar and other equipment reaches 12dBm, and the power consumption is 12W, which causes some electromagnetic radiation to the driving environment. Some people are worried about radiation to babies and are unwilling to install it. In addition, the installation is complicated. The wiring harness of the traditional solution is complicated. The radar installation requires the removal of the rear bumper, and the installation requires professionals to complete the installation.

[0036] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0037] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An Ai large-scale model vehicle external hazard source detection system, characterized by: It includes a main controller module, an image acquisition module, a wireless communication module, a six-axis attitude sensing module, an analysis and processing module, and a result output module; The main controller module controls each module so that the hazard source detection system can work together to accurately detect the hazard sources outside the vehicle; The image acquisition module detects the real-time image and imports it into the analysis and processing module through the wireless communication module; The six-axis attitude sensing module can detect vehicle attitude and motion information, providing key perception capabilities for this product; The analysis and processing module includes a high-performance AI chip and a YOLOv5 model. The high-performance AI chip has a computing power of 3.6tops to provide computing support for image analysis. The YOLOv5 model can efficiently and accurately detect dangerous sources outside the vehicle. The result output module includes an image display module and a voice reminder module, which presents the rear video of the vehicle and the approaching moving target through the display screen to improve the user experience, and the voice reminder module plays the danger sources of different scenes in a classified manner.

2. The Ai large-model vehicle external hazard source detection system according to claim 1 is characterized by: The high-performance AI chip uses the Aixin Yuanzhi AX620A developed by the domestic Aixin Yuanzhi. The chip has a computing power of 3.6tops and is used for image and video processing and analysis, providing this product with environmental perception and decision-making support capabilities.

3. The Ai large-model vehicle external hazard source detection system according to claim 1 is characterized by: The image acquisition module adopts the GC4653 low-light night vision high-definition camera, which can clearly detect the approaching moving targets at the side and rear even in a night environment with weak light.

4. The Ai large-model vehicle external hazard source detection system according to claim 1 is characterized by: The image display module is a 4.5-inch MIPI ultra-high-definition display screen, which can clearly present the video behind the vehicle and nearby moving targets.

5. The Ai large-model vehicle external hazard source detection system according to claim 1 is characterized by: The YOLOv5 model target detection module embeds a Precision-Recall Curve to accurately detect pedestrians, bicycles, and motorcycles.

6. The Ai large-model vehicle external hazard source detection system according to claim 1 is characterized by: The YOLOv5 model includes an input, a backbone network, a neck network, and a prediction head, wherein the backbone is mainly composed of a CBS module, a fast spatial pyramid pooling (Spatial Pyramid PoolingFast, SPPF) module, and a C3 module.

7. The Ai large-model vehicle external hazard source detection system according to claim 6 is characterized by: The design of the C3 module makes use of the ideas of cross-stage local network gradient diversion and residual structure, and consists of three convolution structures and N bottleneck structures.

8. The Ai large-model vehicle external hazard source detection system according to claim 1 is characterized by: The voice reminder module is a broadcasting speaker.

9. The Ai large-model vehicle external hazard source detection system according to claim 1 is characterized by: The Aixin Yuanzhi AX620A runs the YOLOv5s model for inference with a low power consumption of 3.2W. It only takes 23ms for a single 640 x 640 image.

Images