A risc-v architecture-based internet of vehicles multi-scene target detection system and method

By setting a fixed DMA cache area and utilizing a cache consistency protocol on the RISC-V architecture, the problems of high memory usage and slow response speed in vehicle network target detection are solved, achieving efficient and low-energy target detection and improving system performance and flexibility.

CN119672294BActive Publication Date: 2025-10-17CHONGQING UNIV
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202411648036.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-11-18
Publication Date
2025-10-17
Estimated Expiration
2044-11-18

AI Technical Summary

Technical Problem

Traditional embedded processor architectures have high power consumption, high cost, and poor flexibility in target detection in the Internet of Vehicles. In addition, existing technologies for target detection image processing have low efficiency, high memory usage, and slow response speed.

Method used

A multi-scenario target detection system for the Internet of Vehicles based on the RISC-V architecture is adopted. By setting a fixed DMA cache area on the RISC-V hardware computing module, using the mmap mechanism to map image data, combining RVV instructions for preprocessing, and using the cache consistency protocol for model inference and post-processing, it reduces repeated memory requests and I/O operations.

Benefits of technology

It improves the real-time frame rate and system performance of target detection, reduces energy consumption, improves system response speed and processing efficiency, and enhances system flexibility and scalability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119672294B_ABST
    Figure CN119672294B_ABST
Patent Text Reader

Abstract

The present application relates to the technical field of vehicle networking target detection, in particular to a vehicle networking multi-scene target detection system and method based on RISC-V architecture, comprising: step S1: obtaining image data stored in a preset DMA cache area on a RISC-V hardware computing module, the DMA cache area being set to a fixed size; step S2: mapping the image data of the DMA cache area to the address space of a process on the RISC-V hardware computing module by using a mmap mechanism; step S3: pre-processing the image data by using an RVV instruction; step S4: after loading the target detection model parameters for the first time, caching the pre-processed image data into a heterogeneous shared memory by using a cache consistency protocol supported by the RISC-V hardware computing module, performing model reasoning, and obtaining perception information; and step S5: post-processing the model reasoning result by using a self-defined post-processing function, and obtaining display information; the real-time frame rate of target detection, the overall performance of the system and the memory access efficiency are improved.
Need to check novelty before this filing date? Find Prior Art

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of vehicle networking target detection, in particular to a vehicle networking multi-scene target detection system and method based on RISC-V architecture. BACKGROUND

[0002] With the rapid development of intelligent transportation systems, vehicle networking (V2X) technology has become a key factor in realizing autonomous driving and improving road safety. In the vehicle networking environment, real-time communication between vehicles and the surrounding environment (including other vehicles, pedestrians, infrastructure, etc.) is crucial. In order to effectively handle complex traffic scenarios, high-performance target detection algorithms are needed to identify potential safety risks. Traditional embedded processor architectures cannot meet this demand in terms of power consumption, cost and flexibility.

[0003] RISC-V is an open instruction set architecture (ISA) based on the principles of reduced instruction set computing (RISC), aiming to provide a free and open standard for the industry. However, there is currently a lack of research specifically based on RISC-V architecture in the field of vehicle networking target detection.

[0004] In addition, the target detection image usually occupies a large amount of memory space due to each frame of image data. In the prior art, repeated large-capacity memory allocation and release are performed every time a frame of data is processed, resulting in a decrease in performance. The YOLOv5 model is reloaded every time an image is processed. Given that the parameter quantity of the YOLOv5 model can reach millions or even tens of millions, this not only increases the processing time but also causes unnecessary memory occupation. After the image data model inference is completed, the inference result is usually saved to a file, then the result is read from the file, and finally the bounding box result is rendered to the original image. This method not only increases the additional I / O operation but also causes processing delay. SUMMARY

[0005] In view of the above-mentioned shortcomings of the prior art, the present application aims to provide a vehicle networking multi-scene target detection system and method based on RISC-V architecture, which solves the technical problems of low efficiency, high memory occupation, high energy consumption, slow response speed, etc. of the vehicle target detection system.

[0006] To achieve the above-mentioned purpose, the present application provides a vehicle networking multi-scene target detection method based on RISC-V architecture, comprising:

[0007] Step S1: store the obtained image data in the pre-set DMA buffer area on the RISC-V hardware computing module, and set the DMA buffer area to a fixed size;

[0008] Step S2: mapping the image data of the DMA cache area to the address space of the process on the RISC-V hardware computing module by using the mmap mechanism;

[0009] Step S3: preprocessing the image data by using the RVV instruction;

[0010] Step S4: after loading the YOLOv5n target detection model parameters for the first time, caching the preprocessed image data into the heterogeneous shared memory by using the cache coherence protocol supported by the RISC-V hardware computing module, performing model inference, and obtaining perception information;

[0011] Step S5: post-processing the model inference result by using a self-defined post-processing function, and obtaining display information.

[0012] The application also provides a RISC-V architecture-based Internet of Vehicles multi-scenario target detection system, comprising:

[0013] A data acquisition module for capturing surrounding image data of a vehicle;

[0014] A target detection module for processing the captured image data to obtain display information;

[0015] A data communication module for obtaining the display information of the own vehicle and sharing it with other vehicles, and obtaining the display information from other vehicles;

[0016] A user interaction module for obtaining and presenting the display information from the own vehicle and from other vehicles;

[0017] The target detection module and the user interface interaction module are both arranged on the RISC-V hardware computing module;

[0018] The target detection module performs target detection according to the method described above.

[0019] The application has the advantages that: the application predefines a fixed uppercase DMA cache area on the RISC-V hardware computing module, which can reduce the repeated application and release of memory, thereby improving the real-time frame rate of target detection and the overall performance of the system; Step S2 realizes the mapping of virtual addresses to physical addresses, avoiding repeated mapping and memory reallocation during each frame of data processing; in this way, the address mapping can be reused, thereby reducing the overhead of memory management and improving the memory access efficiency. In addition, the low-power feature of the RISC-V architecture development board enables the method to run continuously and efficiently under limited energy conditions, reducing energy consumption; in addition, the flexible scalability of the RISC-V architecture development board ensures the high adaptability and upgradability of the method, providing excellent performance and practicality for the Internet of Vehicles multi-scenario target detection, and improving its portability and flexibility.

[0020] Compared with step S4, previous methods reload the YOLOv5 model when processing each frame of image. Given that the number of parameters of the YOLOv5 model can reach millions or even tens of millions, this not only increases processing time but also leads to unnecessary memory usage. In step S4 of this method, the cache consistency protocol supported by the RISC-V hardware computing module ensures data consistency in the cache between multi-core processors, avoiding repeated loading of model parameters when processing each frame of image, effectively reducing memory usage and improving the overall response speed of the system to which this method is applied.

[0021] Compared with step 5, the previous method is to save the model inference results to a file after the model inference is completed, then read the model inference results from the file, and finally render the bounding box results onto the original image. This method not only adds additional I / O operations but also causes processing delays. However, this method directly post-processes the image data with the model inference results through a custom post-processing function after the model inference is completed, thereby reducing unnecessary I / O operations and improving the real-time performance and processing efficiency of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is a schematic diagram of the structure of a multi-scenario target detection system for an Internet of Vehicles based on the RISC-V architecture in an embodiment of the present invention;

[0023] Figure 2 This is a flowchart of a multi-scenario target detection method for the Internet of Vehicles based on the RISC-V architecture in an embodiment of the present invention. DETAILED DESCRIPTION

[0024] The following describes the embodiments of the present invention through specific examples. Those skilled in the art will readily understand the other advantages and benefits of the present invention from the disclosure herein. The present invention may also be implemented or applied through various other specific embodiments, and the details in this specification may be modified or altered based on different viewpoints and applications without departing from the spirit of the present invention.

[0025] It should be noted that the vehicles in the following embodiments are divided into self-vehicles and other vehicles. "Self-vehicle" specifically refers to the vehicle used as a reference benchmark, while "other vehicles" refers to any other vehicles other than the self-vehicle; the above-mentioned self-vehicle and other vehicles are switched in name according to the changes in the reference benchmark vehicle.

[0026] This embodiment provides a multi-scenario target detection method for Internet of Vehicles based on RISC-V architecture. Figure 1 The system implementation shown includes:

[0027] The data acquisition module is configured to capture the surrounding image data of the vehicle. In this example, the data acquisition module is configured by a vehicle-mounted sensor, which includes but is not limited to a vehicle-mounted camera. The vehicle-mounted sensor is configured to capture the surrounding image data of the vehicle.

[0028] The target detection module is configured to process the captured image data to obtain the display information.

[0029] The data communication module is configured to obtain the display information of the vehicle and share the display information with other vehicles, and obtain the display information from other vehicles.

[0030] The user interaction module is configured to obtain and present the display information of the vehicle and the display information of other vehicles.

[0031] The target detection module and the user interface interaction module are both arranged on the RISC-V hardware computing module. The target detection module includes a convolutional neural network (CNN) accelerator and a matrix operation unit. The image data obtained by the data acquisition module is preprocessed by using the RVV instruction, detected and inferred by using a target detection model, and perception information is obtained. The model inference result is output through the RISC-V hardware accelerator, and the model inference result is post-processed through a self-defined post-processing function to obtain the display information.

[0032] Specifically, the data communication module uses Internet of Vehicles technology to realize the transmission of display information data between vehicles, and can select a V2V communication mode or a V2I communication mode. The user interaction module is configured to present the display information. The user interaction module includes a vehicle-mounted display screen and a visual interface software system. The display information includes the category, position, size and confidence of potential obstacles such as pedestrians and vehicles in front of the vehicle, as well as information such as traffic signal lights and road signs. The display information facilitates the drivers of the vehicle and other vehicles to make corresponding decisions according to different scene requirements as follows:

[0033] Scene one, blind area obstacle detection: the vehicle can observe the potential obstacles such as pedestrians and vehicles in the blind area of the vehicle through the user interface interaction module in real time, so as to take measures in advance to avoid traffic collision accidents such as "ghost probe";

[0034] Scene two, traffic sign recognition: the vehicle can observe the shared detected traffic sign information and traffic signal light state through the user interface interaction module in real time. If the vehicle is about to violate the traffic rules, such as overspeeding or not following the specific sign indication, the vehicle can immediately warn the driver and prompt the driver to slow down or follow the specific sign indication.

[0035] Scene three, lane line detection: the vehicle observes the detected lane line information shared by other vehicles through the user interface interaction module in real time, and can ensure the vehicle to drive in the correct lane in the case of limited vision such as curved road, tunnel entrance, etc., to prevent unintentional lane changing.

[0036] The RISC-V architecture-based vehicle networking multi-scene target detection method in the embodiment is applied to a vehicle equipped with a RISC-V architecture development board as a central control unit. The RISC-V architecture development board constructs a RISC-V hardware computing module. The RISC-V hardware computing module is connected with a vehicle-mounted sensor of a data acquisition module, a vehicle networking device OBU of a data communication module, and a vehicle-mounted display screen of a user interface interaction module through various different interfaces such as USB, HDMI, and Ethernet provided by the RISC-V hardware computing module. Meanwhile, a YOLOv5n target detection model of the target detection module and a visual interface software system of the user interface interaction module are compiled and set on the RISC-V architecture development board. Preferably, the YOLOv5n target detection model is transplanted into the RISC-V hardware computing module through cross-compilation. The flowchart of the method is basically as shown in Figure 2

[0037] Step S1: store the obtained image data in a pre-set DMA (direct memory access) cache area on the RISC-V hardware computing module, and set the DMA cache area as a fixed size;

[0038] Step S2: map the image data of the DMA cache area to the address space of the process on the RISC-V hardware computing module by using the mmap mechanism;

[0039] Step S3: pre-process the image data by using the RVV instruction (RISC-V Vector Extension);

[0040] Step S4: after loading the target detection model parameters for the first time, cache the pre-processed image data in the heterogeneous shared memory by using the cache coherence protocol (such as the MESI or MOESI protocol) supported by the RISC-V hardware computing module, perform model inference, and obtain perception information;

[0041] Step S5: post-process the model inference result by using a self-defined post-processing function, and obtain display information.

[0042] ​In the prior art, due to the large memory space occupied by each frame of image data, the previous method repeatedly applies and releases large-capacity memory every time a frame of data is processed, resulting in performance degradation. The preset DMA cache area on the RISC-V hardware computing module of the present application can reduce the repeated application and release of memory, thereby improving the real-time frame rate of target detection and the overall performance of the system. In addition, considering that the image size output by the camera is fixed, the DMA cache area of the present application is set to a fixed size to avoid wasting memory. Step S2 realizes the mapping of virtual addresses to physical addresses, avoiding repeated mapping and memory reallocation during each frame of data processing. In this way, the address mapping can be reused, thereby reducing the overhead of memory management and improving memory access efficiency. In addition, the low-power feature of the RISC-V architecture development board enables the present method to run efficiently and continuously under limited energy conditions, reducing energy consumption. Furthermore, the flexible scalability of the RISC-V architecture development board ensures the high adaptability and upgradability of the present method, providing excellent performance and practicality for vehicle networking multi-scenario target detection. At the same time, the present application accelerates the landing application of RISC-V architecture AI products in the field of vehicle networking, further enriching its application scenarios in different fields and improving its portability and flexibility.

[0043] In contrast to step S4, the previous method reloads the target detection model when processing each frame of image. The parameter quantity of a general target detection model can reach millions or even tens of millions, which not only increases the processing time but also causes unnecessary memory occupation. In step S4 of the present method, the cache coherency protocol supported by the RISC-V hardware computing module ensures the data consistency of the cache between multi-core processors, avoiding the repeated loading of model parameters when processing each frame of image, effectively reducing memory occupation, and improving the overall response speed of the system to which the present method is applied.

[0044] In contrast to step 5, the previous method saves the model inference result to a file after the model inference is completed, then reads the model inference result from the file, and finally renders the bounding box result to the original image. This method not only increases the additional I / O operation but also causes processing delay. The present method directly processes the image data through a custom post-processing function after the model inference is completed, thereby reducing unnecessary I / O operations and improving the real-time performance and processing efficiency of the system.

[0045] In this example, the post-processing is to draw and render the bounding box contained in the perception information onto the corresponding image.

[0046] The acquired image data includes but is not limited to blind area obstacles, traffic signs, lane lines, etc. These are all objects of target detection.

[0047] In this example, before step S1, there is also step S0: the target detection module on the RISC-V hardware computing module calls the open function to obtain the input image data. The image data is the vehicle surrounding image data captured by the data acquisition module. Calling the open function to obtain the input image data avoids the additional overhead caused by using third-party libraries, because when obtaining the input image, the previous method usually obtains the camera screen by calling a third-party library (such as OpenCV), which causes additional overhead.

[0048] In this example, the preprocessing in step S3 includes scaling and filtering operations on the image data.

[0049] The RVV instruction provides a vectorized computing mode, and the preprocessing in step S3 uses vector load and store instructions to access image data, reducing data access overhead. In addition, the RVV instruction can simultaneously calculate the interpolation values of multiple target pixels, further shortening the operation time. In this example, the following main RVV instructions are used to implement the bilinear interpolation image scaling and 3x3 convolution filtering operations:

[0050] First, the “vsetvlmax_e8m1” instruction is used to determine the vector length of the current batch processing, so that the program can automatically adjust the number of parallel processing elements according to the actual hardware resources;

[0051] Second, the load instruction “vle8_v_i8m1” is used to load the 2x2 neighborhood pixels of the original image, providing basic data for bilinear interpolation;

[0052] Third, the multiplication instruction “vwmul_vx_i16m2” and the accumulation instruction “vadd_vv_i16m2” are used to calculate the weights of horizontal interpolation and vertical interpolation, and the intermediate results are processed by the bit-shifting instruction “vnclip_wx_i8m1”;

[0053] Fourth, the multiplication instruction “vwmul_vx_i16m2” and the accumulation instruction “vadd_vv_i16m2” are used to perform 3x3 convolution filtering operations on the calculated interpolation results;

[0054] Fifth, the storage instruction “vse8_v_i8m1” is used to store the final processing results to the output buffer.

[0055] The target detection model loaded in step S4 is the YOLOv5 model, but it is not limited thereto, and can also be other detection models.

[0056] In the model inference stage, the NPU (Neural Processing Unit) in the RISC-V hardware computing module is used to process multi-channel convolution and matrix multiplication in parallel. Compared with the traditional CPU inference method, the NPU is specially optimized for deep learning operations, and its parallel computing capability significantly improves the inference speed, optimizes the real-time performance of target detection, and further reduces the delay of model inference.

[0057] Preferably, steps S3-S5 are processed in parallel by the multi-computing core unit through the pipeline mechanism. In the prior art, the previous method is to use a serial execution method in the preprocessing, model inference and post-processing stages of image data processing, which cannot fully utilize the hardware resources and limits the overall processing speed. However, the present method uses a multi-computing core unit to process in parallel through a pipeline mechanism, and different image processing tasks of different frames can be executed at different processing stages at the same time, such as Figure 2 As shown in the figure, specifically, at the same time, the first frame is performing post-processing, the second frame is performing model inference, and the third frame is performing preprocessing; this structure can improve processing efficiency and obtain higher throughput when processing continuous multiple frames of images.

[0058] In this example, the Hart (Hardware Thread) in the RISC-V hardware computing module is used to perform collaborative work in a multi-core environment. The preprocessing, model inference and post-processing tasks are divided into multiple threads, each of which is processed by a Hart. The Harts can execute their respective work tasks simultaneously on different core units.

[0059] In this example, a core A is pre-allocated for preprocessing tasks. In core A, multiple Harts are allocated to process different image regions or processing steps in parallel. Hart 1 is responsible for image scaling, and Hart 2 is responsible for filtering;

[0060] A core B is allocated for model inference tasks. In core B, multiple Harts are allocated to execute different model layers or computing tasks. Hart 1 is responsible for convolution operations, Hart 2 is responsible for activation function calculations, Hart 3 is responsible for feature fusion, and Hart 4 is responsible for candidate box decoding and output calculations;

[0061] A core C is allocated for processing tasks. In core C, multiple Harts are allocated to perform different post-processing tasks. Hart 1 is responsible for drawing and rendering bounding boxes, and Hart 2 is responsible for result integration.

[0062] The scheduling mechanism of the RISC-V hardware computing module allows Harts to coordinate work through synchronization mechanisms such as semaphores, mutexes and barriers, ensuring smooth connection of tasks in each stage.

[0063] In this embodiment, after the image preprocessing is completed, the Harts responsible for preprocessing will send a signal through a semaphore or a barrier to inform the Harts responsible for model inference that they can start working; the model inference Harts read the preprocessed image data, perform convolution operations, activation function calculations, feature fusion, candidate box decoding and output calculations. When the model inference task is completed, the model inference Harts will pass the model inference result to the post-processing Harts through shared memory or message passing mechanism, and release the semaphore or reach the barrier to inform the post-processing Harts to start reading the model inference result and perform post-processing tasks.

[0064] The above task division and synchronization mechanism ensures that each stage of image processing can be smoothly connected, and there will be no delay due to resource contention or data inconsistency problems, thereby significantly improving the real-time performance of the overall system; this way not only optimizes the efficiency of each processing stage, but also improves the resource utilization of the overall system, thereby significantly improving the real-time frame rate performance of target detection.

[0065] In this example, the method further includes a step S6: the data communication module obtains the display information of the vehicle itself and shares the display information with other vehicles, and at the same time, the data communication module obtains the display information of other vehicles; the user interface interaction module arranged on the RISC-V hardware calculation module obtains and presents the display information of the vehicle itself and other vehicles.

[0066] Specifically, the data communication module shares the display information with other vehicles by using the Internet of Vehicles technology, preferably, the Internet of Vehicles technology adopts C-V2X technology based on cellular network to realize real-time exchange of data. Specifically, any one of V2V (Vehicle-to-Vehicle) communication or V2I (Vehicle-to-Infrastructure) communication can be selected to realize this function.

[0067] In the V2V communication mode, the on-board unit OBU of other vehicles encodes the display information detected by the vehicle into a specific data packet; secondly, the data packet is sent directly to the on-board unit OBU of the vehicle through C-V2X wireless communication technology;

[0068] In the V2I communication mode, the on-board unit OBU of other vehicles encodes the display information into a specific data packet, and transmits it to the nearest roadside unit RSU through C-V2X wireless communication technology; the roadside unit RSU analyzes the received information and forwards it to the on-board unit OBU of the vehicle.

[0069] Both V2V or V2I communication modes can ensure that the driver of the ego vehicle can receive and observe the display information of other vehicles identified in real time, and then make driving decisions in advance according to the road conditions;

[0070] Further, the user interaction module includes a vehicle-mounted display screen and a visualization interface software system, the vehicle-mounted display screen is used to display the visualization interface software system, and is used to display the real-time display information after target detection, which includes at least one of image information or point cloud information, to assist the driver to make timely decisions.

[0071] Preferably, the visualization interface software system is written using the cross-platform C++ application development framework Qt5, and is transplanted to the RISC-V hardware computing module for compilation and running.

[0072] The visualization interface software system includes a display information acquisition function and a network debugging assistant function, the display information acquisition function is used to display the display information of other vehicles received by the ego vehicle and the display information of the ego vehicle, and the network debugging assistant function is used to test the communication connection state between the Internet of Vehicles devices to ensure normal communication.

[0073] The method can cover the following typical application scenarios:

[0074] Blind area obstacle detection: In the intersection scene, the side area is usually a visual blind area, and pedestrians or vehicles suddenly emerging from the area are extremely easy to cause traffic accidents; in this scenario, obstacles such as pedestrians and vehicles around the environment are identified by sensors such as cameras, and the information is shared with other vehicles to prevent traffic collision accidents such as "ghost probe";

[0075] Traffic sign recognition: there is a large truck blocking the view in front of the vehicle, making it impossible for the vehicle to observe the traffic light status, and it may blindly follow the front vehicle and misjudge the red light; in this scenario, the front vehicle detects and identifies the current traffic signal status and other road signs, and shares them with the rear vehicle to make correct decisions;

[0076] Lane line detection: in the case of limited vision such as curves and tunnel entrances, the driver may have difficulty clearly identifying lane markings, which can easily cause the vehicle to deviate from the correct lane, thereby causing unintentional lane changing accidents; in this scenario, lane markings are detected in real time by sensors such as cameras to ensure that the vehicle stays in the correct lane, effectively preventing unintentional lane changing.

[0077] The openness of RISC-V allows it to be implemented on a variety of architecture platforms, such as processor cores, microcontrollers, and embedded devices. This high degree of portability enables software developers to write architecture-independent code and easily port it between different RISC-V architectures. The RISC-V architecture is highly customizable due to its modular design, and the necessary extended instruction sets can be added according to specific application scenarios, thereby optimizing the execution efficiency of the target detection algorithm. At the same time, RISC-V can significantly reduce power consumption while maintaining high performance, which is especially important for long-term operation and battery-powered in-vehicle equipment. In addition, the open standard instruction set architecture (ISA) makes the design and manufacturing costs of RISC-V chips relatively low.

[0078] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the present invention. Anyone skilled in the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by one of ordinary skill in the art without departing from the spirit and technical concepts disclosed herein are intended to be covered by the claims of the present invention.

Claims

1. A multi-scenario target detection method for Internet of Vehicles based on RISC-V architecture, characterized in that: include: Step S1: storing the acquired image data in a DMA buffer area preset on the RISC-V hardware computing module, where the DMA buffer area is set to a fixed size; Step S2: using the mmap mechanism to map the image data in the DMA buffer to the address space of the process on the RISC-V hardware computing module; Step S3: pre-processing the image data using the RVV instruction; Step S4: After loading the object detection model parameters for the first time, the pre-processed image data is cached into the heterogeneous shared memory using the cache coherence protocol supported by the RISC-V hardware computing module, and model inference is performed to obtain perception information. Step S5: Post-process the model inference results through a custom post-processing function and obtain display information; Step S6: Each vehicle shares its own display information with other vehicles, and simultaneously presents the display information from itself and other vehicles; Wherein, multiple computing core units are used to process steps S3 to S5 in parallel through a pipeline mechanism; A core A is pre-allocated to perform pre-processing tasks. Within core A, multiple Harts are assigned to process different image regions or processing steps in parallel. Hart1 is responsible for image scaling, and Hart2 is responsible for filtering. Assign a core B to perform model inference tasks. Within core B, assign multiple Harts to perform different model layers or calculation tasks. Hart1 is responsible for convolution operations, Hart2 is responsible for activation function calculations, Hart3 is responsible for feature fusion, and Hart4 is responsible for candidate box decoding and output calculations. Assign a core C to perform post-processing tasks. Within core C, assign multiple Harts to perform different post-processing tasks. Hart1 is responsible for drawing the rendering bounding box, and Hart2 is responsible for result integration.

2. The method according to claim 1, characterized in that The loaded target detection model is the YOLOv5 model.

3. The method according to claim 1, characterized in that The pre-processing process in step S3 includes scaling and filtering operations on the image data.

4. The method according to claim 1, wherein During the pre-processing in step S3, the image data is accessed using vector load and store instructions.

5. The method according to claim 1, wherein In the model inference in step S4, the NPU in the RISC-V hardware computing module is used to process multi-channel convolution and matrix multiplication in parallel.

6. The method according to claim 1, characterized in that The post-processing includes rendering the bounding box included in the perception information onto the corresponding image.

7. A multi-scenario target detection system for Internet of Vehicles based on RISC-V architecture, characterized by: include: A data acquisition module, used for capturing image data of the surrounding area of ​​the vehicle; The object detection module is used to process the captured image data to obtain display information; Data communication module, used to obtain the display information of its own vehicle and share it with other vehicles, and at the same time obtain the display information from other vehicles; A user interaction module is used to obtain and present display information from the vehicle itself and other vehicles; The target detection module and the user interface interaction module are both set on the RISC-V hardware computing module; The target detection module performs target detection according to the method according to any one of claims 1 to 6.

Citation Information

Patent Citations

  • Camera-assisted Internet of Vehicles wireless communication method based on privacy protection principle

    CN111967318A

  • RISC-V-based TinyML target detection acceleration system and method, and storage medium

    CN114529797A