ADAS data processing method and device for computer vision and medium

Abstract

The invention discloses an ADAS data processing method and device for computer vision and a medium, and relates to the technical field of computer vision. The method comprises the following steps: establishing an error processing module and an ADAS data analysis module; a driving environment real-time image set is obtained, a new period is triggered to start, and error detection and updating processing are carried out on the target ADAS system; identifying and analyzing the driving environment real-time image set to obtain an obstacle multi-dimensional feature data set; performing operation data dynamic analysis on the information of each obstacle to obtain an obstacle dynamic operation parameter set; and carrying out obstacle risk assessment, determining obstacle risk early warning grade information, and carrying out auxiliary driving early warning on the target ADAS system through the obstacle risk early warning grade information and the obstacle dynamic operation parameter set. The technical problem that in the prior art, an ADAS system cannot accurately recognize obstacles in a complex environment is solved, and the technical effect of improving the obstacle recognition accuracy is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision, and in particular to a computer vision ADAS data processing method, device and medium. Background Art

[0002] With the continuous development of autonomous driving technology and advanced driver assistance systems (ADAS), computer vision technology has been widely used in the field of automobile driving safety. The ADAS system uses cameras, sensors and other devices to perceive the vehicle's surrounding environment in real time, and provides auxiliary decision-making to the driver based on data analysis. However, in the existing technology, due to the complex environment and a wide variety of obstacles, the ADAS system is prone to problems such as low recognition accuracy, inaccurate dynamic analysis, and untimely risk assessment when performing real-time recognition of the driving environment, resulting in unsatisfactory system warning effects and affecting the safety and reliability of assisted driving. Therefore, how to improve the ADAS system's recognition accuracy of obstacles, the accuracy of dynamic analysis, and the timeliness of risk assessment in complex environments has become an important technical problem that needs to be solved in this field. Summary of the invention

[0003] The present application provides a computer vision ADAS data processing method, device and medium, which solves the technical problem in the prior art that ADAS systems cannot accurately identify obstacles in complex environments.

[0004] In view of the above problems, the present application provides a computer vision ADAS data processing method, device and medium.

[0005] In a first aspect of the present application, a computer vision ADAS data processing method is provided, the method comprising: An error handling module and an ADAS data parsing module are established; a real-time image set of the driving environment is obtained by capturing the camera, a new cycle is triggered, and the target ADAS system is subjected to error detection and update processing by the error handling module; the real-time image set of the driving environment is identified and parsed based on the ADAS data parsing module to obtain a multi-dimensional feature data set of obstacles; the operation data of each obstacle information in the multi-dimensional feature data set of obstacles is dynamically analyzed to obtain a dynamic operation parameter set of obstacles; obstacle risk assessment is performed based on the dynamic operation parameter set of obstacles to determine obstacle risk warning level information, and auxiliary driving warning is performed on the target ADAS system through the obstacle risk warning level information and the obstacle dynamic operation parameter set.

[0006] Furthermore, according to the ADAS data parsing module, an image preprocessing module is obtained; noise recognition and filtering processing are performed on the driving environment real-time image set by the image preprocessing module to obtain a denoised driving environment real-time image set; the denoised driving environment real-time image set is mapped to an image grayscale distribution coordinate system for grayscale conversion and equalization enhancement to obtain a usable driving environment real-time image set; obstacle recognition and feature analysis are performed on the usable driving environment real-time image set to obtain a multi-dimensional feature data set of obstacles.

[0007] Furthermore, the driving environment real-time image set is semantically segmented using a driving semantic segmentation model to obtain a driving environment semantic segmentation result; obstacles are identified and numbered based on the driving environment semantic segmentation result to obtain an obstacle target set and an obstacle identification code set; obstacle feature factor information is obtained, and the obstacle feature factor information includes obstacle type, geometric shape, size, and quantity and position; based on the obstacle feature factor information and according to the obstacle identification code set, feature analysis and extraction are performed on the obstacle target set in turn to obtain the obstacle multi-dimensional feature data set.

[0008] Furthermore, a driving environment image set is acquired, and image segmentation and labeling are performed on the driving environment image set according to driving environment safety standards to obtain an image labeling segmentation result set; semantic segmentation training is performed based on the image labeling segmentation result set to obtain a driving environment semantic segmentation model, and the driving environment semantic segmentation model includes a network encoder and a network decoder; the real-time driving environment image set is input into the network encoder to obtain a multi-scale feature set of the driving environment image; segmentation prediction is performed on the multi-scale feature set of the driving environment image based on the network decoder, and the driving environment semantic segmentation result is output.

[0009] Furthermore, the obstacle multidimensional feature data set is arranged and integrated according to the time series information to obtain the obstacle multidimensional feature time series data set; the obstacle multidimensional feature time series data set is diverted according to the obstacle identification code set to obtain the obstacle time series feature identification data set; the speed change calculation and the distance change calculation are performed on the obstacle time series feature identification data set respectively to generate an obstacle speed change data set and an obstacle distance change data set; the obstacle speed change data set and the obstacle distance change data set are dynamically analyzed to obtain the obstacle dynamic operation parameter set.

[0010] Furthermore, an obstacle operation data set is collected and acquired, and the operation mode of the obstacle operation data set is annotated to obtain an obstacle operation sample set; a recurrent neural network structure is used to perform annotated training on the obstacle operation sample set to obtain an obstacle trajectory prediction model and an obstacle behavior recognition model; the obstacle trajectory prediction model and the obstacle behavior recognition model are parallelly integrated to generate an obstacle operation mode analysis model; based on the obstacle operation mode analysis model, an operation dynamic analysis is performed on the obstacle speed change data set and the obstacle distance change data set to obtain the obstacle dynamic operation parameter set.

[0011] Furthermore, an obstacle risk assessment model library is constructed, wherein the obstacle risk assessment model library includes a set of obstacle risk assessment models of various obstacle multidimensional feature types; a target obstacle risk assessment model is obtained by matching and calling the obstacle multidimensional feature data set and the obstacle risk assessment model library; and a risk level assessment is performed on the obstacle dynamic operation parameter set based on the target obstacle risk assessment model to determine the obstacle risk warning level information.

[0012] Furthermore, a driving warning mechanism is triggered according to the obstacle risk warning level information, and an obstacle risk level warning method is determined based on the driving warning mechanism; a driving obstacle avoidance strategy is analyzed based on the obstacle dynamic operation parameter set to obtain assisted driving strategy parameters; and an assisted driving warning is performed on the target ADAS system based on the obstacle risk level warning method and the assisted driving strategy parameters.

[0013] The second aspect of the present application provides an electronic device, comprising: a memory for storing executable instructions; and a processor for executing the executable instructions stored in the memory to implement a computer vision ADAS data processing method provided in the present application.

[0014] The third aspect of the present application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements a computer vision ADAS data processing method provided in the present application.

[0015] One or more technical solutions provided in this application have at least the following technical effects or advantages: First, an error handling module and an ADAS data parsing module are established. Next, a real-time image set of the driving environment is obtained through camera capture, a new cycle is triggered, and the target ADAS system is subjected to error detection and update processing through the error handling module. Then, the real-time image set of the driving environment is identified and parsed based on the ADAS data parsing module to obtain a multi-dimensional feature data set of obstacles. Further. The operation data of each obstacle information in the multi-dimensional feature data set of obstacles is dynamically analyzed to obtain a dynamic operation parameter set of obstacles. Finally, an obstacle risk assessment is performed based on the dynamic operation parameter set of obstacles, the obstacle risk warning level information is determined, and the target ADAS system is given an assisted driving warning through the obstacle risk warning level information and the dynamic operation parameter set of obstacles. The technical problem of inaccurate obstacle recognition in complex environments in the prior art is solved, and the technical effect of improving the accuracy of obstacle recognition is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0017] Figure 1 A flowchart of a computer vision ADAS data processing method provided in an embodiment of the present application; Figure 2 A schematic diagram of a flow chart of obtaining a multi-dimensional feature data set of obstacles in a computer vision ADAS data processing method provided in an embodiment of the present application; Figure 3 This is a schematic diagram of the structure of an exemplary electronic device of the present application.

[0018] Description of the reference numerals: processor 21 , memory 22 , input device 23 , output device 24 . DETAILED DESCRIPTION

[0019] The present application solves the technical problem of inaccurate obstacle recognition by ADAS systems in complex environments in the prior art by providing a computer vision ADAS data processing method, device and medium.

[0020] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0021] It should be noted that the terms "including" and "having" are intended to cover non-exclusive inclusions. For example, a process, method, system, product or server that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or modules that are not explicitly listed or inherent to these processes, methods, products or devices.

[0022] Embodiment 1, as Figure 1 As shown, the present application provides a computer vision ADAS data processing method, wherein the method includes: Establish error handling module and ADAS data parsing module.

[0023] By establishing an error handling module and an ADAS data analysis module, efficient processing of advanced driver assistance system (ADAS) data and real-time detection and repair of system errors are achieved. Among them, the error handling module is responsible for identifying, monitoring and updating the abnormalities or errors that occur during the operation of the system to ensure the stability and accuracy of the system; while the ADAS data analysis module is responsible for identifying and analyzing the real-time image data of the driving environment, providing data support for obstacle feature extraction, dynamic analysis and risk assessment.

[0024] A real-time image set of the driving environment is captured by a camera, a new cycle is triggered, and the error detection and update processing of the target ADAS system is performed through the error processing module.

[0025] The real-time image set of the driving environment is captured by the camera to trigger the start of a new working cycle, and the error handling module is used to perform error detection and update processing on the target ADAS system.

[0026] The error handling module is used to identify errors in the ADAS system. For example, an error detection function named MvAdasErrTask is designed to periodically detect and mark the error status in the ADAS system. In the error detection function, a static variable delayties is declared to record the time delay. The value of this variable will increase with the periodic call of the function, thereby ensuring that the error mark update operation can be accurately executed each time it is called. When delayties=0, the system determines that the current period is the beginning of a new cycle. At this time, the two flags MV_ADAS_END_FLAG and MV_ADAS_EXIST_FLAG will be reset to 0, indicating that there is no error in the current cycle by default, and the system is initialized to a normal state. During operation, when delayties≥50, the system starts the error detection mechanism and updates the error status of the current target ADAS system. Specifically, the system makes a judgment based on the status of the error flag mv_adas_data.err: if MV_ADAS_EXIST_FLAG is true, it means that the current system has not detected an error. At this time, the error flag is updated to 0, indicating that the ADAS system is in normal status; if MV_ADAS_EXIST_FLAG is false, the error flag is updated to 1, indicating that there is an error in the system and subsequent processing is required. At the same time, in order to ensure that the delay variable delayties operates within a reasonable range, the system performs periodic modulo operations on its value, so that it increases cyclically between 0 and 54, so as to re-detect errors and update the status in the next cycle. Through the above mechanism, the system can realize real-time error detection and status reset of the ADAS system in each cycle, ensure accurate error marking and stable operation of the system, and provide reliable data support and system guarantee for subsequent driving environment image analysis and assisted driving decision-making.

[0027] For example, in order to handle the periodic tasks of the ADAS system, a function named MvAdasPeriodTask is designed. The main function of this function is to ensure the stability and continuity of error detection and dynamic data processing in each cycle of the system. In this function, the MvAdasErrTask function is first called to periodically detect errors in the ADAS system, and the system's error flag RUN_DATA_BUF.f_adas_err and error counter are updated according to the detection results, so as to grasp the error state of the system in real time and ensure that the fault can be identified and recorded in time. Subsequently, the mvXspeedfilter function is called to dynamically process the speed and distance data of the obstacle, and the running state of the obstacle is analyzed by calculating the changing trend of the obstacle X speed and distance, and possible safety hazards are identified. For different error states, the system will further reset or update the relevant variables of ADAS to ensure that the running logic of the periodic task is clear and the execution is accurate. For example, when the system detects an error, the dynamic data cache is reset, the historical obstacle speed and position data are cleared, and the error state is prevented from affecting subsequent calculations; when the system is running normally, the relevant data is continuously updated to provide the latest running state for the execution of the next cycle of tasks. Through the above operations, the MvAdasPeriodTask function can realize the periodic maintenance and dynamic data management of the ADAS system, providing stable and reliable system support for subsequent obstacle identification, risk assessment and assisted driving warning.

[0028] For example, in order to parse the data received by ADAS and update the obstacle information, a function named MvAdasWhileTask is designed to process and update the information of the target obstacle in real time. First, the function checks the status of MV_ADAS_END_FLAG. If the flag is 1, it means that ADAS has received the latest data in the current cycle, and the system will start the data parsing process. Then, the system updates the number of obstacles and their multi-dimensional feature information, including key information such as obstacle ID, position, speed and danger signs, based on the real-time data obtained by the ADAS parsing module. During the parsing process, the system processes the attributes of each obstacle according to the specific algorithm, analyzes its dynamic state (such as movement speed and direction changes), and judges the degree of danger of the obstacle in combination with the preset conditions. At the same time, the system records and stores the relevant attribute data of the obstacle, such as the width and height of the obstacle, and the time interval between the Y axis and the X axis, to ensure that accurate dynamic analysis and status monitoring can be carried out later. Finally, the system updates the data structure to ensure that the latest status of each obstacle is synchronized in time. During the update process, the system will decide whether to further update the target obstacle information based on the evaluation results of the danger signs. In particular, when the dynamic characteristics of the obstacle change significantly or there is a potential safety risk, the system will give priority to updating the obstacle's identification and attribute data to provide accurate data support for subsequent risk assessment and assisted driving decisions. Through the design and execution of the MvAdasWhileTask function, the system can efficiently parse ADAS data, dynamically update obstacle information, and ensure the accuracy and real-time status of the target obstacle.

[0029] For example, in order to execute the while task of ADAS, a function named mvWhileTask is designed to manage the continuity and real-time performance of the ADAS data processing process. First, the function checks the status of the ADAS end flag (MV_ADAS_END_FLAG). If the value of the flag is 1, it means that the data processing task of the current ADAS system has been completed and there is new data to be processed. At this time, the function will call MvAdasWhileTask to start parsing and processing the data currently received by ADAS, including updating the number, position, speed, hazard signs and other dynamic feature information of obstacles, and making judgments on the degree of danger and synchronous updates of the data structure. After calling MvAdasWhileTask to complete the data parsing and update tasks, the system will clear the MV_ADAS_END_FLAG flag to indicate that the current data processing cycle has been completed, and ensure that the flag status is reset to prepare for the next new data processing task. Through such logical design, the mvWhileTask function realizes the periodic scheduling and real-time processing of ADAS data, ensuring that the ADAS system can timely parse and update obstacle information in each working cycle, providing stable and accurate data support for assisted driving.

[0030] Based on the ADAS data analysis module, the real-time image set of the driving environment is identified and analyzed to obtain a multi-dimensional feature data set of obstacles.

[0031] The ADAS data analysis module identifies and analyzes the real-time image set to obtain a multi-dimensional feature data set of obstacles, providing data support for subsequent analysis. The multi-dimensional feature data set of obstacles includes multiple parameters such as the location, size, shape, motion trajectory, speed, etc. of the obstacles, ensuring that the system can comprehensively and accurately identify the feature information of different obstacles.

[0032] Based on the ADAS data analysis module, the real-time image set of the driving environment is identified and analyzed. For example, in order to process the speed and distance data of obstacles, a function named mvXspeedfilter is designed to dynamically monitor the motion state and position information of obstacles. At the beginning of the mvXspeedfilter function, relevant variables are defined and initialized, including a buffer for storing old values ​​and indexes, which is used to record the historical speed and position information of obstacles. The system monitors the acceleration and deceleration changes of obstacles by calculating the difference between the current X speed of the obstacle and the last recorded X speed, and performs the same processing on the X distance of the obstacle.

[0033] When it is detected that the last X speed of the obstacle is greater than 1 (indicating that the obstacle is moving rapidly and there may be safety risks), the system will further calculate the difference between the current X speed and the last X speed to evaluate the dynamic acceleration and deceleration of the obstacle; if the X speed is small, the system will set the speed difference to 0 to avoid meaningless calculations at low speed or at rest; after the calculation is completed, the current obstacle's X speed and speed difference are stored in the buffer to provide basic data for the next call.

[0034] Next, the system checks the obstacle ID and location information. If the obstacle ID or location changes, the system will treat the current obstacle as a new obstacle or an obstacle that no longer exists, clear the pre-stored X speed and speed difference information, and reset the X speed difference to 0 to avoid subsequent misjudgment. At the same time, in order to adapt to changes in vehicle speed, the system sets a dynamic distance difference threshold (2 plus 0.3 times the current vehicle speed). If the X distance difference of the current obstacle exceeds this threshold, the data is reset and the buffer is cleared.

[0035] In addition, the system identifies safety hazards by comparing the speed and distance difference of obstacles. If the speed-related distance difference of an obstacle exceeds the threshold, it indicates that the relative speed of the obstacle has changed too much, which may cause risks, and the system marks cpspeed_err as 1. At the same time, the system stores the current obstacle position in the cache and dynamically updates the index based on the usage of the cache: if the index is less than 4, the current data is directly stored; if the index is full, the cached data is moved forward and the new data is stored in the last position.

[0036] To ensure data reliability, the system also checks whether the data in the cache is full. When there are 5 valid obstacle positions in the cache, the distance difference dist_diff of the obstacle is calculated; if the cache is not full, the distance difference is set to 99, indicating that the data is unreliable. In addition, the system further checks whether there is an error in the ADAS system: if mv_adas_data.err is detected to be true, data_err is directly marked as 1; if the 5 historical distance values ​​in the cache are equal, the system believes that there is no valid change in the obstacle information, and data_err is also marked as 1, otherwise data_err is set to 0, indicating that the data is normal and the system is running correctly.

[0037] Furthermore, if Figure 2 As shown, the step of obtaining a multi-dimensional feature dataset of obstacles includes: According to the ADAS data parsing module, an image preprocessing module is obtained; noise recognition and filtering processing are performed on the driving environment real-time image set by the image preprocessing module to obtain a denoised driving environment real-time image set; the denoised driving environment real-time image set is mapped to an image grayscale distribution coordinate system for grayscale conversion and equalization enhancement to obtain a usable driving environment real-time image set; obstacle recognition and feature analysis are performed on the usable driving environment real-time image set to obtain a multi-dimensional feature data set of obstacles.

[0038] Specifically, the ADAS data analysis module calls the image preprocessing module to process the real-time image set of the driving environment, uses the image preprocessing module to identify the noise in the real-time image set, and uses the filtering method to perform denoising to obtain the denoised real-time image set of the driving environment; the denoised real-time image set of the driving environment is mapped to the image grayscale distribution coordinate system, the grayscale conversion operation is performed on the image, and the brightness and contrast of the image are optimized through the equalization enhancement technology to generate a usable real-time image set of the driving environment; based on the usable real-time image set of the driving environment, obstacles are automatically identified, and the characteristic information of the obstacles is analyzed to generate a multi-dimensional feature data set of obstacles containing multi-dimensional information such as position, size, shape, and motion state, so as to provide accurate data support for subsequent dynamic analysis and risk assessment.

[0039] Exemplarily, a filtering method is used to denoise a set of real-time images of a driving environment, specifically including: calling an image preprocessing module through an ADAS data parsing module to identify the noise in the collected real-time image set of the driving environment, the source of which may include electronic noise generated by sensor equipment, random noise caused by changes in ambient light, and dynamic blur noise generated during motion, etc.; using a noise detection algorithm to classify and locate the type of noise in the image, for example, by analyzing the distribution of pixel grayscale values ​​and image edge texture features, distinguishing Gaussian noise, salt and pepper noise, or periodic interference noise; according to the detection results, selecting a suitable filtering method to denoise the image. Specifically, for Gaussian noise, a Gaussian filtering method is used to smooth noise points by calculating a weighted average within the pixel neighborhood of the image while retaining the detailed information of the image; for salt and pepper noise, a median filter is applied to remove abnormal highlights or low highlights by calculating the median within a local window; for periodic interference noise, a frequency domain filtering technique is used to convert the image to the frequency domain through Fourier transform, and detect and remove noise components of specific frequencies. During the filtering process, filter parameters (such as filter window size or weight coefficient) are dynamically adjusted to balance the denoising effect and image detail retention requirements, ensuring that the denoised image has high clarity without losing key driving environment information. After filtering, the generated denoised driving environment real-time image set significantly reduces noise interference compared to the original image, while retaining complete scene information, providing a high-quality data foundation for subsequent image enhancement, obstacle recognition and feature analysis.

[0040] Exemplarily, mapping the denoised real-time image set of the driving environment to the image grayscale distribution coordinate system specifically includes: loading the denoised image data into the image processing module, determining the current grayscale distribution range of the image by analyzing the pixel distribution of the image, generating the grayscale histogram of the image by pixel intensity statistics, and intuitively reflecting the distribution of light and dark areas in the image; mapping the denoised image to the standard grayscale distribution coordinate system, adjusting the grayscale value of the image to a preset range, such as a standardized grayscale range of 0 to 255, by linear stretching or nonlinear mapping, for subsequent processing. After the grayscale mapping is completed, a grayscale conversion operation is performed on the image, and the specific method includes reallocating the grayscale value to each pixel point of the image according to the mapping function. For example, through the contrast enhancement algorithm, the contrast between the dark and bright areas in the image can be enhanced, so that the image more clearly presents key information of the driving environment, such as the edge, texture and shadow features of the obstacle. In order to further optimize the image quality, the equalization enhancement technology is applied to adjust the brightness and contrast of the image; the distribution of the image grayscale value is redistributed through the histogram equalization method, so that the grayscale values ​​originally concentrated in a certain range are expanded to the entire grayscale range, thereby enhancing the details of the bright and dark parts of the image; at the same time, the equalization parameters can be dynamically adjusted to optimize the enhancement intensity according to the real-time conditions of the driving environment (such as too bright or too dark light) to avoid problems such as over-enhancement or loss of details. Ultimately, the generated set of real-time images of the available driving environment not only has higher clarity, but also significantly improves brightness and contrast, making obstacles and road information in the image more prominent, providing a high-quality data foundation for subsequent obstacle recognition and feature analysis.

[0041] Furthermore, obtaining the obstacle multi-dimensional feature data set includes: The driving environment real-time image set is semantically segmented by using a driving semantic segmentation model to obtain a driving environment semantic segmentation result; obstacles are identified and numbered based on the driving environment semantic segmentation result to obtain an obstacle target set and an obstacle identification code set; obstacle feature factor information is obtained, and the obstacle feature factor information includes obstacle type, geometric shape, size, and quantity position; based on the obstacle feature factor information and according to the obstacle identification code set, feature analysis and extraction are performed on the obstacle target set in turn to obtain the obstacle multi-dimensional feature data set.

[0042] Specifically, the driving environment real-time image set is semantically segmented using the driving semantic segmentation model to generate a driving environment semantic segmentation result, which contains the semantic category information of each area in the image, such as roads, signs, vehicles, pedestrians, obstacles, etc. Based on the driving environment semantic segmentation result, the obstacles in the image are identified, and a unique number identification is generated for each obstacle to obtain an obstacle target set and a corresponding obstacle identification code set, and establish an associated index for subsequent feature analysis; For each obstacle in the obstacle target set, its key feature factor information is extracted, including but not limited to obstacle type (such as vehicle, pedestrian, animal, etc.), geometric shape (such as rectangle, circle), size (such as length, height, area) and quantity and position information, to form a basic feature information set of the obstacle; Based on the extracted obstacle feature factor information, the obstacle target set is feature parsed and extracted one by one in the order of the obstacle identification code set, and finally a complete obstacle multi-dimensional feature data set is obtained, which contains the geometric features, semantic attributes, spatial location information and dynamic behavior parameters of the obstacles, providing high-quality input for subsequent dynamic analysis and risk assessment.

[0043] Furthermore, obtaining the driving environment semantic segmentation result includes: Acquire a driving environment image set, perform image segmentation and labeling on the driving environment image set according to driving environment safety standards, and obtain an image labeling segmentation result set; perform semantic segmentation training based on the image labeling segmentation result set to obtain a driving environment semantic segmentation model, the driving environment semantic segmentation model includes a network encoder and a network decoder; input the real-time driving environment image set into the network encoder to obtain a multi-scale feature set of the driving environment image; perform segmentation prediction on the multi-scale feature set of the driving environment image based on the network decoder, and output the driving environment semantic segmentation result.

[0044] Specifically, a driving environment image set is collected and acquired, and semantic annotation is performed on the driving environment image set in accordance with driving environment safety standards. Category labels are assigned to each pixel in the image, such as road, vehicle, pedestrian, etc., and the image is segmented at the pixel level based on the annotation results to generate an image labeled segmentation result set; the image labeled segmentation result set is used to train the semantic segmentation model, and a segmentation architecture including a network encoder and a network decoder is constructed during the training process, wherein the network encoder is responsible for extracting multi-scale feature information of the driving environment image, and the network decoder is responsible for spatial restoration and segmentation prediction of the encoded features; after the training is completed, the real-time driving environment image set is input into the trained semantic segmentation model, and the multi-scale feature set is first extracted through the network encoder, and then the extracted feature set is predicted based on the network decoder to generate a driving environment semantic segmentation result, which annotates the semantic category and segmentation boundary of each pixel, providing high-quality data support for obstacle recognition and feature analysis.

[0045] Exemplarily, the semantic segmentation model is trained using the image labeled segmentation result set, specifically including: using the generated image labeled segmentation result set as training data, the result set contains the semantic category label of each pixel in the driving environment image, such as roads, vehicles, pedestrians, buildings, etc. These pixel-level annotation information provides a rich supervision signal for the model, ensuring that the model can learn accurate category division and semantic features from the data; during the training process, a semantic segmentation model architecture based on a network encoder and a network decoder is designed and constructed, the network encoder is responsible for extracting multi-scale feature information from the driving environment image, extracting local and global semantic features of the image through layers of convolutional neural networks, and compressing the image spatial resolution by gradually downsampling, thereby obtaining a compact and efficient multi-scale feature representation, which can describe the complex scene relationships in the image, including the boundary characteristics and spatial distribution of different categories; the network decoder is responsible for gradually restoring the multi-scale features extracted by the encoder, restoring the spatial resolution of the image through upsampling or deconvolution operations, and mapping the semantic information back to the pixel-level resolution, so that the output result is consistent with the input image size. By continuously adjusting the model parameters, the semantic segmentation model gradually learns the complex semantic relationships and category distribution in the driving environment, and finally obtains a high-precision semantic segmentation model of the driving environment. The model can classify each pixel in the driving environment image and output segmentation results containing semantic categories, providing accurate basic data support for subsequent obstacle recognition and dynamic analysis.

[0046] The operation data of each obstacle information in the obstacle multi-dimensional feature data set is dynamically analyzed to obtain an obstacle dynamic operation parameter set.

[0047] The basic information of each obstacle is extracted from the obstacle multi-dimensional feature data set, including key features such as position, size, geometric shape, and motion state. The operation data is dynamically analyzed to obtain the dynamic operation parameter set of the obstacle.

[0048] Furthermore, obtaining the obstacle dynamic operation parameter set includes: The obstacle multidimensional feature data set is arranged and integrated according to the time series information to obtain the obstacle multidimensional feature time series data set; the obstacle multidimensional feature time series data set is shunted according to the obstacle identification code set to obtain the obstacle time series feature identification data set; the speed change calculation and the distance change calculation are respectively performed on the obstacle time series feature identification data set to generate an obstacle speed change data set and an obstacle distance change data set; the obstacle speed change data set and the obstacle distance change data set are dynamically analyzed to obtain the obstacle dynamic operation parameter set.

[0049] Specifically, the obstacle multi-dimensional feature data sets are arranged and integrated in chronological order to generate an obstacle multi-dimensional feature time series data set to ensure that the dynamic analysis data has continuity and time logic; then, based on the obstacle identification code set, the integrated time series data set is diverted according to the identification code to generate an independent obstacle time series feature identification data set for each obstacle, which contains the dynamic feature information of the obstacle at different times; then, for each obstacle time series feature identification data set, its speed change and distance change with other targets are calculated respectively to generate an obstacle speed change data set and an obstacle distance change data set; finally, by comprehensively analyzing the speed change data set and the distance change data set, the dynamic characteristics of the obstacle are predicted and the behavior pattern is analyzed, and finally the obstacle dynamic operation parameter set is generated to provide accurate data support for subsequent risk assessment and assisted driving warning.

[0050] Furthermore, obtaining the obstacle dynamic operation parameter set includes: An obstacle operation data set is collected and acquired, and an operation mode is annotated on the obstacle operation data set to obtain an obstacle operation sample set; a recurrent neural network structure is used to annotate and train the obstacle operation sample set to obtain an obstacle trajectory prediction model and an obstacle behavior recognition model; the obstacle trajectory prediction model and the obstacle behavior recognition model are connected in parallel to fuse to generate an obstacle operation mode analysis model; and based on the obstacle operation mode analysis model, an operation dynamic analysis is performed on the obstacle speed change data set and the obstacle distance change data set to obtain the obstacle dynamic operation parameter set.

[0051] Specifically, the obstacle operation data set is collected and acquired, including the dynamic characteristic information of the obstacle such as speed, position, acceleration, etc. in different time periods, and the operation mode of the obstacle operation data set is annotated, such as annotating the behaviors such as stillness, uniform motion, acceleration, deceleration, and turning, to generate an obstacle operation sample set; then, the obstacle operation sample set is trained using a recurrent neural network (RNN) structure, and an obstacle trajectory prediction model and an obstacle behavior recognition model are constructed and obtained respectively; then, the obstacle trajectory prediction model and the obstacle behavior recognition model are fused in parallel to generate a unified obstacle operation mode analysis model for comprehensive analysis of the obstacle operation trajectory and behavior mode; finally, the obstacle operation mode analysis model is used to perform operation dynamic analysis on the obstacle speed change data set and the obstacle distance change data set, and the real-time trajectory, motion trend, behavior mode and other information of the obstacle are extracted, and finally a complete obstacle dynamic operation parameter set is obtained to provide accurate dynamic data support for the assisted driving system.

[0052] An obstacle risk assessment is performed based on the obstacle dynamic operation parameter set, obstacle risk warning level information is determined, and an assisted driving warning is performed on the target ADAS system through the obstacle risk warning level information and the obstacle dynamic operation parameter set.

[0053] The potential collision risk value of the obstacle is calculated by using the real-time trajectory, speed, acceleration and motion trend information in the obstacle dynamic operation parameter set, combined with the relative position and distance change data between the obstacle and the vehicle; according to the risk value and the preset risk assessment standard, the obstacles are divided into different risk levels, such as low risk, medium risk and high risk, and the corresponding obstacle risk warning level information is generated; then, the obstacle risk warning level information and the dynamic operation parameter set are combined to provide assisted driving warning to the target ADAS system. The warning information is output in real time through the vehicle's interactive interface or voice prompts, including risk level, obstacle type, location and recommended avoidance measures (such as deceleration, braking or lane change).

[0054] Furthermore, the determining of obstacle risk warning level information includes: An obstacle risk assessment model library is constructed, wherein the obstacle risk assessment model library includes a set of obstacle risk assessment models of various obstacle multidimensional feature types; a target obstacle risk assessment model is obtained by matching and calling the obstacle multidimensional feature data set and the obstacle risk assessment model library; and a risk level assessment is performed on the obstacle dynamic operation parameter set based on the target obstacle risk assessment model to determine the obstacle risk warning level information.

[0055] Preferably, an obstacle risk assessment model library is constructed, which contains a set of risk assessment models for different types of multi-dimensional features of obstacles, such as assessment models for specific categories such as static obstacles, dynamic obstacles, vehicles, pedestrians, etc., each model defines exclusive risk assessment logic according to obstacle characteristics and motion patterns; based on the type, geometric shape and motion feature information in the obstacle multi-dimensional feature data set, a matching call is made in the model library to select the target obstacle risk assessment model that best matches the current obstacle characteristics; the target obstacle risk assessment model is used to comprehensively evaluate the speed, acceleration, distance change and other data in the obstacle dynamic operation parameter set, and the potential risk value of the obstacle is calculated; the risk value is combined with the preset level classification standard (such as low, medium and high risk levels) to finally determine the risk warning level information of the obstacle.

[0056] Furthermore, the visual driving warning for the target ADAS system using the obstacle risk warning level information includes: A driving warning mechanism is triggered according to the obstacle risk warning level information, and an obstacle risk level warning method is determined based on the driving warning mechanism; a driving obstacle avoidance strategy is analyzed based on the obstacle dynamic operation parameter set to obtain assisted driving strategy parameters; and an assisted driving warning is performed on the target ADAS system based on the obstacle risk level warning method and the assisted driving strategy parameters.

[0057] Specifically, the driving warning mechanism is triggered according to the obstacle risk warning level information, and the warning method suitable for the current risk level is determined based on the mechanism, such as using dashboard icons for low risk levels, audio alarms for medium risk levels, and multi-modal warning methods such as visual enhanced display and vibration feedback for high risk levels; based on the obstacle dynamic operation parameter set, the driving obstacle avoidance strategy is analyzed, including calculating the appropriate braking distance, steering angle, and acceleration and deceleration parameters, and generating auxiliary driving strategy parameters to guide the vehicle's obstacle avoidance behavior; combining the obstacle risk level warning method with the auxiliary driving strategy parameters, the auxiliary driving warning information is transmitted to the target ADAS system through a visual interface, audio prompts or vehicle dynamic control. Through this process, the ADAS system can transmit risk information to the driver in real time and intuitively, and provide reliable driving assistance strategies, thereby improving driving safety and user experience.

[0058] In summary, the embodiments of the present application have at least the following technical effects: First, an error handling module and an ADAS data parsing module are established. Next, a real-time image set of the driving environment is obtained through camera capture, a new cycle is triggered, and the target ADAS system is subjected to error detection and update processing through the error handling module. Then, the real-time image set of the driving environment is identified and parsed based on the ADAS data parsing module to obtain a multi-dimensional feature data set of obstacles. Further. The operation data of each obstacle information in the multi-dimensional feature data set of obstacles is dynamically analyzed to obtain a dynamic operation parameter set of obstacles. Finally, an obstacle risk assessment is performed based on the dynamic operation parameter set of obstacles, the obstacle risk warning level information is determined, and the target ADAS system is given an assisted driving warning through the obstacle risk warning level information and the dynamic operation parameter set of obstacles. The technical problem of inaccurate obstacle recognition in complex environments in the prior art is solved, and the technical effect of improving the accuracy of obstacle recognition is achieved.

[0059] Embodiment 2: Figure 3 The schematic diagram of the structure of the electronic device provided in the third embodiment of the present invention shows a block diagram of an exemplary electronic device suitable for implementing the implementation mode of the present invention. Figure 3 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present invention. Figure 3 As shown, the electronic device includes a processor 21, a memory 22, an input device 23 and an output device 24; the number of processors 21 in the electronic device can be one or more. Figure 3 Taking a processor 21 as an example, the processor 21, memory 22, input device 23 and output device 24 in the electronic device can be connected through a bus or other means. Figure 3 The example of connecting through bus is taken in the following.

[0060] Embodiment 3, based on the same inventive concept as the ADAS data processing method of computer vision in the above-mentioned embodiment, this embodiment provides a computer-readable storage medium, which can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the ADAS data processing method of computer vision in the embodiment of the present application. The processor executes various functional applications and data processing of the computer device by running the software programs, instructions and modules stored in the memory, that is, realizing the above-mentioned ADAS data processing method of computer vision.

[0061] It should be noted that the above-mentioned sequence of the embodiments of the present application is only for description and does not represent the advantages and disadvantages of the embodiments. And the above-mentioned specific embodiments of this specification are described. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0062] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

[0063] This specification and drawings are merely exemplary illustrations of the present application and are deemed to cover any and all modifications, variations, combinations or equivalents within the scope of the present application. Obviously, a person skilled in the art may make various modifications and variations to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present application and its equivalents, the present application intends to include these modifications and variations.

Claims

1. A computer vision ADAS data processing method, characterized in that: The method comprises: Establish error handling module and ADAS data analysis module; A real-time image set of the driving environment is obtained by capturing the camera, a new cycle is triggered, and the error detection and update processing of the target ADAS system is performed through the error processing module; Based on the ADAS data analysis module, the real-time image set of the driving environment is identified and analyzed to obtain a multi-dimensional feature data set of obstacles; Performing a dynamic analysis of the operation data of each obstacle information in the obstacle multi-dimensional feature data set to obtain a dynamic operation parameter set of the obstacle; Performing obstacle risk assessment based on the obstacle dynamic operation parameter set, determining obstacle risk warning level information, and performing assisted driving warning on the target ADAS system through the obstacle risk warning level information and the obstacle dynamic operation parameter set; The step of obtaining a multi-dimensional feature dataset of an obstacle comprises: According to the ADAS data parsing module, an image preprocessing module is obtained; Performing noise identification and filtering processing on the driving environment real-time image set by the image preprocessing module to obtain a denoised driving environment real-time image set; Mapping the denoised driving environment real-time image set to an image grayscale distribution coordinate system for grayscale conversion and equalization enhancement to obtain a usable driving environment real-time image set; Obstacle recognition and feature analysis are performed on the available driving environment real-time image set to obtain a multi-dimensional feature data set of obstacles.

2. A computer vision ADAS data processing method as claimed in claim 1, characterized in that: The step of obtaining a multi-dimensional feature dataset of an obstacle comprises: Using a driving semantic segmentation model to perform semantic segmentation on the real-time image set of the driving environment to obtain a driving environment semantic segmentation result; Obstacle identification and numbering are performed based on the driving environment semantic segmentation result to obtain an obstacle target set and an obstacle identification code set; Obtaining obstacle characteristic factor information, wherein the obstacle characteristic factor information includes obstacle type, geometric shape, size, and quantity and location; Based on the obstacle characteristic factor information and in accordance with the obstacle identification code set, feature analysis and extraction are performed on the obstacle target set in sequence to obtain the obstacle multi-dimensional feature data set.

3. A computer vision ADAS data processing method as claimed in claim 2, characterized in that: The obtaining of the driving environment semantic segmentation result includes: Acquire a driving environment image set, perform image segmentation and labeling on the driving environment image set according to a driving environment safety standard, and obtain an image labeling segmentation result set; Performing semantic segmentation training based on the image labeled segmentation result set to obtain a driving environment semantic segmentation model, wherein the driving environment semantic segmentation model includes a network encoder and a network decoder; Inputting the real-time image set of the driving environment into the network encoder to obtain a multi-scale feature set of the driving environment image; Based on the network decoder, segmentation prediction is performed on the multi-scale feature set of the driving environment image, and the driving environment semantic segmentation result is output.

4. A computer vision ADAS data processing method as claimed in claim 2, characterized in that: The step of obtaining the obstacle dynamic operation parameter set includes: Arranging and integrating the obstacle multidimensional feature data set according to time series information to obtain an obstacle multidimensional feature time series data set; Dividing the obstacle multi-dimensional feature time series data set according to the obstacle identification code set to obtain an obstacle time series feature identification data set; Performing speed change calculation and distance change calculation on the obstacle time series feature identification data set respectively to generate an obstacle speed change data set and an obstacle distance change data set; Dynamic analysis is performed on the obstacle speed change data set and the obstacle distance change data set to obtain an obstacle dynamic operation parameter set.

5. A computer vision ADAS data processing method as claimed in claim 4, characterized in that: The step of obtaining the obstacle dynamic operation parameter set includes: Acquiring an obstacle operation data set, annotating the operation mode of the obstacle operation data set, and obtaining an obstacle operation sample set; Using a recurrent neural network structure to perform labeling training on the obstacle operation sample set to obtain an obstacle trajectory prediction model and an obstacle behavior recognition model; The obstacle trajectory prediction model and the obstacle behavior recognition model are integrated in parallel to generate an obstacle operation mode analysis model; Based on the obstacle operation mode analysis model, the obstacle speed change data set and the obstacle distance change data set are subjected to operation dynamic analysis to obtain the obstacle dynamic operation parameter set.

6. A computer vision ADAS data processing method as claimed in claim 1, characterized in that: The determining of obstacle risk warning level information includes: Constructing an obstacle risk assessment model library, wherein the obstacle risk assessment model library includes a set of obstacle risk assessment models of multi-dimensional feature types of various obstacles; Based on the obstacle multidimensional feature data set and the obstacle risk assessment model library, matching and calling are performed to obtain a target obstacle risk assessment model; Based on the target obstacle risk assessment model, a risk level assessment is performed on the obstacle dynamic operation parameter set to determine the obstacle risk warning level information.

7. A computer vision ADAS data processing method as claimed in claim 1, characterized in that: The step of performing an assisted driving warning on the target ADAS system by using the obstacle risk warning level information and the obstacle dynamic operation parameter set includes: triggering a driving warning mechanism according to the obstacle risk warning level information, and determining an obstacle risk level warning method based on the driving warning mechanism; Performing driving obstacle avoidance strategy analysis based on the obstacle dynamic operation parameter set to obtain auxiliary driving strategy parameters; Based on the obstacle risk level warning method and the assisted driving strategy parameters, an assisted driving warning is performed on the target ADAS system.

8. An electronic device, characterized in that: The electronic device comprises: A memory for storing executable instructions; A processor, configured to implement a computer vision ADAS data processing method as described in any one of claims 1 to 7 when executing executable instructions stored in the memory.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, a computer vision ADAS data processing method as described in any one of claims 1 to 7 is implemented.

Images