A riding safety early warning and active anti-collision method based on scene matching and multi-modal feature fusion

Abstract

The application discloses a riding safety early warning and active anti-collision method based on scene matching and multi-modal feature fusion, which collects image data, obstacle distance and motion data, posture data and voice data, carries out pretreatment, feature extraction and feature fusion on the collected data to obtain fusion feature information; determines the current riding scene according to the road environment features and the vehicle driving state, calls the corresponding safety threshold and anti-collision threshold, and inputs the fusion feature information into an early warning decision model to obtain a risk level; then, corresponding early warning control instructions or brake control instructions are outputted, and hierarchical early warning or braking is executed. The method can adapt to the safety early warning and active anti-collision requirements in different riding scenes.

Description

Technical Field

[0001] This invention relates to the field of cycling safety technology, specifically to a cycling safety warning and active collision avoidance method based on scene matching and multimodal feature fusion. Background Technology

[0002] With the widespread use of cycling devices in urban commuting and short-distance travel, problems such as insufficient environmental perception, delayed warnings, and inadequate collision prevention capabilities during cycling have become increasingly prominent. Existing cycling safety solutions mostly use a single sensor for distance detection or image recognition, or only use single methods such as sound or light to remind cyclists, making it difficult to simultaneously take into account complex road environments, the movement of obstacles, and changes in cycling posture.

[0003] Meanwhile, existing solutions mostly rely on fixed thresholds for risk assessment, lacking parameter matching mechanisms for different scenarios such as urban commuting and highway cycling. They also rarely use multi-source perception results for unified decision-making and active braking control, which can easily lead to false alarms, missed alarms, or untimely warnings.

[0004] Therefore, there is an urgent need for a cycling safety warning and active collision avoidance method that can fuse multimodal data and dynamically call thresholds, output graded warnings, and active collision avoidance control in combination with cycling scenarios.

[0005] A search revealed a Chinese patent document disclosing an intelligent helmet and collision warning method [Application No.: 202011186334.4, Publication No.: CN114424851B], comprising a helmet shell, a sensing module, and a feedback module. The sensing module and feedback module are respectively located on the first and second sides of the helmet shell, which are opposite to each other. While the comparative patent achieves a similar purpose as this invention—acquiring moving object monitoring data via radar, assessing collision risk based on relative distance or speed, and issuing warnings to the user via vibration—this invention employs multimodal data acquisition using a camera, radar, attitude sensor, and microphone. It preprocesses, extracts, and fuses the acquired data, then combines this data with the current riding scenario to call corresponding safety and collision avoidance thresholds, further outputting tiered warning control commands or braking control commands. The comparative patent lacks the multimodal feature fusion, scene matching, dual threshold calling, and active braking control features of this invention. Summary of the Invention

[0006] To address the problems existing in the prior art, the purpose of this invention is to provide a cycling safety warning and active collision avoidance method based on scene matching and multimodal feature fusion.

[0007] A cycling safety warning and active collision avoidance method based on scene matching and multimodal feature fusion, characterized by the following steps: S1. Acquire image data, obstacle distance and motion data, posture data, and voice data; S2. The data is preprocessed, features are extracted, and features are fused to obtain fused feature information; S3. Determine the current riding scenario based on road environment characteristics, vehicle speed, and vehicle posture, and call the corresponding safety threshold and collision avoidance threshold; S4. Input the fused feature information into the early warning decision model to obtain the risk level; S5. Output a warning control command or a braking control command according to the risk level, the safety threshold, and the collision avoidance threshold, and execute the corresponding warning or braking.

[0008] Preferably, the preprocessing in step S2 includes: denoising the image data and radar data, normalizing the data of each modality, and performing time synchronization and alignment.

[0009] Through the above technical solutions, this invention can improve the consistency and usability of multimodal data. By denoising image and radar data, and normalizing and aligning the data of each modality, interference and deviation between different sensor data can be reduced, providing a stable data foundation for subsequent feature extraction and fusion, thereby improving the accuracy of cycling scene recognition, risk assessment, and early warning control.

[0010] Specifically, image and radar data are easily affected by environmental noise, equipment fluctuations, and external interference during acquisition. If used directly for subsequent analysis, they can easily lead to inaccurate obstacle recognition, distance judgment errors, and confusion in the correspondence between multi-source information. Denoising can reduce the impact of invalid signals on valid information; normalization can unify the dimensions and ranges of different modal data, facilitating subsequent fusion processing; and time synchronization alignment can ensure that data collected by different sensors at the same time correspond to each other, thereby ensuring that the fused features can accurately reflect the current riding environment and vehicle status.

[0011] In practical applications, this preprocessing method can better adapt to the data processing needs of different scenarios such as urban commuting and high-speed cycling. Especially in situations with a variety of obstacles and rapid environmental changes, it can still provide relatively stable and reliable data input to the system. Based on this, subsequent scene matching, threshold invocation, risk level judgment, and warning and braking control can all be performed more stably, thereby improving the practicality and reliability of the entire cycling safety warning and active collision avoidance method.

[0012] Preferably, the feature fusion in step S2 adopts a multi-head attention feature fusion model based on an attention mechanism.

[0013] Through the above technical solutions, this invention can improve the completeness and relevance of multimodal feature fusion. By adopting a multi-head attention feature fusion model based on an attention mechanism, it is possible to extract interrelated key feature information from multi-source data, reduce the limitations caused by isolated processing of single-modal information, and enable the output fused features to more comprehensively reflect obstacle states, road environments, and riding states.

[0014] Specifically, different modalities of data differ in their information representation methods. Image data is better suited for representing obstacle shapes and road environments, radar data is better suited for representing obstacle distances and motion states, attitude data is better suited for representing vehicle operating states, and voice data can supplement user commands or environmental prompts. Through a multi-head attention feature fusion model, key components in different modalities can be focused on separately, and correspondences between modalities can be established. This allows for the retention of effective information and the suppression of redundant information during the fusion process, resulting in more representative and consistent fused feature information.

[0015] In practical applications, this feature fusion method can better handle situations involving diverse obstacles, rapid environmental changes, and complex information sources in cycling scenarios, enabling the system to obtain relatively complete environmental perception results even in complex road environments. This provides more sufficient evidence for subsequent early warning decisions and active collision avoidance control, improving the overall stability and applicability of the system.

[0016] Preferably, the early warning decision model in step S4 is a deep Q-network reinforcement learning model, wherein the state space of the deep Q-network reinforcement learning model is the fused feature information, and the action space is different early warning control commands and different braking control commands.

[0017] Through the above technical solution, this invention can improve the rationality of early warning and braking decisions. By using fused feature information as state input and different early warning control commands and different braking control commands as action outputs, and utilizing a deep Q-network reinforcement learning model for decision-making, the system can select the corresponding response method according to the current riding state, thereby improving the matching degree between the decision output and the actual risk state.

[0018] Specifically, the fused feature information includes obstacle locations, motion states, road environment features, and vehicle operating states, which together constitute the state information of the current cycling scenario. The deep Q-network reinforcement learning model continuously learns from the action results under different states, gradually establishing a correspondence between states and warning / braking measures, enabling the model to output different levels of control commands under varying risk conditions. Compared to relying solely on a single fixed rule for judgment, this approach makes warning and braking decisions more consistent with the dynamic changes in actual cycling.

[0019] In practical applications, this decision-making method is suitable for urban roads, suburban roads, and cycling environments with significant speed variations, consistently delivering relatively stable decision results as cycling conditions change. Based on this, the system can more effectively implement warnings or braking control when facing different types of obstacles and varying motion states, thereby enhancing the practical value of cycling safety assistance.

[0020] Preferably, the current cycling scenarios include urban commuting scenarios and highway cycling scenarios, with different safety thresholds and collision avoidance thresholds corresponding to different cycling scenarios.

[0021] Through the above technical solution, this invention can improve the targeting of parameter calls in different cycling scenarios. By dividing the current cycling scenario into urban commuting scenario and highway cycling scenario, and assigning different safety thresholds and collision avoidance thresholds to different cycling scenarios, the risk assessment standard can be matched with the actual cycling environment, thereby improving the adaptability of warning and collision avoidance control.

[0022] Specifically, urban commuting scenarios involve a greater variety of obstacles, frequent road changes, and relatively low cycling speeds, while high-speed cycling scenarios involve higher vehicle speeds and shorter obstacle detection and handling times. These two scenarios differ significantly in their risk formation mechanisms and response requirements. By differentiating cycling scenarios and setting corresponding safety and collision avoidance thresholds for each, the system can adopt judgment benchmarks that better meet actual needs in different environments, avoiding the problem of judgment imbalance when using a single parameter in multiple scenarios.

[0023] In practical applications, this scenario-based parameter calling method enables the present invention to better adapt to daily commutes, long-distance cycling, and relatively high-speed cycling, enhancing the system's adaptability to different road environments. Therefore, when switching between different scenarios, the system can maintain relatively continuous risk assessment and control logic, improving the overall stability of the solution.

[0024] Preferably, when the current cycling scenario is an urban commuting scenario, millimeter-wave radar is used, the safety threshold is 5 meters, and the collision avoidance threshold is 2 meters; when the distance to the obstacle is greater than 2 meters and less than or equal to 5 meters, a moderate warning control command is output; when the distance to the obstacle is less than or equal to 2 meters, a severe warning control command and a braking control command are output.

[0025] Through the above technical solution, this invention can improve the efficiency of risk identification and response in urban commuting scenarios. By employing millimeter-wave radar in urban commuting scenarios and setting the safety threshold and collision avoidance threshold to corresponding distance ranges, the system can distinguish between the warning phase and the braking phase during obstacle approach, thereby achieving a more layered risk response.

[0026] Specifically, in urban commuting scenarios, there are many road users, with motor vehicles, non-motor vehicles, pedestrians, and temporary obstacles appearing alternately. Cycling risks typically exhibit characteristics of gradually decreasing distance and continuously changing conditions. By setting corresponding safety thresholds and collision avoidance thresholds, the system can first output a moderate warning control command when the obstacle distance is within the warning range, allowing cyclists to be aware of the surrounding risks as early as possible. When the obstacle distance further decreases and reaches the collision avoidance range, a severe warning control command and braking control command are then output, giving the system response a hierarchical relationship of warning first and braking later.

[0027] In practical applications, this technical solution is particularly suitable for urban road environments with intersections, heavy traffic, and frequent obstacles, providing more practical warnings and collision avoidance control under complex commuting conditions. This makes the system's use in urban commuting scenarios more seamless and enhances safety during cycling.

[0028] Preferably, when the current cycling scenario is a high-speed cycling scenario, a lidar is used, the safety threshold is 15 meters, and the anti-collision threshold is 8 meters; when the distance to the obstacle is greater than 8 meters and less than or equal to 15 meters, a warning control command is output; when the distance to the obstacle is less than or equal to 8 meters, a braking control command is output.

[0029] Through the above technical solution, this invention can improve the forward risk response capability in high-speed cycling scenarios. By employing lidar in high-speed cycling scenarios, and in conjunction with larger safety thresholds and collision avoidance thresholds, the system can identify obstacle states at a greater distance and execute warnings or braking control when obstacles enter different distance ranges, thereby meeting the requirements for early response in high-speed cycling scenarios.

[0030] Specifically, in high-speed cycling scenarios, vehicles travel at higher speeds and cover greater distances per unit time. If the same distance judgment standard as in urban commuting scenarios is used, the warning timing may be too late. By using LiDAR to acquire obstacle information at a greater distance and correspondingly increasing the safety and collision avoidance thresholds, the system can issue a warning control command when the obstacle is still at a considerable distance, and then issue a braking control command when the obstacle continues to approach and reaches the collision avoidance zone. This allows the entire response chain to adapt to the actual needs of high-speed cycling.

[0031] In practical applications, this solution is suitable for suburban roads, long-distance cycling routes, and relatively open but high-speed cycling environments, providing cyclists with earlier risk warnings and more timely braking trigger conditions. Based on this, the system's adaptability and control stability in high-speed cycling scenarios can be improved.

[0032] Preferably, the warning control commands include mild warning control commands, moderate warning control commands, and severe warning control commands; the mild warning control command corresponds to an audible warning, the moderate warning control command corresponds to an audible warning and a visual warning, and the severe warning control command corresponds to an audible warning, a visual warning, and a vibration warning; the braking control commands include braking control commands with different braking force levels.

[0033] Through the above technical solution, the present invention can improve the hierarchy and perceptibility of early warning information transmission. By dividing the early warning control commands into mild, moderate, and severe early warning control commands, and corresponding to sound warnings, sound and light warnings, and sound, light, and vibration warnings respectively, and by dividing the braking control commands into different braking force levels, the system can adopt different forms and intensities of response methods for different risk states.

[0034] Specifically, cyclists' needs for receiving warning information and the system's control requirements differ depending on the risk level. By tiering the warning methods, using audio prompts for mild risk conditions reduces interference with normal riding; as the risk level increases, adding light and vibration prompts enhances the alerting effect on the user. Simultaneously, setting different braking force levels for braking control commands establishes a correspondence between braking output and risk levels, creating a clearer hierarchical structure in the control process.

[0035] In practical applications, this tiered warning and braking system can better adapt to actual conditions such as road noise, changes in lighting, and differences in rider attention allocation, enabling the system to provide clearer risk warnings and control outputs in various usage environments. This improves the overall effectiveness and engineering application value of the solution in real-world cycling scenarios.

[0036] Compared with the prior art, the present invention has the following advantages: 1. This invention incorporates multi-source information such as images, radar, attitude, and voice into the same processing link, and forms a unified environmental and state representation through preprocessing, feature extraction, and fusion analysis. This can reduce the judgment bias caused by a single perception method in complex road conditions, make risk identification during cycling closer to the actual road environment, and make the connection between warning and collision avoidance decisions smoother.

[0037] 2. This invention invokes corresponding safety and collision avoidance thresholds based on different riding scenarios and sets up layered warning and braking execution, enabling the system to first issue a warning during risk development before initiating braking control, resulting in a clearer overall response logic. Simultaneously, warning methods such as sound, light, and vibration can be output in combination according to risk level, which helps improve the information delivery effect in different environments. Attached Figure Description

[0038] Figure 1 This is a schematic diagram of the system architecture of the present invention; Figure 2 This is a schematic diagram of the data fusion processing flow of the present invention; Figure 3 This is a schematic diagram of the early warning and collision avoidance decision-making process of the present invention; Figure 4 This is a schematic diagram of the overall process of the method of the present invention. Detailed Implementation

[0039] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0040] Equivalent substitutions, conventional adjustments, and reasonable improvements made by those skilled in the art based on the disclosure of this invention should all fall within the protection scope of this invention.

[0041] Working principle like Figure 1As shown, this invention adopts a layered architecture consisting of a sensor layer 1, a data fusion layer 2, a warning decision layer 3, and an execution layer 4. The sensor layer 1 includes a camera 11, a radar 12, an attitude sensor 13, and a microphone 14, used to simultaneously collect information about the surrounding environment, obstacle movement, riding status, and voice information during cycling. The data fusion layer 2 includes a data fusion module 21, used to preprocess, extract features, and fuse the collected multimodal raw data. The warning decision layer 3 includes a warning decision module 31, used to determine the risk level based on the fused feature information and the current riding scenario, and output corresponding warning control commands and braking control commands. The execution layer 4 includes a sound module 41, a light module 42, a vibration module 43, and a braking execution module 44, used to implement corresponding warning and braking measures according to the control commands output by the warning decision layer 3. The layers interact with each other through a data transmission channel, thus forming a complete processing chain from data acquisition to warning execution.

[0042] like Figure 2 As shown, the processing flow of data fusion layer 2 includes raw data acquisition, data preprocessing, single-modal feature extraction, multi-modal feature fusion, and fused feature information output. Specifically, the image data acquired by camera 11 can represent the shape of obstacles, road markings, and traffic signs; the radar data acquired by radar 12 can represent the distance, relative speed, and direction of movement of obstacles; the attitude data acquired by attitude sensor 13 can represent the rider's head posture and the vehicle's tilt, acceleration, and deceleration states; and the voice data acquired by microphone 14 can represent the rider's voice commands and surrounding traffic prompts. The data fusion module 21 first performs denoising processing on the image data and radar data, and normalizes and aligns the data of each modality to reduce noise interference and ensure the temporal correspondence between different modal data. Then, it extracts features from each modal data to obtain key features such as obstacle position, obstacle motion state, road environment features, vehicle speed, vehicle posture, and speech semantic information. The key features are then input into a multi-head attention feature fusion model based on an attention mechanism to form unified fused feature information and output to the early warning decision layer 3.

[0043] like Figure 3As shown, after receiving the fused feature information output by the data fusion layer 2, the early warning decision layer 3 first determines the current riding scenario based on road environment characteristics, vehicle speed, and vehicle posture, and then calls the safety threshold and collision avoidance threshold corresponding to the riding scenario. The riding scenario includes at least urban commuting scenarios and highway riding scenarios. For urban commuting scenarios, there are many types of obstacles that appear frequently, so the system calls a relatively small threshold group; for highway riding scenarios, the vehicle speed is faster, so the system calls a relatively large threshold group to enter the early warning judgment process earlier. Subsequently, the early warning decision module 31 inputs the fused feature information and the threshold parameters corresponding to the current riding scenario into the collision avoidance model, and outputs the risk level and the corresponding early warning control command and braking control command.

[0044] The collision avoidance model is trained using a deep Q-network reinforcement learning algorithm. During the training phase, training samples covering different cycling scenarios are first constructed. These samples include image data, radar data, posture data, and voice data collected in urban commuting and high-speed cycling scenarios. The data fusion module 21 then preprocesses, extracts, and fuses the training samples to form fused feature information. During training, the fused feature information at the current moment is used as the state input, which includes at least obstacle distance, obstacle motion state, road environment features, vehicle speed, and vehicle posture. Different levels of warning control commands and different levels of braking control commands are used as action outputs. The system constructs reward values ​​based on the results of the actions. A positive reward is given when the output warning control command or braking control command corresponds to the current cycling risk state and no collision occurs; a negative reward is given when the warning timing is delayed, the warning level is incompatible, the braking measures are incompatible, or a collision occurs. Through continuous training on sample data from different cycling scenarios, the collision avoidance model gradually establishes a correspondence between the fused feature information and the warning and braking measures.

[0045] During actual operation, the early warning decision module 31 inputs the fused feature information of the current moment into the trained collision avoidance model and judges the risk level by combining the safety threshold and collision avoidance threshold corresponding to the current riding scenario. When the obstacle distance is between the safety threshold and the collision avoidance threshold, the corresponding level of early warning control command is output; when the obstacle distance reaches the collision avoidance threshold range, the corresponding braking control command is output. After receiving the control command, the execution layer 4 activates the corresponding modules in the sound module 41, light module 42, and vibration module 43 to implement graded early warning, or controls the braking execution module 44 to implement braking according to the corresponding braking force level.

[0046] like Figure 4As shown, the method of this invention generally follows the sequence of "multimodal data acquisition—data fusion processing—early warning and collision avoidance decision-making—early warning and collision avoidance execution," with the output of the previous stage serving as the input of the next stage, thus forming a complete closed-loop data processing process. Through the above process, this invention can complete multi-source information collaborative perception, risk assessment, and hierarchical execution in different riding environments.

[0047] Example 1: Urban commuting cycling scenario This embodiment is applied to an urban commuting cycling scenario. In this scenario, road traffic volume is high, there are many intersections, and obstacle types include motor vehicles, non-motor vehicles, pedestrians, and roadblocks, with a rapidly changing cycling environment. To adapt to this scenario, this embodiment uses a smart helmet with a built-in camera 11, millimeter-wave radar 12, attitude sensor 13, and microphone 14, as well as a smart electric bicycle equipped with a main controller, communication module, braking execution module 44, lighting module 42, and vibration module 43; the system software includes a data acquisition module, a data fusion module 21, a warning decision module 31, and a warning execution module, and data interaction between the modules is achieved through a mobile communication network.

[0048] After the ride begins, camera 11 collects real-time image data of the road ahead and surrounding areas, millimeter-wave radar 12 collects data on the distance, speed, and direction of movement of surrounding obstacles, posture sensor 13 collects data on the rider's head posture and vehicle tilt, acceleration, and deceleration, and microphone 14 collects the rider's voice commands and surrounding traffic prompts. This multimodal raw data is sent in real-time to the data fusion module 21 in the main controller. The data fusion module 21 first performs noise reduction processing on the image and radar data, normalizes and aligns the data of each modality, and then extracts key features such as obstacle position, motion state, road environment characteristics, vehicle speed, vehicle posture, and speech semantics, forming unified fused feature information.

[0049] Subsequently, the early warning decision module 31 determines the current scenario as an urban commuting scenario based on road environment characteristics, vehicle speed, and vehicle attitude, and calls the corresponding safety threshold and collision avoidance threshold. In this embodiment, the urban commuting scenario uses millimeter-wave radar 12 for obstacle detection, with a safety threshold set to 5 meters and a collision avoidance threshold set to 2 meters. The early warning decision module 31 inputs the fused feature information output by the data fusion module 21 into the trained collision avoidance model and combines it with the threshold corresponding to the urban commuting scenario to determine the risk level. The collision avoidance model is trained using a deep Q-network reinforcement learning algorithm. During training, the fused feature information is used as the state input, and different levels of early warning control commands and different levels of braking control commands are used as action outputs. A reward value is set based on the result after the action is executed. During operation, when the obstacle distance is greater than 2 meters and less than or equal to 5 meters, a moderate early warning control command is output; when the obstacle distance is less than or equal to 2 meters, a severe early warning control command and a braking control command are output; when the obstacle distance is greater than 5 meters, no early warning control command or braking control command is output.

[0050] Upon receiving a moderate warning control command, execution layer 4 activates the sound module 41 and the light module 42 for joint warning; upon receiving a severe warning control command and a braking control command, it activates the sound module 41, the light module 42, and the vibration module 43 for combined warning, and controls the braking execution module 44 to apply braking according to the corresponding braking force level. Thus, the system can achieve cycling safety protection in urban commuting scenarios by issuing warnings first and then braking.

[0051] Example 2: High-speed cycling scenario This embodiment applies to high-speed cycling scenarios. These scenarios may include high-speed cycling environments such as long-distance cycling in suburban areas, where obstacles are often distant moving targets, placing higher demands on advance warning and braking timing. Compared to Embodiment 1, the overall architecture of this embodiment still adopts... Figure 1 The sensor layer 1, data fusion layer 2, early warning decision layer 3, and execution layer 4 shown are only adjusted in terms of obstacle detection components and scene parameters. Specifically, in this embodiment, radar 12 is configured as a lidar, and the remaining hardware and software configurations are basically the same as in embodiment one.

[0052] In this embodiment, the data acquisition and data fusion processing are basically the same as in Embodiment 1, that is, multimodal raw data are acquired through camera 11, lidar 12, attitude sensor 13, and microphone 14, and preprocessing, feature extraction, and fusion processing are completed by data fusion module 21. The difference is that after analyzing the current road environment characteristics, vehicle speed, and vehicle posture, the early warning decision module 31 determines the current riding scenario as a high-speed riding scenario and calls the corresponding safety threshold and collision avoidance threshold. In this embodiment, the safety threshold for the high-speed riding scenario is set to 15 meters, and the collision avoidance threshold is set to 8 meters. The early warning decision module 31 inputs the fused feature information output by data fusion module 21 into the trained collision avoidance model and judges the risk level by combining it with the threshold corresponding to the high-speed riding scenario. The collision avoidance model is trained using a deep Q-network reinforcement learning algorithm. During training, the fused feature information is used as the state input, and different levels of early warning control commands and different levels of braking control commands are used as action outputs. A reward value is set according to the result after the action is executed. During operation, when the distance to the obstacle is greater than 8 meters and less than or equal to 15 meters, a warning control command is output; when the distance to the obstacle is less than or equal to 8 meters, a braking control command is output.

[0053] The execution layer 4 implements corresponding measures based on the control commands output by the warning decision module 31. In a high-speed riding scenario, when a warning control command is output, the sound module 41 and the light module 42 are activated according to their corresponding levels; when a braking control command is output, the brake execution module 44 applies braking according to the corresponding braking force level, and coordinates with the warning outputs of the sound module 41, the light module 42, and the vibration module 43. Thus, this embodiment maintains... Figure 4 Based on the overall closed-loop data flow method shown, by adjusting the detection components and scene thresholds, the system can adapt to the needs of earlier warning and more timely braking in high-speed riding environments.

[0054] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A cycling safety warning and active collision avoidance method based on scene matching and multimodal feature fusion, characterized in that, Includes the following steps: S1. Acquire image data, obstacle distance and motion data, posture data, and voice data; S2. The data is preprocessed, features are extracted, and features are fused to obtain fused feature information; S3. Determine the current riding scenario based on road environment characteristics, vehicle speed, and vehicle posture, and call the corresponding safety threshold and collision avoidance threshold; S4. Input the fused feature information into the early warning decision model to obtain the risk level; S5. Output a warning control command or a braking control command according to the risk level, the safety threshold, and the collision avoidance threshold, and execute the corresponding warning or braking.

2. The cycling safety warning and active collision avoidance method based on scene matching and multimodal feature fusion according to claim 1, characterized in that, The preprocessing in step S2 includes: denoising the image data and radar data, normalizing the data of each modality, and performing time synchronization and alignment.

3. The cycling safety warning and active collision avoidance method based on scene matching and multimodal feature fusion according to claim 1, characterized in that, The feature fusion in step S2 adopts a multi-head attention feature fusion model based on the attention mechanism.

4. The cycling safety warning and active collision avoidance method based on scene matching and multimodal feature fusion according to claim 1, characterized in that, The early warning decision model in step S4 is a deep Q-network reinforcement learning model. The state space of the deep Q-network reinforcement learning model is the fused feature information, and the action space is different early warning control commands and different braking control commands.

5. The cycling safety warning and active collision avoidance method based on scene matching and multimodal feature fusion according to claim 1, characterized in that, The current cycling scenarios include urban commuting scenarios and highway cycling scenarios, with different safety thresholds and collision avoidance thresholds corresponding to different cycling scenarios.

6. The cycling safety warning and active collision avoidance method based on scene matching and multimodal feature fusion according to claim 5, characterized in that, When the current cycling scenario is an urban commuting scenario, millimeter-wave radar is used, the safety threshold is 5 meters, and the collision avoidance threshold is 2 meters; when the distance to the obstacle is greater than 2 meters and less than or equal to 5 meters, a moderate warning control command is output; when the distance to the obstacle is less than or equal to 2 meters, a severe warning control command and a braking control command are output.

7. The cycling safety warning and active collision avoidance method based on scene matching and multimodal feature fusion according to claim 5, characterized in that, When the current cycling scenario is a high-speed cycling scenario, a lidar is used, the safety threshold is 15 meters, and the anti-collision threshold is 8 meters; when the distance to the obstacle is greater than 8 meters and less than or equal to 15 meters, a warning control command is output; when the distance to the obstacle is less than or equal to 8 meters, a braking control command is output.

8. The cycling safety warning and active collision avoidance method based on scene matching and multimodal feature fusion according to claim 1, characterized in that, The warning control commands include mild warning control commands, moderate warning control commands, and severe warning control commands; the mild warning control command corresponds to an audible warning, the moderate warning control command corresponds to an audible warning and a visual warning, and the severe warning control command corresponds to an audible warning, a visual warning, and a vibration warning. The braking control commands include braking control commands for different braking force levels.

Images