Vehicle projection warning method, device, equipment, system, vehicle, storage medium and program product

Abstract

The invention relates to a vehicle projection warning method, device, equipment and system, a vehicle, a storage medium and a program product. The method comprises the steps of obtaining video data of an environment where a current vehicle is located; performing target detection processing on the video data to obtain target movement information; wherein the target movement information comprises detected movement trend information and arrival time of each moving target; determining a risk level and motion direction information of the moving target according to the motion trend information and the arrival time of the moving target; and performing projection warning processing outside the vehicle according to the risk level and the movement direction information of the moving target. According to the invention, traffic participants around the vehicle can be reminded to improve attention, driving is controlled, traffic accidents can be reduced, and traffic safety is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicles, and particularly to a vehicle projection warning method, device, equipment, system, vehicle, storage medium, and program product. Background Art

[0002] In the modern traffic environment, ensuring driving safety is of utmost importance; among them, based on the warning function of the vehicle, vehicle safety control and vehicle warning can be realized.

[0003] Currently, in related technologies, a variety of passive and active safety devices and functions are equipped in vehicles, such as an Anti-lock Braking System (ABS), an Electronic Stability Program (ESP), a Forward Collision Warning (FCW) system, an Autonomous Emergency Braking (AEB) system, etc.; among them, the FCW system analyzes the driving state of the vehicle ahead, and if it determines that there is a collision risk, it will issue a warning to the driver; if the driver does not respond in time, the AEB system in combination with the ESP system will automatically apply brakes to decelerate the vehicle and help the driver avoid obstacles.

[0004] Or, by collecting the vehicle's front image data, the pre-movement trajectory of the obstacle in front of the vehicle is judged and displayed to provide prompt information for the driver of the vehicle during driving.

[0005] However, in the above methods, there is a situation where a surrounding traffic participant in the visual blind area suddenly appears in front of the current vehicle, resulting in the inability to timely perform vehicle safety control and vehicle warning, and further leading to traffic accidents and affecting traffic safety. Summary of the Invention

[0006] One of the purposes of the present invention is to provide a vehicle projection warning method, which can remind the traffic participants around the vehicle to improve their attention and control their driving, reduce traffic accidents, and improve traffic safety; the second purpose is to provide a vehicle projection warning device; the third purpose is to provide a central control unit; the fourth purpose is to provide a dynamic environment perception and warning system; the fifth purpose is to provide a vehicle; the sixth purpose is to provide a computer-readable storage medium; the seventh purpose is to provide a computer program product.

[0007] To achieve the above purposes, the technical solutions adopted by the present invention are as follows:

[0008] A vehicle projection warning method, which is applied to a central control unit in a dynamic environment perception and warning system; the method includes:

[0009] Obtain video data of the environment where the current vehicle is located; and perform target detection processing on the video data to obtain target movement information; wherein, the target movement information includes the movement trend information and arrival time of each detected moving target; the arrival time is the time required for the moving target to reach in front of the current vehicle;

[0010] According to the movement trend information and arrival time of the moving target, determine the risk level and movement direction information of the moving target; wherein, the risk level is the traffic safety risk level existing between the target traffic participant and the moving target;

[0011] Perform out-of-vehicle projection warning processing according to the risk level and movement direction information of the moving target.

[0012] Further, the determining the risk level and movement direction information of the moving target according to the movement trend information and arrival time of the moving target includes:

[0013] Determine the risk level of the moving target according to the arrival time and target type of the moving target;

[0014] Determine the movement direction information of the moving target according to the movement trend information of the moving target.

[0015] Further, the determining the risk level of the moving target according to the arrival time and target type of the moving target includes:

[0016] Determine the compensation coefficient corresponding to the target type according to the target type of the moving target;

[0017] Determine the comparison time corresponding to the moving target according to the compensation coefficient corresponding to the target type;

[0018] Determine the risk level of the moving target according to the arrival time and comparison time of the moving target.

[0019] Further, the performing target detection processing on the video data to obtain target movement information includes:

[0020] Perform target detection processing on each image in the video data to determine target coordinate information; wherein, the target coordinate information includes the image coordinates of each detected moving target in each image of the video data;

[0021] Determine the motion trend information of the moving target according to the image coordinates of the moving target in each image; wherein, the motion trend information includes a lateral moving speed and a lateral distance, and the lateral distance is the lateral distance between the moving target and the current vehicle;

[0022] Determine the arrival time of the moving target according to the lateral moving speed and the lateral distance of the moving target.

[0023] Further, the performing target detection processing on each image in the video data to determine target coordinate information includes:

[0024] Perform target detection processing on each image in the video data to determine initial coordinate information; wherein, the initial coordinate information includes the image coordinates of each detected initial target in each image of the video data;

[0025] Determine the initial target closest to the current vehicle as the moving target according to the initial coordinate information and the position of the current vehicle, so as to obtain the image coordinates of the moving target in each image of the video data.

[0026] Further, the determining the motion trend information of the moving target according to the image coordinates of the moving target in each image includes:

[0027] Perform geographic coordinate conversion processing on the image coordinates to obtain the geographic coordinates corresponding to the image coordinates;

[0028] Determine the motion trend information of the moving target according to all the geographic coordinates corresponding to the moving target.

[0029] Further, the dynamic environment perception and warning system further includes a projection unit; the performing out-of-vehicle projection warning processing according to the risk level and the motion direction information of the moving target includes:

[0030] If it is determined that the risk level of the moving target is a preset level, determine the risk information of the moving target; wherein, the risk information includes the image of the moving target, the lateral distance in the motion trend information, the risk level, the arrival time, and the motion direction information; the lateral distance is the lateral distance between the moving target and the current vehicle;

[0031] Send the risk information of the moving target to the projection unit; wherein, the projection unit is used to generate a first warning information according to the risk information and perform out-of-vehicle projection processing on the first warning information.

[0032] Further, before sending the risk information of the moving target to the projection unit, it further includes:

[0033] Determine the target projection position according to the motion direction information of the moving target;

[0034] If it is determined that there is no fixed obstacle between the target projection position and the position of the moving target, send the target projection position to the projection unit; wherein, the projection unit is used for performing external vehicle projection processing on the moving target.

[0035] Further, the method further includes:

[0036] Obtain the light data and ground data of the environment where the current vehicle is located; wherein, the ground data includes the ground attribute data of the environment where the current vehicle is located;

[0037] Determine the actual projection brightness according to the light data and the ground data; wherein, the actual projection brightness is used for performing external vehicle projection warning processing.

[0038] Further, the determining the actual projection brightness according to the light data and the ground data includes:

[0039] Determine the material gain coefficient corresponding to the ground data according to the ground data;

[0040] Determine the actual projection brightness according to the light data and the material gain coefficient.

[0041] Further, the method further includes:

[0042] Obtain the side environment image of the environment where the current vehicle is located; and perform recognition processing on the side environment image to obtain the flat area ratio; wherein, the flat area ratio is the ratio of the flat ground area in the side environment image.

[0043] Perform conversion processing on the flat area ratio to obtain the flat area size; wherein, the flat area size is the actual size of the available flat ground area in the environment represented by the side environment image.

[0044] Determine the projection area size according to the flat area size; wherein, the projection area size is used for performing external vehicle projection warning processing.

[0045] Further, the dynamic environment perception and warning system further includes a cloud server unit; the method further includes:

[0046] Obtain the driving data of the current vehicle; and send the driving data of the current vehicle to the cloud server unit; wherein, the cloud server unit is used to generate the vehicle distance of at least one target vehicle according to the driving data of the current vehicle and the driving data of at least one other vehicle, and send it to the target vehicle; the target vehicle is another vehicle whose distance from the rear of the current vehicle is within a preset distance range and whose driving direction is the same as that of the current vehicle; the vehicle distance is the actual distance between the current vehicle and the target vehicle; the vehicle distance is used for in-vehicle warning processing.

[0047] Further, the dynamic environment perception and warning system further includes a cloud server unit; after performing the out-of-vehicle projection warning processing according to the risk level and movement direction information of the moving target, the method further includes:

[0048] Generate early warning event record data; and send the early warning event record data to the cloud server unit; wherein, the early warning event record data includes the image of the moving target, the lateral distance in the movement trend information, the risk level, the arrival time, and the movement direction information; the lateral distance is the lateral distance between the moving target and the current vehicle.

[0049] According to the above technical means, by combining the movement trend information of each detected moving target and the time required to reach in front of the current vehicle, the risk level and movement direction are determined, and according to the determined risk level and movement direction, out-of-vehicle projection warning is performed on the corresponding moving target, which can remind traffic participants around the vehicle to concentrate and control driving, and reduce the occurrence of traffic safety accidents.

[0050] A vehicle projection warning device, which is applied to the central control unit in the dynamic environment perception and warning system; the device includes:

[0051] A detection module, configured to obtain video data of the environment where the current vehicle is located; and perform target detection processing on the video data to obtain target movement information; wherein, the target movement information includes the movement trend information and arrival time of each detected moving target; the arrival time is the time required for the moving target to reach in front of the current vehicle;

[0052] A determination module, configured to determine the risk level and movement direction information of the moving target according to the movement trend information and arrival time of the moving target; wherein, the risk level is the traffic safety risk level existing between the target traffic participant and the moving target;

[0053] A warning module, configured to perform out-of-vehicle projection warning processing according to the risk level and movement direction information of the moving target.

[0054] Further, the determining module is specifically configured to: determine the risk level of the moving target according to the arrival time and target type of the moving target; determine the moving direction information of the moving target according to the movement trend information of the moving target.

[0055] Further, the determining module is also specifically configured to: determine the compensation coefficient corresponding to the target type according to the target type of the moving target; determine the comparison time corresponding to the moving target according to the compensation coefficient corresponding to the target type; determine the risk level of the moving target according to the arrival time and comparison time of the moving target.

[0056] Further, the detection module is specifically configured to: perform target detection processing on each image in the video data to determine target coordinate information; wherein, the target coordinate information includes the image coordinates of each detected moving target in each image in the video data; determine the movement trend information of the moving target according to the image coordinates of the moving target in each image; wherein, the movement trend information includes the lateral movement speed and the lateral distance, and the lateral distance is the lateral distance between the moving target and the current vehicle; determine the arrival time of the moving target according to the lateral movement speed and the lateral distance of the moving target.

[0057] Further, the detection module is also specifically configured to: perform target detection processing on each image in the video data to determine initial coordinate information; wherein, the initial coordinate information includes the image coordinates of each detected initial target in each image in the video data; determine the initial target closest to the current vehicle as the moving target according to the initial coordinate information and the position of the current vehicle, so as to obtain the image coordinates of the moving target in each image in the video data.

[0058] Further, the detection module is also specifically configured to: perform geographic coordinate conversion processing on the image coordinates to obtain the geographic coordinates corresponding to the image coordinates; determine the movement trend information of the moving target according to all the geographic coordinates corresponding to the moving target.

[0059] Further, the dynamic environment perception and warning system further includes a projection unit; the warning module is specifically configured to: if it is determined that the risk level of the moving target is a preset level, determine the risk information of the moving target; wherein, the risk information includes the image of the moving target, the lateral distance in the movement trend information, the risk level, the arrival time, and the moving direction information; the lateral distance is the lateral distance between the moving target and the current vehicle; send the risk information of the moving target to the projection unit; wherein, the projection unit is configured to generate a first warning information according to the risk information and perform an out-of-vehicle projection process on the first warning information.

[0060] Further, before the warning module is used to send the risk information of the moving target to the projection unit, the warning module is further used to: determine a target projection position according to the movement direction information of the moving target; if it is determined that there is no fixed obstacle between the target projection position and the position of the moving target, send the target projection position to the projection unit; wherein, the projection unit is used to perform an external vehicle projection process on the moving target.

[0061] Further, the device is further used to: obtain the light data and ground data of the environment where the current vehicle is located; wherein, the ground data includes the ground attribute data of the environment where the current vehicle is located; determine the actual projection brightness according to the light data and the ground data; wherein, the actual projection brightness is used for external vehicle projection warning processing.

[0062] Further, the device is specifically further used to: determine the material gain coefficient corresponding to the ground data according to the ground data; determine the actual projection brightness according to the light data and the material gain coefficient.

[0063] Further, the device is further used to: obtain a side environment image of the environment where the current vehicle is located; and perform an identification process on the side environment image to obtain a flat area ratio; wherein, the flat area ratio is the ratio of the flat ground area in the side environment image; perform a conversion process on the flat area ratio to obtain a flat area size; wherein, the flat area size is the actual size of the available flat ground area in the environment represented by the side environment image; determine a projection area size according to the flat area size; wherein, the projection area size is used for external vehicle projection warning processing.

[0064] Further, the dynamic environment perception and warning system further includes a cloud server unit; the device is further used to: obtain the driving data of the current vehicle; and send the driving data of the current vehicle to the cloud server unit; wherein, the cloud server unit is used to generate the vehicle distance of at least one target vehicle according to the driving data of the current vehicle and the driving data of at least one other vehicle and send it to the target vehicle; the target vehicle is another vehicle whose distance from the rear of the current vehicle is within a preset distance range and whose driving direction is the same as that of the current vehicle; the vehicle distance is the actual distance between the current vehicle and the target vehicle; the vehicle distance is used for in-vehicle warning processing.

[0065] Further, the dynamic environment perception and warning system further includes a cloud server unit; after the warning module is used to perform an external projection warning process according to the risk level and movement direction information of the moving target, the device is further used to: generate warning event record data; and send the warning event record data to the cloud server unit; wherein, the warning event record data includes an image of the moving target, the lateral distance in the movement trend information, the risk level, the arrival time, and the movement direction information; the lateral distance is the lateral distance between the moving target and the current vehicle.

[0066] A central control unit includes: a memory, a processor;

[0067] The memory stores computer execution instructions;

[0068] The processor executes the computer execution instructions stored in the memory, so that the processor executes the above first aspect and / or various possible implementation manners of the first aspect.

[0069] A dynamic environment perception and warning system includes a central control unit, and the central control unit is used to execute the above first aspect and / or various possible implementation manners of the first aspect.

[0070] A vehicle includes a dynamic environment perception and warning system, and the dynamic environment perception and warning system includes a central control unit; the central control unit is used to execute the above first aspect and / or various possible implementation manners of the first aspect.

[0071] A computer-readable storage medium stores computer execution instructions, and when the computer execution instructions are executed by a processor, they are used to implement the above first aspect and / or various possible implementation manners of the first aspect.

[0072] A computer program product includes a computer program, and when the computer program is executed by a processor, it implements the above first aspect and / or various possible implementation manners of the first aspect.

[0073] Advantages of the present invention:

[0074] By combining the movement trend information of each detected moving target and the time required to reach in front of the current vehicle, the present invention determines the risk level and movement direction, and according to the determined risk level and movement direction, performs an external projection warning on the corresponding surrounding traffic participants, which can remind the surrounding traffic participants of the vehicle to concentrate and control driving, and reduce the occurrence of traffic safety accidents. Description of the drawings

[0075] Figure 1 It is an application scenario diagram provided by an embodiment of the present invention;

[0076] Figure 2 Flow chart of the vehicle projection warning method provided by an embodiment of the present invention Figure 1 ;

[0077] Figure 3 Flow chart of the vehicle projection warning method provided by an embodiment of the present invention Figure 2 ;

[0078] Figure 4 System composition diagram of the dynamic environment perception and warning system provided by an embodiment of the present invention;

[0079] Figure 5 Working flow chart of the dynamic environment perception and warning system provided by an embodiment of the present invention;

[0080] Figure 6 Flow chart of the conversion between image coordinates and geographical coordinates provided by an embodiment of the present invention;

[0081] Figure 7 Composition diagram of the default projection scheme provided by an embodiment of the present invention;

[0082] Figure 8 Example diagram of the projection scheme provided by an embodiment of the present invention;

[0083] Figure 9 Flow chart of the vehicle projection warning method provided by an embodiment of the present invention Figure 3 ;

[0084] Figure 10 Working flow chart of the multi-vehicle interconnection warning provided by an embodiment of the present invention;

[0085] Figure 11 Structural schematic diagram of the simulated driving control device provided by an embodiment of the present invention;

[0086] Figure 12 Structural schematic diagram of the central control unit provided by an embodiment of the present invention. Detailed implementation manners

[0087] The following will describe the embodiments of the present invention with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for explaining the present invention, rather than for limiting the protection scope of the present invention.

[0088] It should be noted that the illustrations provided in the following embodiments only schematically illustrate the basic concept of the present invention. Therefore, only the components related to the present invention are shown in the drawings, rather than being drawn according to the number, shape, and size of the components in actual implementation. The types, quantities, and proportions of the components in actual implementation can be arbitrarily changed, and the component layout type may also be more complex.

[0089] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present invention are all information and data that have been authorized by the user or fully authorized by all parties. And the collection, use, and processing of relevant data need to comply with relevant laws, regulations, and standards, and corresponding operation entrances are provided for users to choose to authorize or refuse.

[0090] In the modern traffic environment, ensuring driving safety is of utmost importance; among them, based on the warning function of the vehicle, vehicle safety control and vehicle warning can be achieved.

[0091] In one example, a variety of passive and active safety devices and functions such as Anti-lock Braking System (ABS for short), Electronic Stability Program (ESP for short), Forward Collision Warning (FCW for short) system, and Autonomous Emergency Braking (AEB for short) system are equipped in the vehicle; among them, the FCW system will analyze the driving state of the vehicle ahead. If it judges that there is a collision risk, it will issue a warning to the driver; if the driver does not respond in time, the AEB system combined with the ESP system will automatically apply the brakes to decelerate the vehicle to help the driver avoid obstacles. However, there is a situation where surrounding traffic participants in the visual blind area suddenly appear in front of the current vehicle, resulting in the inability to timely perform vehicle safety control and vehicle warning, and then leading to traffic accidents and affecting traffic safety.

[0092] In another example, the current state data and the vehicle front image data of the vehicle are collected; the pre-driving trajectory and driving direction of the vehicle are obtained according to the current state data, and the pre-driving trajectory and driving direction of the vehicle are displayed through the rear ISD vehicle lights of the vehicle, and the pre-movement trajectory of the obstacle in front of the vehicle is judged, and the pre-movement trajectory of the obstacle is projected through the front projection vehicle lights of the vehicle. This solution projects the obstacle and the driving trajectory of the vehicle itself during driving, mainly providing prompt information for the driver of the vehicle itself during driving, and cannot effectively remind the rear traffic participants in a static state.

[0093] In view of this, an embodiment of the present invention provides a vehicle projection warning method. By combining the motion trend information of the detected moving target and the time required to reach in front of the current vehicle, the risk level and the moving direction are determined. And according to the determined risk level and moving direction, an out-of-vehicle projection warning is performed on the corresponding moving target or target traffic participant, which can remind the moving target or the traffic participants around the vehicle to concentrate and control driving, and reduce the occurrence of traffic safety accidents.

[0094] Figure 1 This is an application scenario diagram provided by an embodiment of the present invention. As Figure 1 shown, the scenario may include a vehicle, a vehicle behind the side, and a pedestrian. This vehicle is used to provide a vehicle projection warning to the vehicle behind the side of the vehicle to remind the vehicle behind the side that there is a pedestrian in front and pay attention to traffic safety.

[0095] The execution subject of the embodiment of the present invention may be a central control unit in a dynamic environment perception and warning system in a vehicle, and can be specifically set according to actual needs.

[0096] Next, the technical solution of the present invention will be described in detail through specific embodiments. It should be noted that these specific embodiments may be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0097] Figure 2 This is the flow of the vehicle projection warning method provided by an embodiment of the present invention Figure 1 As Figure 2 shown, the method includes:

[0098] 201. Obtain video data of the environment where the current vehicle is located; and perform target detection processing on the video data to obtain target movement information; wherein, the target movement information includes the motion trend information and the arrival time of the detected moving target; the arrival time is the time required for the moving target to reach in front of the current vehicle.

[0099] Exemplarily, a Dynamic Environment Sensing and Warning System (DESWS) is deployed in a vehicle. The DESWS includes a central control unit and an image acquisition device. When the vehicle is in a stationary state, such as waiting for a traffic light, the central control unit can obtain video data of the environment where the current vehicle is located through the image acquisition device in the current vehicle, including multiple consecutive environmental images within a current period of time. Based on a preset target detection technology, target detection processing is performed on the acquired video data, and thus each moving target, such as a pedestrian or a cyclist, can be detected from the video data. Also, target traffic participants around the current vehicle can be detected from the video data, for example, the first vehicle, cyclist, or pedestrian behind at least one side of the current vehicle. And based on a target tracking technology, the motion trend information and arrival time of the detected moving target can be obtained, where the arrival time is the time required for the moving target to reach in front of the current vehicle.

[0100] For example, when the vehicle is powered on, the DESWS automatically performs a hardware self-check to ensure that all units are working properly. At this time, the vehicle is in a stationary state, such as waiting at a traffic signal. At the same time, camera parameters are calibrated, including focusing, exposure time, etc. The camera continuously acquires video data in front of the vehicle at a high frame rate to ensure that no potential dangerous situations are missed, and the acquired video data is transmitted to the central control unit. The central control unit applies a target detection model based on deep learning to simultaneously complete the moving target positioning and tracking tasks in the same network, so as to perform target detection and tracking processing on the video data and the driving position of the current vehicle, and thus data such as the motion speed and motion trajectory of the detected moving target can be obtained, as well as the driving position of the first vehicle behind the right side of the current vehicle. Further, through the algorithm of the target detection model, the data such as the motion speed and motion trajectory of the moving target and the driving position of the current vehicle are processed to obtain the time required for the moving target to reach in front of the current vehicle for further processing.

[0101] 202. Determine the risk level information and motion direction information of the moving target according to the motion trend information and arrival time of the moving target; where the risk level is the traffic safety risk level existing between the target traffic participant and the moving target.

[0102] Exemplarily, based on a preset information processing rule, the central control unit comprehensively determines the risk level and movement direction of the moving target by processing the movement trend information of the moving target and the time required in front of the current vehicle, and obtains the risk level and movement direction information corresponding to the moving target. Among them, the risk level is the traffic safety risk level existing between the target traffic participant and the moving target. For example, according to the degree of risk, the risk level is divided into three risk levels: low risk, medium risk, and high risk; the movement direction information represents the predicted movement direction of the moving target, such as going straight to the left or going straight to the right, etc.

[0103] For example, based on a preset information processing rule, if it is determined that the movement trajectory represented by the movement trend information of the moving target coincides with the position of the target traffic participant, and the time required for the moving target to reach in front of the current vehicle is within a preset time range, it is determined that the traffic safety risk level existing between the moving target and the target traffic participant is high risk.

[0104] 203. Perform an external projection warning process according to the risk level and movement direction information of the moving target.

[0105] Exemplarily, based on the prediction warning rule of the central control unit, analyze and process the risk level and movement direction information of the moving target to determine whether to perform a projection warning process. If it is determined to perform a projection warning process, perform an external projection warning process through the projection warning function based on the central control unit.

[0106] For example, if the central control unit determines that the risk level of the moving target is the preset medium risk or high risk, generate a warning message based on the projection warning function of the central control unit, process according to the movement direction information of the moving target, determine the target position of the external projection warning corresponding to it, and perform a projection display process of the generated warning message at the target position to warn the moving target to pay attention to traffic safety; and / or, according to the warning message, display the corresponding warning information in front of the target traffic participant to remind the corresponding target traffic participant that there is a moving target with a collision risk currently, and they need to increase their attention and slow down.

[0107] It is worth adding that the traffic safety risk level existing between the current vehicle and the moving target can also be determined according to the movement trend information and arrival time of the moving target, and an in-vehicle warning can be performed according to the traffic safety risk level to remind the driver inside the current vehicle to pay attention ahead and drive carefully.

[0108] The vehicle projection warning method provided by the embodiment of the present invention determines the risk level and movement direction by combining the detected movement trend information of the moving target and the time required to reach in front of the current vehicle, and performs an external vehicle projection warning on the corresponding moving target or target traffic participant according to the determined risk level and movement direction, which can remind the moving target or traffic participants around the vehicle to concentrate and control driving, and reduce the occurrence of traffic safety accidents.

[0109] Figure 3 is the flowchart of the vehicle projection warning method provided by an embodiment of the present invention Figure 2 As Figure 3 shown, on the basis of the embodiment shown in Figure 2 , the vehicle projection warning method provided by the present invention is described in detail. The method includes:

[0110] 301. Obtain video data of the environment where the current vehicle is located.

[0111] Exemplarily, Figure 4 is the system composition diagram of the dynamic environment perception and warning system provided by an embodiment of the present invention. As Figure 4 shown, a dynamic environment perception and warning system 1 is deployed in the vehicle. The dynamic environment perception and warning system 1 is composed of a front camera unit 11, a side camera unit 12, a light sensor unit 13, an image processing unit 14, a multimedia unit 15, a central control unit 16, a projection unit 17, a cloud server unit 18, etc. The above units can be inherited or further split according to requirements. This embodiment is described with the above units. Among them, the front camera unit is used to collect image data of the environment in front of the vehicle. A single camera wide-angle / fisheye lens is adopted, with a resolution ≥ 1080p to ensure clear collection of the video data in front of the current vehicle; the horizontal field of view angle ≥ 150° to ensure complete collection of the video information in front of the current vehicle. Distortion correction is required during image processing; the aperture is F / 2.0, and the frame rate ≥ 60fps to ensure the brightness and smoothness of the video data collected in front of the current vehicle; infrared supplementary lighting can also be equipped to meet the extremely low brightness environment. The side camera unit is used to collect image data of the environments on both sides of the current vehicle. Similarly, the same wide-angle / fisheye lens as the front camera unit is adopted. Figure 5 is the working flowchart of the dynamic environment perception and warning system provided by an embodiment of the present invention. As Figure 5 shown, when the vehicle is powered on, the dynamic environment perception and warning system automatically performs hardware self-check and software initialization to ensure that all units work properly; at the same time, camera parameters are calibrated, including focusing, exposure time, etc. The front camera unit continuously collects video data in front of the current vehicle (i.e., the "target vehicle" in Figure 5 ) at a high frame rate, that is, a video stream is obtained to ensure that no potential dangerous situations are missed.

[0112] It should be noted that, in order to reduce the energy consumption of the current vehicle and ensure the driving safety with other participating vehicles, certain working conditions are set. When the current vehicle is stationary in the sentry mode, the DESWS system enters the low-power mode and performs data acquisition through the front camera unit at relatively long intervals, for example, with a period Ts = 3s; when the current vehicle is idling, such as waiting at a traffic light, the front camera unit maintains high sensitivity and performs real-time data acquisition, for example, with a period Ts = 100ms.

[0113] 302. Perform object detection processing on each image in the video data to determine the target coordinate information; among them, the target coordinate information includes the image coordinates of each detected moving target in each image of the video data.

[0114] Exemplarily, based on the central control unit, by applying a deep learning-based object detection model, the moving target positioning and tracking tasks are simultaneously completed in the same network to perform object detection processing on each image in the video data, and record the target coordinate information of the moving target, including the image coordinates of the detected moving target in each image of the video data.

[0115] In a possible implementation manner, step 302 includes:

[0116] Step 1. Perform object detection processing on each image in the video data to determine the initial coordinate information; among them, the initial coordinate information includes the image coordinates of each detected initial target in each image of the video data.

[0117] Step 2. According to the initial coordinate information and the position of the current vehicle, determine the initial target closest to the current vehicle as the moving target, so as to obtain the image coordinates of the moving target in each image of the video data.

[0118] Exemplarily, by applying a deep learning-based object detection model, the moving target positioning and tracking tasks are simultaneously completed in the same network to perform object detection processing on each image in the video data, detect each initial moving target in the video data, that is, each initial target, and record the target coordinate information, including the image coordinates of each detected initial target in each image of the video data. In order to reduce the computational amount, according to the distances between the image coordinates of all initial targets in each image of the video data and the position of the current vehicle, each initial target is screened, and the initial target with the smallest distance, that is, the initial target closest to the current vehicle, is determined as the moving target, and then the image coordinates of the moving target in each image of the video data are obtained.

[0119] For example, in combination with Figure 4, the image processing unit is used to process each image in the acquired video data for recognition. For example, the image processing unit performs preprocessing operations such as noise reduction, contrast enhancement, and distortion correction on each frame of the image to improve the quality of subsequent analysis; then transmits each processed image to the central control unit. Combined with Figure 5 , the central control unit integrates a deep learning-based moving target detection model to complete the tasks of moving target localization and tracking simultaneously in the same network. For example, the preprocessed image is scaled down to a square with a height of 320 * 320 pixels. Compared with the original high-definition image, the reduced size can speed up the calculation speed while retaining sufficient details for recognition; the three-frame difference method is used to judge the moving state of the target, comparing the differences of three consecutive frames (t, t1, t2). Only when there are changes in two consecutive frames is it considered a movement. After obtaining the movement result determination, record the image coordinates of the initial target in each image, and record each initial target with an independent identity document number (Identity Document, abbreviated as ID) to process each initial target separately. At this time, to reduce the calculation amount, the initial targets closest to the current vehicle on both sides of the current vehicle can be determined from each initial target according to the geographical coordinates corresponding to the image coordinates of each initial target and the geographical coordinates of the current vehicle, which are the moving targets, and only calculate for this moving target.

[0120] 303. Determine the movement trend information of the moving target according to the image coordinates of the moving target in each image; wherein, the movement trend information includes the lateral movement speed and the lateral distance, and the lateral distance is the lateral distance between the moving target and the current vehicle.

[0121] Exemplarily, after obtaining the image coordinates of the moving target in each image, the central control unit processes the image coordinates of the moving target in each image through coordinate data processing technology to obtain the movement trend information of the moving target, including the current lateral movement speed and lateral distance of the moving target, wherein the lateral distance is the lateral distance between the moving target and the current vehicle; it also includes the movement trajectory, pose data, etc. of the moving target.

[0122] In a possible implementation manner, step 303 includes:

[0123] The first step: Perform geographical coordinate conversion processing on the image coordinates to obtain the geographical coordinates corresponding to the image coordinates.

[0124] The second step: Determine the movement trend information of the moving target according to all the geographical coordinates corresponding to the moving target.

[0125] Specifically, the central control unit performs geographical coordinate transformation on each image coordinate based on the coordinate transformation technology to obtain the geographical coordinate corresponding to each image coordinate, so as to represent the geographical location of each moving target at each moment. According to the preset algorithm, all the geographical coordinates corresponding to each moving target are calculated and processed to obtain the motion trend information of each moving target, including the lateral moving speed and the lateral distance.

[0126] For example, after the motion result is determined in the previous step, three frames of coordinates of the moving target are recorded, and the image coordinates (u, v), (u1, v1), (u2, v2) are converted into geographical coordinates (x, y), (x1, y1), (x2, y2), so as to obtain the lateral moving speed S of the moving target and the lateral distance D between the moving target and the target vehicle.

[0127] 304. Determine the arrival time of the moving target according to the lateral moving speed and the lateral distance of the moving target.

[0128] Exemplarily, the central control unit performs calculation and processing on the lateral moving speed and the lateral distance of each moving target according to the preset formula to obtain the arrival time of the moving target.

[0129] For example, combined with Figure 5 , the image coordinates of three frames of the moving target are converted into geographical coordinates in the world coordinate system, and the lateral moving speed S of the moving target is calculated as S = ((x - x1) / T + (x1 - x2) / T) / 2, and the lateral distance D between the moving target and the current vehicle is obtained. Thus, the time T for the moving target to reach in front of the target vehicle can be updated periodically and calculated as T = D / S.

[0130] Furthermore, Figure 6 is a flowchart of the conversion between the image coordinate and the geographical coordinate provided by an embodiment of the present invention. As Figure 6 shown, the above conversion of the image coordinate into the geographical coordinate adopts affine transformation; during the design, initial calibration needs to be carried out in advance, 4 points are marked in the image, such as the four corners of the image, and the corresponding pixel coordinates (u1, v1), (u2, v2), (u3, v3), (u4, v4) are obtained; then their corresponding geographical coordinates (x1, y1), (x2, y2), (x3, y3), (x4, y4) are recorded; and then the core conversion equation of the affine transformation is constructed:

[0131]

[0132] Among them, a and e represent the scaling factors from geographical coordinates to image coordinates; a controls the influence of the geographical coordinate in the x direction on the u axis, i.e., horizontal scaling; e controls the influence of the geographical coordinate in the y direction on the image v axis, i.e., vertical scaling; b and d represent the shear factors, i.e., the inclination of the coordinate axes; b reflects the inclination of the geographical coordinate in the y direction on the image u axis, i.e., horizontal shear; d reflects the inclination of the geographical coordinate in the x direction on the image v axis, i.e., vertical shear; c and f identify the translation offsets; c is the overall translation of the image u axis, i.e., horizontal offset; f is the overall translation of the image v axis, i.e., vertical offset; writing this system of equations in matrix form M·p = U is as follows:

[0133]

[0134] Among them, for each reference point (x i , y i ), two rows of equations are generated, corresponding to (u i , v i ). The purpose is to separate a, b, c, d, e, f into two sets of linear equations to process the mappings of u and v respectively:

[0135]

[0136] The first 3 columns correspond to the coefficients a, b, c of u = ax + by + c; the last 3 columns correspond to the coefficients d, e, f of v = dx + ey + f; through the matrix equation M·p = U, a, b, c, d, e, f are obtained; where p = [a, b, c, d, e, f] T : the vector of affine transformation parameters to be solved; U = [u1, v1, u2, v2,..., u4, v4] T : the known coordinate observations. To inversely deduce the geographical coordinates (x, y) from the image coordinates (u, v), the geographical coordinate formula is solved in reverse as follows:

[0137]

[0138] 305. Determine the risk level of the moving target according to the arrival time and target type of the moving target.

[0139] Exemplarily, in combination with Figure 5 , for each moving target, the central control unit first determines the target type of each moving target, such as including vehicles, pedestrians, bicycles, and combines the target type, movement trend of the moving target, and the time T to reach in front of the current vehicle to perform a comprehensive risk level determination process to obtain the risk level of the moving target, such as one of the three risk levels of low risk, medium risk, and high risk.

[0140] For example, each preset risk level corresponds to each combination of a target type, a movement trend, and a preset time range. Based on the arrival time T, if it is determined that the arrival time T is within the corresponding preset time range 1, then the traffic safety risk level between the target traffic participant and the moving target can be determined by combining the target type, the movement trend of the moving target, and the corresponding preset time range 1. It is worth adding that, through the target type of each moving target, such as including vehicles, pedestrians, and bicycles, the traffic safety risk level between the current vehicle and the moving target can be obtained by comprehensively determining the risk level in combination with the target type, the movement trend of the moving target, and the time T to reach in front of the current vehicle.

[0141] In one possible implementation, step 305 includes:

[0142] The first step: Determine the compensation coefficient corresponding to the target type according to the target type of the moving target.

[0143] The second step: Determine the comparison time corresponding to the moving target according to the compensation coefficient corresponding to the target type.

[0144] The third step: Determine the risk level of the moving target according to the arrival time and the comparison time of the moving target.

[0145] Specifically, the central control unit can determine the compensation coefficient corresponding to the target type of the moving target according to the mapping relationship between each preset target type of the moving target and each compensation coefficient; according to a preset formula, perform calculation processing on the compensation coefficient corresponding to the moving target to determine the comparison time Tr corresponding to the moving target, and compare the comparison time Tr of the moving target with the arrival time T, then the traffic safety risk level between the moving target and the target traffic participant can be judged.

[0146] For example, when T ≤ Tr = xb * 2s, it is determined as a high risk; when T ≤ Tr = xb * 5s, it is determined as a medium risk; when T ≤ Tr = xb * 10s, it is determined as a low risk; where xb is the compensation coefficient. Because the uncertainties are different for different moving target types, it is necessary to set a certain defined time weight; for example, when the moving target type is a motor vehicle, xb = 1.0, when the moving target type is a pedestrian, xb = 1.3, and when the moving target type is a bicycle, xb = 1.2.

[0147] 306. Determine the movement direction information of the moving target according to the movement trend information of the moving target.

[0148] Exemplarily, for the motion trend information of a moving target, such as motion trajectory data, the central control unit can perform model processing on the motion trajectory data of the moving target through a prediction model, and can predict the motion direction of the moving target, including motion direction information such as going straight in the direction approaching the vehicle and going straight in the direction away from the vehicle.

[0149] 307. If it is determined that the risk level of the moving target is a preset level, determine the risk information of the moving target; wherein, the risk information includes the image of the moving target, the lateral distance in the motion trend information, the risk level, the arrival time, and the motion direction information; the lateral distance is the lateral distance between the moving target and the current vehicle.

[0150] Exemplarily, the central control unit can compare the determined risk level of the moving target with the target traffic participant and the preset level. If it is determined that the risk level of the moving target is a preset level, such as medium risk or high risk, then generate the risk information of the moving target, including relevant information of the moving target such as the image of the moving target, the lateral distance in the motion trend information, the risk level, the arrival time, and the motion direction information. The lateral distance is the lateral distance between the moving target and the current vehicle, for further processing.

[0151] 308. Send the risk information of the moving target to the projection unit; wherein, the projection unit is used to generate a first warning message according to the risk information and perform out-of-vehicle projection processing on the first warning message.

[0152] Exemplarily, combined with Figure 4 , Figure 5 , the dynamic environment perception and warning system further includes a projection unit. The central control unit can be used for moving target detection, data calculation and processing, determining the projection trigger signal, issuing projection instructions, etc. The projection unit is used to execute the projection instructions and display the corresponding side projection content according to the moving direction of the moving target. For example, a digital light processing (DLP) high-pixel color projection lamp or a high-definition black-and-white projection lamp is selected, and the brightness is adjustable within the range of 300 lm - 2000 lm. That is, the central control unit can send the sorted risk information of the moving target to the projection unit, so that the projection unit processes the risk information, selects different projection schemes, can generate the first warning message corresponding to the risk level in the risk information, and perform out-of-vehicle projection processing on the first warning message. Furthermore, when a risk is detected, a warning pattern or warning information is immediately projected to provide warning information to the participants around the vehicle and improve road safety.

[0153] For example, if the central control unit confirms that the potential traffic safety risk level between the moving target and the current vehicle is high, it will send information such as the moving target image, risk level, lateral distance D between the moving target and the current vehicle, moving direction, and the time T when the moving target reaches the front of the current vehicle to the corresponding side projection unit; of course, when the risk level does not change for two consecutive times, only the time T when the moving target reaches the front of the target vehicle needs to be updated, and other information does not need to be updated; the projection unit projects the pre-set warning pattern according to the risk level onto the appropriate position on the corresponding side of the ground within the shortest time. The default projection area size P is 2.5m * 2m, and the default projection brightness L = 1000lm.

[0154] For another example, Figure 7 is a composition diagram of the default projection scheme provided by an embodiment of the present invention, Figure 8 is an example diagram of the projection scheme provided by an embodiment of the present invention. As Figure 7 , Figure 8 shown, first, the moving target image is used to generate a 3D dynamic map through artificial intelligence. The main body of the picture is a 3D dynamically displayed zebra crossing. The moving target dynamic map is on the left or right side of the zebra crossing (determined according to the moving direction). A collision warning icon is displayed and flashing in front of the moving target dynamic map. The lateral distance D between the moving target and the target vehicle and the time S (estimated collision time) when the moving target reaches the front of the target vehicle are displayed in front of the collision warning icon, and are updated periodically according to image acquisition. According to high-risk, medium-risk, and low-risk levels, further text prompts are added behind the main zebra crossing (i.e., in the direction of the vehicle behind). Red text "Decelerate immediately" for high risk, yellow text "Slow down and proceed" for medium risk, and blue text "Pay attention and observe" for low risk to remind the vehicle behind that there are pedestrians ahead and to pay attention to driving. Among them, the warning pattern has a default projection scheme according to different risk levels. Combining Figure 4 , the multimedia unit can be used for users to select and customize projection patterns or video content, and can also be used to adjust the projection brightness of the projection lamp and control active projection. Furthermore, users can customize the projection content to improve the flexibility and satisfaction of use.

[0155] In a possible implementation manner, step 308 includes:

[0156] Step 1: Determine the target projection position according to the movement direction information of the moving target.

[0157] Step 2: If it is determined that there are no fixed obstacles between the target projection position and the position of the moving target, send the target projection position to the projection unit; wherein, the projection unit is used to perform out-of-vehicle projection processing on the moving target.

[0158] Exemplarily, to improve the accuracy of projection warning, when it is determined that the potential traffic safety risk level between the moving target and the target traffic participant is high and waiting for projection warning, the target projection position can be determined according to the movement direction information of the moving target, that is, the projection position where the target traffic participant or the moving target needs to be warned currently, and the target projection position, the position of the moving target, and the movement direction of the moving target are used to detect through the target detection system in the current vehicle to determine whether there are fixed obstacles in the traffic road between the target projection position and the position of the moving target and conforming to the movement direction of the moving target, such as road guardrails, concrete piers or plastic piers that restrict vehicle passage. If it is determined that there are no fixed obstacles between the target projection position and the position of the moving target, the target projection position of the moving target is sent to the projection unit so that the projection unit can perform out-of-vehicle projection warning processing on the moving target according to the target projection position.

[0159] In a possible implementation, it further includes: obtaining the light data and ground data of the environment where the current vehicle is located; where the ground data includes the ground attribute data of the environment where the current vehicle is located; determining the actual projection brightness according to the light data and the ground data; where the actual projection brightness is used for out-of-vehicle projection warning processing.

[0160] Exemplarily, for the visibility of the projection warning information, the central control unit needs to consider the brightness of the projections on both sides in different environments, that is, combined with Figure 4 , Figure 5 , the light data of the environment where the current vehicle is located is obtained through the light sensor unit to characterize the light brightness of the side environment of the current vehicle, and the ground data of the environment where the current vehicle is located is obtained, including the ground attribute data of the environment where the current vehicle is located, such as dark asphalt ground, slippery road surface, smooth cement road surface, etc. Based on the preset analysis rules or deep learning model, the light brightness of the side environment of the current vehicle and the ground data of the environment where the current vehicle is located are processed to determine the actual projection brightness. The actual projection brightness is used for out-of-vehicle projection warning processing. For example, the central control unit can generate a corresponding projection brightness instruction based on the actual projection brightness and send it to the projection unit, so that when the projection unit performs out-of-vehicle projection warning processing, the projection brightness of the displayed warning information can be adjusted to the determined actual projection brightness. Furthermore, the visibility of the projected warning information can be improved, enabling traffic participants such as pedestrians around the current vehicle to notice the warning information in time and improving traffic safety.

[0161] In a possible implementation, determining the actual projection brightness according to the light data and the ground data includes: determining the material gain coefficient corresponding to the ground data according to the ground data; determining the actual projection brightness according to the light data and the material gain coefficient.

[0162] Specifically, according to the mapping relationship between the preset ground data and the material gain coefficient, the central control unit can determine the material gain coefficient corresponding to the obtained ground data, and based on the preset calculation formula, calculate and process the material gain coefficient and the light data to obtain the actual projection brightness. Furthermore, the visibility of the projected warning information can be further improved, enabling traffic participants such as pedestrians around the current vehicle to notice the warning information in a timely manner, and further enhancing traffic safety.

[0163] For example, the actual projection brightness L_s = default brightness L * (1 + 0.5 * (1 - actual ambient illuminance / standard natural illuminance)) * material gain coefficient A, where for the material gain coefficient A, for example, dark asphalt corresponds to 1.8, slippery road surface corresponds to 1.2, smooth cement road corresponds to 0.8, etc., and the actual projection brightness L_s does not exceed the upper or lower limit value of the projection lamp.

[0164] In a possible implementation manner, it further includes: obtaining a side environmental image of the environment where the current vehicle is located; and performing recognition processing on the side environmental image to obtain the flat area ratio; where the flat area ratio is the ratio of the flat ground area in the side environmental image; performing conversion processing on the flat area ratio to obtain the flat area size; where the flat area size is the actual size of the available flat ground area in the environment represented by the side environmental image; determining the projection area size according to the flat area size; where the projection area size is used for performing external vehicle projection warning processing.

[0165] Exemplarily, combined with Figure 5 , the central control unit collects side environmental image data through the side camera unit, that is, obtains the side environmental image, and based on the preset recognition model, performs recognition processing on the side environmental image to obtain the ratio of the flat ground area in the side environmental image, that is, the flat area ratio. Through image processing technology, perform conversion processing on the flat area ratio to obtain the actual size of the available flat ground area in the environment represented by the side environmental image, that is, the flat area size, and then the external vehicle projection warning processing can be performed according to the flat area size. Furthermore, the integrity of the projected warning information can be further improved, enabling traffic participants such as pedestrians around the current vehicle to notice the warning information in a timely manner, and further enhancing traffic safety.

[0166] For example, the side environment image is collected by the side camera unit, and the image preprocessing is performed on the side environment image by the image processing unit; then, through the recognition model of the central control unit, the preprocessed side environment image is recognized to identify the proportion of the flat area in the environment image, and then it is converted into the actual size P1 of the available flat area in the world map. According to the actual size P1, it is determined whether to project and the projection ratio; for example, when P1 ≥ P (preset size) (both width and height need to be satisfied), the default size is projected completely; when 0.3P ≤ P1 < P (either width or height meets the condition), it is projected in proportion; when P1 < 0.3P (either width or height meets the condition), it is not projected.

[0167] Based on the above embodiments, the vehicle projection warning method provided by the embodiments of the present invention can quickly and accurately identify moving objects in front of the vehicle, especially pedestrians and other moving obstacles, and immediately project a warning pattern or warning information to traffic participants around the current vehicle when a risk is detected, providing warning information to traffic participants around the vehicle and improving road safety.

[0168] Figure 9 The flowchart of the vehicle projection warning method provided by an embodiment of the present invention Figure 3 As Figure 9 shown, on the basis of the embodiments shown in Figure 2 and 3 shown, the vehicle projection warning method provided by the present invention is described in detail. The method includes:

[0169] Step 401: Obtain the video data of the environment where the current vehicle is located; and perform target detection processing on the video data to obtain target movement information; wherein, the target movement information includes the movement trend information and arrival time of the detected moving target; the arrival time is the time required for the moving target to reach in front of the current vehicle.

[0170] It should be noted that this step is similar to the foregoing step 201 and will not be elaborated here.

[0171] Step 402: Determine the risk level and movement direction information of the moving target according to the movement trend information and arrival time of the moving target; wherein, the risk level is the traffic safety risk level existing between the target traffic participant and the moving target.

[0172] It should be noted that this step is similar to the foregoing step 202 and will not be elaborated here.

[0173] Step 403: Perform out-of-vehicle projection warning processing according to the risk level and movement direction information of the moving target.

[0174] It should be noted that this step is similar to the foregoing step 203 and will not be elaborated here.

[0175] Step 404: Generate early warning event record data; and send the early warning event record data to the cloud server unit; wherein, the early warning event record data includes the image of the moving target, the lateral distance in the movement trend information, the risk level, the arrival time, and the movement direction information; the lateral distance is the lateral distance between the moving target and the current vehicle.

[0176] Exemplarily, in combination with Figure 4 , Figure 5 , the dynamic environment perception and warning system further includes a cloud server unit. After triggering an early warning event, that is, after performing the out-of-vehicle projection warning, the central control unit will automatically record information such as the image of the moving target, the risk level, the lateral distance D between the moving target and the target vehicle, the moving direction, and the time T when the moving target reaches in front of the target vehicle before and after each triggering of the early warning event, organize it into early warning event record data, and upload it to the cloud server unit for self-learning optimization and improvement of the cloud model.

[0177] Furthermore, in combination with Figure 5 , the cloud server unit regularly trains a new model version using a machine learning platform and pushes it to all connected DESWS devices through Over-the-Air Technology (OTA) to keep the system in the latest state.

[0178] Step 405: Obtain the driving data of the current vehicle; and send the driving data of the current vehicle to the cloud server unit; wherein, the cloud server unit is used to generate the vehicle distance of at least one target vehicle based on the driving data of the current vehicle and the driving data of at least one other vehicle and send it to the target vehicle; the target vehicle is another vehicle whose distance from the rear of the current vehicle is within a preset distance range and whose driving direction is the same as that of the current vehicle; the vehicle distance is the actual distance between the current vehicle and the target vehicle; the vehicle distance is used for in-vehicle warning processing.

[0179] Exemplarily, a communication module between vehicles and everything (Vehicle-to-Everything, V2X for short) is added to the dynamic environment perception and warning system, enabling it to communicate with other vehicles or infrastructure such as traffic lights. The multi-vehicle information sharing platform in the cloud server unit is a system for all traffic participating vehicles to share information and process information. All traffic participating vehicles can register and access it. Moreover, the participating vehicles accessing the cloud multi-vehicle information sharing platform system periodically report vehicle driving data such as their own coordinates and moving directions. That is, when the current vehicle is one of the numerous participating vehicles, the vehicle driving data of the current vehicle, including vehicle driving coordinates, driving speed, pose information, etc., can be obtained through the central control unit of the current vehicle, and the vehicle driving data of the current vehicle is sent to the cloud server unit. When the cloud server unit receives the vehicle driving data of the current vehicle, including the vehicle driving data of other vehicles, and processes the vehicle driving data of all vehicles, other vehicles whose distance behind the current vehicle is within a preset distance range and whose driving direction is the same as that of the target vehicle are selected as the target vehicles. At this time, the target traffic participant is the target vehicle, and the actual distance between each target vehicle and the current vehicle, that is, the target vehicle distance, can be calculated and sent to each target vehicle. The target vehicle can receive the target vehicle distance sent by the cloud server unit and process it accordingly. The central control unit can receive the actual distance between the current vehicle and the target vehicle sent by the cloud server unit by accessing the V2X communication module. The target vehicle analyzes the target vehicle distance based on a preset warning rule to determine whether in-vehicle warning processing is required currently; for example, combined with Figure 4 , when the target vehicle distance is less than the preset threshold, in-vehicle warning processing is performed through the multimedia of the target vehicle.

[0180] For example, Figure 10 FIG. is a flowchart of the operation of multi-vehicle interconnection and warning provided by an embodiment of the present invention. Combined with Figure 10 ( Figure 10(where the target vehicle in it is the current vehicle), after the central control unit uploads information such as the moving target image, risk level, lateral distance D between the moving target and the current vehicle, moving direction, and the time T for the moving target to reach in front of the current vehicle to the cloud server unit, the cloud multi-vehicle information sharing platform system of the cloud server unit will, through the coordinates of the current vehicle and the rear vehicle, screen out the participating vehicles traveling in the same direction within 100 m behind the current vehicle, and calculate the actual distance between each participating vehicle and the current vehicle. The cloud multi-vehicle information sharing platform system will distribute the actual distance between each of the above participating vehicles and the current vehicle to the corresponding participating vehicles. After receiving the coordinate information, the participating vehicle calculates the time T1 for the participating vehicle to reach in front of the current vehicle based on the speed information of the participating vehicle and the actual distance from the current vehicle, and compares the time T1 for the participating vehicle to reach in front of the current vehicle with the time T for the moving target to reach in front of the current vehicle. When T≤T1, a warning message is displayed on the multimedia unit in the participating vehicle to prompt the driver of the participating vehicle to pay attention to the blind area.

[0181] On the basis of the foregoing embodiments, in this embodiment, by adding V2X linkage through the DESWS system, other participants around the current vehicle can more conveniently obtain blind spot information during driving, and can effectively remind the rear traffic participants, so as to improve traffic safety.

[0182] Figure 11 is a schematic structural diagram of a simulated driving control device provided by an embodiment of the present invention. As Figure 11 shown, the simulated driving control device of the embodiment of the present invention is applied to the central control unit in the dynamic environment perception and warning system; the device includes:

[0183] A detection module 501, configured to obtain video data of the environment where the current vehicle is located; and perform target detection processing on the video data to obtain target movement information; wherein, the target movement information includes the movement trend information and arrival time of each detected moving target; the arrival time is the time required for the moving target to reach in front of the current vehicle;

[0184] A determination module 502, configured to determine the risk level and movement direction information of the moving target according to the movement trend information and arrival time of the moving target; wherein, the risk level is the traffic safety risk level existing between the target traffic participant and the moving target;

[0185] A warning module 503, configured to perform external projection warning processing according to the risk level and movement direction information of the moving target.

[0186] Further, the determination module 502 is specifically configured to: determine the risk level of the moving target according to the arrival time and target type of the moving target; determine the movement direction information of the moving target according to the movement trend information of the moving target.

[0187] Further, the determination module 502 is also specifically configured to: determine the compensation coefficient corresponding to the target type according to the target type of the moving target; determine the comparison time corresponding to the moving target according to the compensation coefficient corresponding to the target type; determine the risk level of the moving target according to the arrival time and comparison time of the moving target.

[0188] Further, the detection module 501 is specifically configured to: perform target detection processing on each image in the video data to determine the target coordinate information; wherein, the target coordinate information includes the image coordinates of each detected moving target in each image of the video data; determine the movement trend information of the moving target according to the image coordinates of the moving target in each image; wherein, the movement trend information includes the lateral movement speed and the lateral distance, and the lateral distance is the lateral distance between the moving target and the current vehicle; determine the arrival time of the moving target according to the lateral movement speed and the lateral distance of the moving target.

[0189] Further, the detection module 501 is also specifically configured to: perform target detection processing on each image in the video data to determine the initial coordinate information; wherein, the initial coordinate information includes the image coordinates of each detected initial target in each image of the video data; determine the initial target closest to the current vehicle as the moving target according to the initial coordinate information and the position of the current vehicle, so as to obtain the image coordinates of the moving target in each image of the video data.

[0190] Further, the detection module 501 is also specifically configured to: perform geographic coordinate conversion processing on the image coordinates to obtain the geographic coordinates corresponding to the image coordinates; determine the movement trend information of the moving target according to all the geographic coordinates corresponding to the moving target.

[0191] Further, the dynamic environment perception and warning system further includes a projection unit; the warning module 503 is specifically configured to: if it is determined that the risk level of the moving target is a preset level, determine the risk information of the moving target; wherein, the risk information includes the image of the moving target, the lateral distance in the movement trend information, the risk level, the arrival time, and the movement direction information; the lateral distance is the lateral distance between the moving target and the current vehicle; send the risk information of the moving target to the projection unit; wherein, the projection unit is configured to generate a first warning information according to the risk information and perform an external vehicle projection process on the first warning information.

[0192] Further, before the warning module 503 is used to send the risk information of the moving target to the projection unit, the warning module is further used to: determine the target projection position according to the movement direction information of the moving target; if it is determined that there is no fixed obstacle between the target projection position and the position of the moving target, send the target projection position to the projection unit; wherein, the projection unit is used to perform an external vehicle projection process on the moving target.

[0193] Further, the device is further used to: obtain the light data and ground data of the environment where the current vehicle is located; wherein, the ground data includes the ground attribute data of the environment where the current vehicle is located; determine the actual projection brightness according to the light data and the ground data; wherein, the actual projection brightness is used for performing an external vehicle projection warning process.

[0194] Further, the device is specifically further used to: determine the material gain coefficient corresponding to the ground data according to the ground data; determine the actual projection brightness according to the light data and the material gain coefficient.

[0195] Further, the device is further used to: obtain the side environment image of the environment where the current vehicle is located; perform an identification process on the side environment image to obtain the flat area ratio; wherein, the flat area ratio is the ratio of the flat ground area in the side environment image; perform a conversion process on the flat area ratio to obtain the flat area size; wherein, the flat area size is the actual size of the available flat ground area in the environment represented by the side environment image; determine the projection area size according to the flat area size; wherein, the projection area size is used for performing an external vehicle projection warning process.

[0196] Further, the dynamic environment perception and warning system further includes a cloud server unit; the device is further used to: obtain the driving data of the current vehicle; and send the driving data of the current vehicle to the cloud server unit; wherein, the cloud server unit is used to generate the vehicle distance of at least one target vehicle according to the driving data of the current vehicle and the driving data of at least one other vehicle and send it to the target vehicle; the target vehicle is another vehicle whose distance from the rear of the current vehicle is within a preset distance range and whose driving direction is the same as that of the current vehicle; the vehicle distance is the actual distance between the current vehicle and the target vehicle; the vehicle distance is used for performing an in-vehicle warning process.

[0197] Further, the dynamic environment perception and warning system further includes a cloud server unit; after the warning module is used to perform an external vehicle projection warning process according to the risk level and movement direction information of the moving target, the device is further used to: generate warning event record data; and send the warning event record data to the cloud server unit; wherein, the warning event record data includes the image of the moving target, the lateral distance in the movement trend information, the risk level, the arrival time, and the movement direction information; the lateral distance is the lateral distance between the moving target and the current vehicle.

[0198] The device according to an embodiment of the present invention can be used to execute the technical solutions of any of the above - shown method embodiments. Its implementation principle and technical effects are similar, and will not be elaborated here.

[0199] Figure 12 It is a schematic structural diagram of a central control unit provided by an embodiment of the present invention. As Figure 12 shown, the central control unit may include: at least one processor 601 and a memory 602.

[0200] The memory 602 is used to store programs. Specifically, the program may include program code, and the program code includes computer - executable instructions.

[0201] The memory 602 may include a high - speed random access memory (Random Access Memory, abbreviated as RAM), and may also include non - volatile memory, such as at least one disk memory.

[0202] The processor 601 is used to execute the computer - executable instructions stored in the memory 602 to implement the application running method based on intelligent driving described in the foregoing method embodiments. Among them, the processor 601 may be a central processing unit (Central Processing Unit, abbreviated as CPU), or an application - specific integrated circuit (Application Specific Integrated Circuit, abbreviated as ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention. Specifically, when implementing the application running method based on intelligent driving described in the foregoing method embodiments, the central control unit may be, for example, an electronic control unit, a server, or a domain controller, etc. with processing functions on a vehicle.

[0203] Optionally, the central control unit may further include a receiver 603 and a transmitter 604. In a specific implementation, if the receiver 603, transmitter 604, memory 602, and processor 601 are implemented independently, the receiver 603, transmitter 604, memory 602, and processor 601 may be interconnected through a bus and communicate with each other. The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc., but it does not mean that there is only one bus or one type of bus.

[0204] Optionally, in a specific implementation, if the receiver 603, the transmitter 604, the memory 602, and the processor 601 are integrated on a single chip, the receiver 603, the transmitter 604, the memory 602, and the processor 601 can communicate through an internal interface.

[0205] The present invention also provides a dynamic environment perception and warning system, including a central control unit, and the central control unit is used to execute the vehicle projection warning method implemented as above.

[0206] The present invention also provides a vehicle, including a dynamic environment perception and warning system, and the dynamic environment perception and warning system includes a central control unit; the central control unit is used to implement the vehicle projection warning method as above.

[0207] The present invention also provides a computer-readable storage medium, in which computer program instructions are stored, and when the processor executes the computer program instructions, the solutions in the above embodiments are implemented.

[0208] The present invention also provides a computer program product, including a computer program, and when the computer program is executed by a processor, the solutions in the above embodiments are implemented.

[0209] The above-mentioned computer-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM for short), Electrically Erasable Programmable Read Only Memory (EEPROM for short), Erasable Programmable Read Only Memory (EPROM for short), Programmable Read Only Memory (PROM for short), Read Only Memory (ROM for short), magnetic memory, flash memory, a magnetic disk, or an optical disc. The readable storage medium can be any available medium accessible by a general-purpose or special-purpose computer.

[0210] An exemplary readable storage medium is coupled to the processor, so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application-specific integrated circuit. Of course, the processor and the readable storage medium can also exist as discrete components in an application running device based on intelligent driving.

[0211] Those of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions. The foregoing program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps including the above method embodiments; and the foregoing storage medium includes: various media such as ROM, RAM, magnetic disk, or optical disc that can store program codes.

[0212] Finally, it should be noted that the above embodiments are only preferred embodiments given to fully illustrate the present invention, and the protection scope of the present invention is not limited thereto. Equivalent substitutions or transformations made by those skilled in the art on the basis of the present invention are all within the protection scope of the present invention.

Claims

1. A vehicle projection warning method, characterized in that, The method is applied to a central control unit in a dynamic environment perception and warning system; the method includes: Obtain video data of the current vehicle's environment; and perform object detection processing on the video data to obtain object movement information; wherein, the object movement information includes the movement trend information and arrival time of the detected moving object; the arrival time is the time required for the moving object to reach in front of the current vehicle; determine the risk level and movement direction information of the moving object according to the movement trend information and arrival time of the moving object; wherein, the risk level is the traffic safety risk level existing between the target traffic participant and the moving object. Perform an external projection warning process according to the risk level and movement direction information of the moving object.

2. The method according to claim 1, wherein The determining the risk level and movement direction information of the moving object according to the movement trend information and arrival time of the moving object includes: Determine the risk level of the moving object according to the arrival time and object type of the moving object; determine the movement direction information of the moving object according to the movement trend information of the moving object.

3. The method according to claim 2, wherein The determining the risk level of the moving object according to the arrival time and object type of the moving object includes: Determine the compensation coefficient corresponding to the object type according to the object type of the moving object; Determine the comparison time corresponding to the moving object according to the compensation coefficient corresponding to the object type; Determine the risk level of the moving object according to the arrival time and comparison time of the moving object.

4. The method according to claim 1, wherein The performing object detection processing on the video data to obtain object movement information includes: Perform object detection processing on each image in the video data to determine object coordinate information; wherein, the object coordinate information includes the image coordinates of the detected moving object in each image of the video data. Determine the movement trend information of the moving object according to the image coordinates of the moving object in each image; wherein, the movement trend information includes the lateral movement speed and lateral distance, and the lateral distance is the lateral distance between the moving object and the current vehicle. Determine the arrival time of the moving object according to the lateral movement speed and lateral distance of the moving object.

5. The method according to claim 4, wherein The performing object detection processing on each image in the video data to determine object coordinate information includes: Perform object detection processing on each image in the video data to determine initial coordinate information; wherein, the initial coordinate information includes the image coordinates of each detected initial object in each image of the video data. Determine the initial object closest to the current vehicle as the moving object according to the initial coordinate information and the position of the current vehicle, so as to obtain the image coordinates of the moving object in each image of the video data.

6. The method according to claim 4, wherein The determining the movement trend information of the moving object according to the image coordinates of the moving object in each image includes: Perform geographic coordinate conversion processing on the image coordinates to obtain the geographic coordinates corresponding to the image coordinates. Determine the motion trend information of the moving target according to all the geographical coordinates corresponding to the moving target.

7. The method according to claim 1, characterized in that, The dynamic environment perception and warning system further includes a projection unit; the performing of the out-of-vehicle projection warning process according to the risk level and the motion direction information of the moving target includes: If it is determined that the risk level of the moving target is a preset level, determine the risk information of the moving target; wherein, the risk information includes the image of the moving target, the lateral distance in the motion trend information, the risk level, the arrival time, and the motion direction information; the lateral distance is the lateral distance between the moving target and the current vehicle; Send the risk information of the moving target to the projection unit; wherein, the projection unit is used to generate a first warning information according to the risk information and perform an out-of-vehicle projection process on the first warning information.

8. The method according to claim 7, wherein Before sending the risk information of the moving target to the projection unit, it further includes: Determine a target projection position according to the motion direction information of the moving target; If it is determined that there is no fixed obstacle between the target projection position and the position of the moving target, send the target projection position to the projection unit; wherein, the projection unit is used to perform an out-of-vehicle projection process on the moving target.

9. The method according to claim 1, characterized in that, The method further includes: Obtain the light data and the ground data of the environment where the current vehicle is located; wherein, the ground data includes the ground attribute data of the environment where the current vehicle is located; Determine the actual projection brightness according to the light data and the ground data; wherein, the actual projection brightness is used for the out-of-vehicle projection warning process.

10. The method according to claim 9, wherein The determining of the actual projection brightness according to the light data and the ground data includes: Determine the material gain coefficient corresponding to the ground data according to the ground data; Determine the actual projection brightness according to the light data and the material gain coefficient.

11. The method according to claim 1, wherein The method further includes: Obtain the side environment image of the environment where the current vehicle is located; and perform an identification process on the side environment image to obtain the flat area ratio; wherein, the flat area ratio is the ratio of the flat ground area in the side environment image; Perform a conversion process on the flat area ratio to obtain the flat area size; wherein, the flat area size is the actual size of the available flat ground area in the environment represented by the side environment image; Determine the projection area size according to the flat area size; wherein, the projection area size is used for the out-of-vehicle projection warning process.

12. The method according to any one of claims 1-11, characterized in that, The dynamic environment perception and warning system further includes a cloud server unit; the method further includes: Obtain the driving data of the current vehicle; and send the driving data of the current vehicle to the cloud server unit; wherein, the cloud server unit is used to generate the vehicle distance of at least one target vehicle according to the driving data of the current vehicle and the driving data of at least one other vehicle, and send it to the target vehicle; the target vehicle is another vehicle whose distance from the rear of the current vehicle is within a preset distance range and whose driving direction is the same as that of the current vehicle; the vehicle distance is the actual distance between the current vehicle and the target vehicle; the vehicle distance is used for in-vehicle warning processing.

13. The method according to any one of claims 1-11, characterized in that, The dynamic environment perception and warning system further includes a cloud server unit; after performing the out-of-vehicle projection warning processing according to the risk level and movement direction information of the moving target, the method further includes: Generate early warning event record data; and send the early warning event record data to the cloud server unit; wherein, the early warning event record data includes the image of the moving target, the lateral distance, risk level, arrival time and movement direction information in the movement trend information; the lateral distance is the lateral distance between the moving target and the current vehicle.

14. A vehicle projection warning device, characterized in that, The device is applied to the central control unit in the dynamic environment perception and warning system; the device includes: A detection module, configured to obtain video data of the environment where the current vehicle is located; and perform target detection processing on the video data to obtain target movement information; wherein, the target movement information includes the movement trend information and arrival time of the detected moving target; the arrival time is the time required for the moving target to reach in front of the current vehicle. A determination module, configured to determine the risk level and movement direction information of the moving target according to the movement trend information and arrival time of the moving target; wherein, the risk level is the traffic safety risk level existing between the target traffic participant and the moving target. A warning module, configured to perform out-of-vehicle projection warning processing according to the risk level and movement direction information of the moving target.

15. A central control unit, characterized in that, Comprising: A memory, a processor; The memory stores computer execution instructions; The processor executes the computer execution instructions stored in the memory, so that the processor executes the method according to any one of claims 1-13.

16. A dynamic environment perception and warning system, characterized in that Comprising a central control unit, and the central control unit is configured to execute the method according to any one of claims 1-13.

17. A vehicle, characterized in that, Comprising a dynamic environment perception and warning system, and the dynamic environment perception and warning system includes a central control unit; the central control unit is configured to execute the method according to any one of claims 1-13.

18. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer execution instructions, and when the computer execution instructions are executed by a processor, they are used to implement the method according to any one of claims 1-13.

19. A computer program product, characterized in that, Comprising a computer program, and when the computer program is executed by a processor, it implements the method according to any one of claims 1-13.

Images