Early warning method, device, equipment and readable storage medium

Abstract

This application is applicable to the field of computer technology, and provides an early warning method, device, equipment and storage medium. By obtaining the environmental information at the current moment within a preset range, and according to the environmental information and the motion model corresponding to the target traffic participant, determining the predicted trajectory of the target traffic participant within a preset duration starting from the current moment, and at the same time obtaining the predicted trajectories of other traffic participants within a preset duration starting from the current moment. Furthermore, when there is an intersection between the predicted trajectory of the target traffic participant and the predicted trajectories of other traffic participants, an early warning message is sent, so that the early warning message sent to the target traffic participant is sent when there is an intersection between the predicted trajectory of the target traffic participant within a preset duration starting from the current moment and the predicted trajectories of other traffic participants, avoiding the occurrence of traffic accidents caused by the failure to timely remind traffic participants.

Description

Technical Field

[0001] This application relates to the field of computer technology, and particularly to an early warning method, device, equipment, and storage medium. Background Art

[0002] With the rapid growth of the number of automobiles in use, automobile safety issues have received increasing attention. In order to improve the safety of vehicle driving and reduce the occurrence of traffic accidents, early warning of traffic collisions will become an important method to improve driving safety. Among them, traffic collisions include not only collisions between vehicles but also collisions between vehicles and pedestrians.

[0003] Traditional early warning methods usually use sensors such as millimeter-wave radars and infrared detectors to detect the distances between various traffic participants, and send early warning messages to traffic participants when the distance between traffic participants is less than a preset distance threshold, so as to avoid traffic safety problems caused by too close distances.

[0004] However, when using traditional early warning methods, if the speed of traffic participants is too fast and early warning messages are sent only when the distance between traffic participants is less than a preset distance threshold, traffic participants cannot be reminded in time, which may lead to the occurrence of traffic accidents. Summary of the Invention

[0005] This application provides an early warning method, device, equipment, and computer-readable storage medium, which can remind traffic participants in time.

[0006] In a first aspect, an embodiment of this application provides an early warning method, including:

[0007] Obtain the environmental information at the current moment within a preset range; the environmental information includes the rotational speed, position information, and driving speed of traffic participants within the preset range;

[0008] According to the environmental information and the motion model corresponding to the target traffic participant, determine the predicted trajectory of the target traffic participant within a preset duration starting from the current moment; the target traffic participant is at least one of the traffic participants in the environmental information;

[0009] Obtain the predicted trajectories of other traffic participants within a preset duration starting from the current moment; the other traffic participants are the traffic participants in the environmental information except the target traffic participant;

[0010] When there is an intersection between the predicted trajectory of the target traffic participant and the predicted trajectories of other traffic participants, send an early warning message.

[0011] In one embodiment, before determining the motion trajectory of the target traffic participant within a preset duration starting from the current moment according to the environmental information and the motion model corresponding to the target traffic participant, the method further includes:

[0012] Determine the motion model corresponding to the target traffic participant according to whether the rotational speed of the target traffic participant is 0.

[0013] In one embodiment, when the rotational speed of the target traffic participant is 0, the motion model corresponding to the target traffic participant is the first formula; the first formula includes:

[0014]

[0015] Wherein, represents the position of the traffic participant at the next moment, g(x(t)) represents the predicted position of the traffic participant at the current moment, v represents the driving speed of the traffic participant at the current moment, θ represents the circular motion angle of the traffic participant at the current moment, x(t) represents the coordinate of the traffic participant on the X-axis at the current moment, y(t) represents the coordinate of the traffic participant on the Y-axis at the current moment, ω represents the rotational speed of the traffic participant at the current moment, and Δt represents the time interval between the current moment and the next moment;

[0016] When the rotational speed of the target traffic participant is not 0, the motion model corresponding to the target traffic participant is the second formula; the second formula includes:

[0017]

[0018] Wherein, represents the position of the traffic participant at the next moment, g(x(t)) represents the predicted position of the traffic participant at the current moment, v represents the driving speed of the traffic participant at the current moment, ω represents the rotational speed of the traffic participant at the current moment, θ represents the circular motion angle of the traffic participant at the current moment, x(t) represents the coordinate of the traffic participant on the X-axis at the current moment, y(t) represents the coordinate of the traffic participant on the Y-axis at the current moment, and Δt represents the time interval between the current moment and the next moment.

[0019] In one embodiment, the above-mentioned obtaining the predicted trajectories of other traffic participants within a preset duration starting from the current moment includes:

[0020] Determine the predicted trajectories of other traffic participants within a preset duration starting from the current moment according to the environmental information and the motion models corresponding to the other traffic participants.

[0021] In one embodiment, the above-mentioned preset range includes a traffic intersection indicated by a traffic signal, and the other traffic participants are vehicles on the target lanes of the traffic intersection; the target lanes are at least one lane in each direction of the traffic intersection; the above-mentioned obtaining the predicted trajectories of other traffic participants within a preset duration starting from the current moment includes:

[0022] Obtain the passing time period of the target lane and the driving direction corresponding to the target lane;

[0023] Determine the predicted trajectories of other traffic participants according to the driving direction corresponding to the target lane and the passing time period of the target lane.

[0024] In one embodiment, when there is an intersection between the predicted trajectory of the target traffic participant and the predicted trajectories of other traffic participants, the warning information is sent, including:

[0025] Determine the target traffic participants prohibited from driving according to the passing time period of the target lane and the environmental information;

[0026] When the predicted trajectory of the target traffic participant prohibited from driving intersects with the predicted trajectories of other traffic participants, send warning information to the target traffic participant prohibited from driving.

[0027] In one embodiment, obtaining the environmental information at the current moment within a preset range includes:

[0028] Obtain video data and point cloud data within a preset range according to a preset sampling frequency;

[0029] Obtain the environmental information at the current moment according to the video data and the point cloud data.

[0030] In one embodiment, obtaining the environmental information at the current moment according to the video data and the point cloud data includes:

[0031] Perform first target detection processing on the video data to obtain a video detection result; the video detection result includes the first position information and target type of each traffic participant;

[0032] Perform second target detection processing on the point cloud data to obtain a point cloud detection result; the point cloud detection result includes the rotational speed, driving speed, and second position information of each traffic participant;

[0033] Obtain the environmental information at the current moment according to the video detection result and the point cloud detection result.

[0034] In one embodiment, obtaining the environmental information at the current moment according to the video detection result and the point cloud detection result includes:

[0035] Based on the first position information and the second position information, register the video detection result and the point cloud detection result, and perform fusion processing on the video detection result and the point cloud detection result by using a fusion algorithm to obtain the environmental information at the current moment.

[0036] In a second aspect, an embodiment of the present application provides a warning device, and the device includes:

[0037] An acquisition module, configured to acquire environmental information at the current moment within a preset range; the environmental information includes the rotational speed, position information, and driving speed of traffic participants within the preset range;

[0038] A first determination module, configured to determine a predicted trajectory of a target traffic participant within a preset duration starting from the current moment according to the environmental information and a motion model corresponding to the target traffic participant; the target traffic participant is at least one of the traffic participants in the environmental information;

[0039] A second determination module, which acquires the predicted trajectories of other traffic participants within a preset duration starting from the current moment; the other traffic participants are the traffic participants in the environmental information except the target traffic participant;

[0040] An early warning module, configured to send out an early warning message when there is an intersection between the predicted trajectory of the target traffic participant and the predicted trajectories of other traffic participants.

[0041] In a third aspect, an embodiment of the present application provides an electronic device, including a memory and a processor, where the memory stores a computer program, and when the processor executes the computer program, the steps of the method described in the first aspect are implemented.

[0042] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method described in the first aspect are implemented.

[0043] The above-mentioned early warning method, device, equipment, and storage medium acquire environmental information at the current moment within a preset range, and determine the predicted trajectory of a target traffic participant within a preset duration starting from the current moment according to the environmental information and a motion model corresponding to the target traffic participant. At the same time, the predicted trajectories of other traffic participants within a preset duration starting from the current moment are acquired. Furthermore, when there is an intersection between the predicted trajectory of the target traffic participant and the predicted trajectories of other traffic participants, an early warning message is sent out. Among them, the environmental information includes the rotational speed, position information, and driving speed of traffic participants within the preset range, the target traffic participant is at least one of the traffic participants in the environmental information, and the other traffic participants are the traffic participants in the environmental information except the target traffic participant, so that the early warning message sent to the target traffic participant is sent when there is an intersection between the predicted trajectory of the target traffic participant within a preset duration starting from the current moment and the predicted trajectories of other traffic participants, rather than when the distance between the target traffic participant and other traffic participants is less than a preset threshold, avoiding the occurrence of traffic accidents due to the failure to timely remind traffic participants. Description of the Drawings

[0044] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0045] Figure 1 It is a schematic diagram of the application environment of the early warning method in an embodiment of the present application;

[0046] Figure 2 It is a schematic flowchart of the early warning method in an embodiment of the present application;

[0047] Figure 3 It is a schematic flowchart of the early warning method in another embodiment of the present application;

[0048] Figure 4 It is a schematic diagram of a traffic intersection in an embodiment of the present application;

[0049] Figure 5 It is a schematic flowchart of the early warning method in another embodiment of the present application;

[0050] Figure 6 It is a schematic flowchart of the early warning method in another embodiment of the present application;

[0051] Figure 7 It is a schematic flowchart of the early warning method in another embodiment of the present application;

[0052] Figure 8 It is a schematic structural diagram of the early warning device provided in an embodiment of the present application;

[0053] Figure 9 It is the internal structure diagram of the device in an embodiment of the present application. Detailed implementation manners

[0054] In order to make the purpose, technical solutions and advantages of the present application clearer, the following further details the present application in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0055] It can be understood that the terms "first", "second", "third", "fourth", etc. (if any) in the embodiments of the present application are used to distinguish similar objects and do not necessarily need to describe a specific order or sequence.

[0056] The early warning method provided in this embodiment can be applicable to, for example Figure 1In the application environment shown. Among them, the intelligent base station 100 (also known as the roadside integrated perception system or roadside base station) set on the roadside can collect data within a preset range, and issue a warning to the target traffic participants within the preset range according to the collected data. The intelligent base station 100 is integrated with a collection device 110 and a processor 120. The environmental information within the preset range is collected through the collection device 110, and the environmental information is processed by the processor 120, and a warning is issued to the target traffic participants.

[0057] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application.

[0058] It should be noted that the execution subject of the following method embodiments can also be a warning device, and this device can be implemented as part or all of the above-mentioned electronic device in a software, hardware, or software-hardware combination manner. The following method embodiments are described by taking the execution subject as an electronic device as an example.

[0059] Figure 2 It is a flowchart of a warning method provided by an embodiment of the present application. This embodiment relates to the specific process of how to issue a warning to target traffic participants. As Figure 2 shown, the method includes the following steps:

[0060] S101. Obtain the environmental information at the current moment within a preset range; the environmental information includes the rotational speed, position information, and driving speed of traffic participants within the preset range.

[0061] The environmental information can be obtained from the data collected by intelligent base stations installed on the roadside. Correspondingly, the preset range can be the collection range of the collection devices on the intelligent base stations. Of course, the preset range can also be a range within the collection range of the collection devices on the intelligent base stations. For example, the collection range of the collection devices on the intelligent base stations is a circular area with the intelligent base station as the center and a preset length as the radius. The preset range in the time domain can be half of this circular area, such as a semi-circular area on the south side of this circular area. Traffic participants can refer to motor vehicles, non-motor vehicles, and pedestrians on traffic roads. Traffic participants can communicate with the intelligent base stations through vehicle to everything (V2X), 5th generation mobile networks (5G), or the 4th generation communication system (4G) technology to achieve data transmission. In a possible situation, traffic participants may not communicate with the intelligent base stations either.

[0062] The intelligent base station can collect video data within the preset range through a camera and obtain environmental information including the rotation speed, position information, and driving speed of traffic participants within the preset range based on the collected video data. Among them, the rotation speed of the traffic participant can be used to determine whether the traffic participant is turning. For example, when the rotation speed of the traffic participant is not 0, it indicates that the traffic participant is turning. The position information can be represented by the X-axis coordinate and Y-axis coordinate of the traffic participant in the preset coordinate system. The intelligent base station can also collect point cloud data within the preset range through a millimeter-wave radar and obtain the above environmental information based on the collected point cloud data. The intelligent base station can also collect point cloud data within the preset range through a lidar and obtain the above environmental information based on the collected point cloud data. The embodiments of the present application do not limit this. In a possible situation, the intelligent base station can also collect video data within the preset range through a camera and, at the same time, collect point cloud data within the preset range through a millimeter-wave radar, and then obtain the above environmental information based on the video data and the point cloud data.

[0063] S102. Determine the predicted trajectory of the target traffic participant within a preset duration starting from the current moment according to the environmental information and the motion model corresponding to the target traffic participant; the target traffic participant is at least one traffic participant in the environmental information.

[0064] The target traffic participant is at least one traffic participant in the environmental information, and each target traffic participant has its corresponding motion model. The motion model corresponding to the target traffic participant can be determined according to the preset corresponding relationship between the traffic participant and the motion model; it can also be determined according to the type of the target traffic participant; it can also be determined according to the rotational speed of the target traffic participant. The embodiments of the present application do not limit this. The motion model can be a mathematical model or a neural network model. The embodiments of the present application do not limit this. The rotational speed, position information, and driving speed of the target traffic participant indicated in the environmental information can be input into the motion model corresponding to the target traffic participant to obtain the predicted trajectory of the target traffic participant within a preset duration starting from the current moment. Among them, the preset duration can be the duration marked in seconds or the duration marked by the number of acquisition frames of the acquisition device. The embodiments of the present application do not limit this. For example, the preset duration can be 5 seconds, and the preset duration can also be 100 frames.

[0065] S103. Obtain the predicted trajectory of other traffic participants within a preset duration starting from the current moment; the other traffic participants are traffic participants in the environmental information other than the target traffic participant.

[0066] The other traffic participants are traffic participants in the environmental information other than the target traffic participant. When determining the predicted trajectory of the target traffic participant within a preset duration starting from the current moment according to the environmental information and the motion model corresponding to the target traffic participant, a similar method can be adopted, that is, according to the environmental information and the motion model corresponding to the other traffic participants, determine the predicted trajectory of the other traffic participants within a preset duration starting from the current moment. In a possible situation, the other traffic participants can have their corresponding preset driving routes, or the predicted trajectory of the other traffic participants within a preset duration starting from the current moment can be determined according to the preset driving routes of the other traffic participants. The embodiments of the present application do not limit this.

[0067] S104. When there is an intersection between the predicted trajectory of the target traffic participant and the predicted trajectory of the other traffic participants, send a warning message.

[0068] When there is an intersection between the predicted trajectory of the target traffic participant and the predicted trajectory of the other traffic participants, that is to say, there is a risk of collision between the target traffic participant and the other traffic participants within the preset duration. The intelligent base station can send a warning message to the target traffic participant or can also send a warning message to the other traffic participants. The embodiments of the present application do not limit this. The intelligent base station can broadcast a warning to the area where a collision may occur through voice, or can also send a warning message to the traffic participants communicatively connected to the intelligent base station in the way of V2X, 5G or 4G. The embodiments of the present application do not limit this.

[0069] The above warning method obtains the environmental information at the current moment within a preset range, and determines the predicted trajectory of the target traffic participant within a preset duration starting from the current moment according to the environmental information and the motion model corresponding to the target traffic participant. At the same time, it obtains the predicted trajectories of other traffic participants within a preset duration starting from the current moment. Then, when there is an intersection between the predicted trajectory of the target traffic participant and the predicted trajectories of other traffic participants, a warning message is sent. Among them, the environmental information includes the rotational speed, position information, and driving speed of traffic participants within the preset range. The target traffic participant is at least one of the traffic participants in the environmental information, and other traffic participants are the traffic participants in the environmental information except the target traffic participant. This makes the warning message sent to the target traffic participant when there is an intersection between the predicted trajectory of the target traffic participant within a preset duration starting from the current moment and the predicted trajectories of other traffic participants, rather than when the distance between the target traffic participant and other traffic participants is less than a preset threshold, thus avoiding the occurrence of traffic accidents caused by the inability to timely remind traffic participants.

[0070] When determining the predicted trajectory of the target traffic participant within a preset duration starting from the current moment according to the environmental information and the motion model corresponding to the target traffic participant, the corresponding motion model for the target traffic participant can be determined first according to the rotational speed of the target traffic participant. Optionally, the motion model corresponding to the target traffic participant is determined according to whether the rotational speed of the target traffic participant is 0.

[0071] Whether the rotational speed of the target traffic participant is 0 can indicate whether the target traffic participant is turning. There are significant differences in the future motion trajectories between the target traffic participant turning and going straight. If only a single motion model is used, it may lead to inaccurate predicted trajectories of the target traffic participant predicted according to the motion model and environmental information. Therefore, the corresponding motion model for the target traffic participant can be determined based on whether the rotational speed of the target traffic participant is 0.

[0072] In a possible case, when the rotational speed of the target traffic participant is 0, the motion model corresponding to the target traffic participant is the first formula; the first formula includes:

[0073]

[0074] Among them, It represents the position of the traffic participant at the next moment. g(x(t)) represents the predicted position of the traffic participant at the current moment. v represents the driving speed of the traffic participant at the current moment. θ represents the angular motion angle of the traffic participant at the current moment. x(t) represents the coordinate of the traffic participant on the X-axis at the current moment. y(t) represents the coordinate of the traffic participant on the Y-axis at the current moment. ω represents the rotational speed of the traffic participant at the current moment. Among them, the position information of the traffic participant includes x(t) and y(t), and Δt represents the time interval between the current moment and the next moment.

[0075] It should be noted that when the intelligent base station collects data, it usually marks the positions of each traffic participant according to a preset coordinate system. Among them, the preset coordinate system can be the coordinate system set by the intelligent base station. For example, the preset coordinate system takes the position where the intelligent base station is located as the coordinate origin, the due south direction as the X-axis, and the due east direction as the Y-axis.

[0076] In a possible case, when the rotational speed of the target traffic participant is not 0, the motion model corresponding to the target traffic participant is the second formula; the second formula includes:

[0077]

[0078] Among them, It represents the position of the traffic participant at the next moment. g(x(t)) represents the predicted position of the traffic participant at the current moment. v represents the driving speed of the traffic participant at the current moment. ω represents the rotational speed of the traffic participant at the current moment. θ represents the angular motion angle of the traffic participant at the current moment. x(t) represents the coordinate of the traffic participant on the X-axis at the current moment. y(t) represents the coordinate of the traffic participant on the Y-axis at the current moment. Δt represents the time interval between the current moment and the next moment.

[0079] The above warning method determines the motion model corresponding to the target traffic participant according to whether the rotational speed of the target traffic participant is 0, so that when obtaining the predicted trajectory of the target traffic participant, a motion model with a high matching degree with the target traffic participant can be used, and then the predicted trajectory of the target traffic participant obtained by using the motion model with a high matching degree and environmental information is more accurate.

[0080] The above embodiments focus on describing the specific process of how to obtain the predicted trajectory of the target traffic participant. The following will describe in detail the specific process of how to obtain the predicted trajectories of other traffic participants through the following embodiments.

[0081] Optionally, a possible implementation method of the above S103 "obtaining the predicted trajectory of other traffic participants within a preset duration starting from the current moment" includes: determining the predicted trajectory of other traffic participants within a preset duration starting from the current moment according to the environmental information and the motion model corresponding to other traffic participants.

[0082] In a possible case, the above preset range includes a traffic intersection indicated by a traffic signal, and other traffic participants are vehicles on the target lane of the traffic intersection; the target lane is at least one lane among the lanes in each direction of the traffic intersection; a possible implementation method of the above S103 "obtaining the predicted trajectory of other traffic participants within a preset duration starting from the current moment" can be as Figure 3 shown.

[0083] S201. Obtain the passing period of the target lane and the driving direction corresponding to the target lane.

[0084] In a possible case, the preset range includes a traffic intersection indicated by a traffic signal. For example, the preset range can be a traffic intersection as Figure 4 shown, which includes lanes 1-1, 1-2, 1-3, 1-4, 2-1, 2-2, 2-3, and 2-4. Correspondingly, there are crosswalks for pedestrians on each lane. The target lane is at least one lane among the lanes in each direction of the traffic intersection. The intelligent base station can collect the image of the traffic signal and identify the passing period of the traffic signal indicating the target lane from the image; the intelligent base station can also be communicatively connected to the traffic signal to receive the passing period of the target lane sent by the traffic signal; the embodiments of the present application do not limit this. The intelligent base station can also perform image recognition on the collected images within the preset range to identify the driving direction of the target lane.

[0085] S202. Determine the predicted trajectory of other traffic participants according to the driving direction corresponding to the target lane and the passing period of the target lane.

[0086] Other traffic participants are vehicles on the target lane of the traffic intersection, that is to say, other traffic participants need to drive according to the passing duration indicated by the traffic signal. When the passing period of the target lane is determined, the trajectory of other traffic participants driving on the target lane can be predicted according to the driving direction corresponding to the target lane and the passing period of the target lane to obtain the predicted trajectory of other traffic participants. For example, if the driving direction corresponding to the target lane is straight from south to north, and at the same time, the passing period of the target lane is obtained as 9:10:30 to 9:11:00, then the predicted trajectory of other traffic participants on the target lane can be determined as straight from south to north from 9:10:30 to 9:11:00.

[0087] In the above warning method, by obtaining the passing time period of the target lane and the driving direction corresponding to the target lane, and determining the predicted trajectories of other traffic participants according to the driving direction corresponding to the target lane and the passing time period of the target lane, it avoids obtaining the predicted trajectories of other traffic participants through a complex motion model, and improves the convenience of obtaining the predicted trajectories of other traffic participants.

[0088] Based on the above embodiments, in a possible situation, a warning message can be sent to the target traffic participants prohibited from driving indicated by the traffic signal. The following will describe in detail through Figure 5 the embodiments shown. As Figure 5 shown, for the above S103 "When there is an intersection between the predicted trajectory of the target traffic participant and the predicted trajectories of other traffic participants, send a warning message", a possible implementation method includes:

[0089] S301. Determine the target traffic participants prohibited from driving indicated by the traffic signal according to the passing time period of the target lane and the environmental information.

[0090] S302. When the predicted trajectory of the target traffic participant prohibited from driving intersects with the predicted trajectories of other traffic participants, send a warning message to the target traffic participant prohibited from driving.

[0091] Based on the above embodiments, when the traffic signal indicates that the target lane is passable, correspondingly, there are other lanes prohibited from passing. For example, continuing as Figure 4 shown, when the traffic signal indicates that lanes 1-1, 1-2, 1-3, and 1-4 can go straight, correspondingly, the crosswalks on lane 1-1, lane 1-2, lane 1-3, and lane 1-4 are prohibited from passing, and lanes 2-1, 2-2, 2-3, and 2-4 are also prohibited from passing. That is to say, the pedestrians on the crosswalks on lane 1-1, lane 1-2, lane 1-3, and lane 1-4 are the target traffic participants prohibited from driving, and the vehicles on lanes 2-1, 2-2, 2-3, and 2-4 are the target traffic participants prohibited from passing. For another example, when the traffic signal indicates that lane 2-1 can turn left, correspondingly, the crosswalks on lane 2-1 and lane 1-2 are prohibited from passing. That is to say, the pedestrians on the crosswalks on lane 2-1 and lane 1-2 are the target traffic participants. When the predicted trajectory of the vehicle (other traffic participant) on lane 2-1 intersects with the predicted trajectories of the pedestrians (target traffic participants prohibited from driving) on the crosswalks on lane 2-1 and lane 1-2, a warning message can be sent to the pedestrians on the crosswalks on lane 2-1 and lane 1-2.

[0092] For the above warning method, according to the passing time period and environmental information of the target lane, the target traffic participants prohibited from driving by the traffic signal are determined. When the predicted trajectory of the target traffic participants prohibited from driving intersects with the predicted trajectories of other traffic participants, a warning message is sent to the target traffic participants prohibited from driving, so that the sent warning message can be targeted at the target traffic participants prohibited from driving, without the need to send it to all traffic participants, improving the pertinence of the warning message.

[0093] The above embodiments focus on describing the specific process of sending warning messages. The following embodiments will be used to describe in detail the specific process of obtaining the environmental information at the current moment within a preset range.

[0094] Figure 6 It is a schematic flowchart of the warning method provided by another embodiment of the present application. As Figure 6 shown, this embodiment is related to the specific process of obtaining the environmental information at the current moment within a preset range. As Figure 6 shown, a possible implementation method for the above S101 "Obtain the environmental information at the current moment within a preset range" includes:

[0095] S401. Obtain video data and point cloud data within a preset range according to a preset sampling frequency.

[0096] The video data can be collected by a camera set on a smart base station, and the point cloud data can be collected by a radar set on the smart base station. Among them, the preset sampling frequency can include the frequency of the camera collecting video data and the frequency of the radar collecting point cloud data. The frequency of the camera collecting video data and the frequency of the radar collecting point cloud data can be the same or different, and the embodiments of the present application do not limit this.

[0097] S402. Obtain the environmental information at the current moment according to the video data and the point cloud data.

[0098] After obtaining the above video data and point cloud data, the target type of traffic participants in the video data at the current moment can be fused with the rotational speed and driving speed of traffic participants in the point cloud data to obtain the above environmental information at the current moment.

[0099] In a possible situation, a possible implementation method for the above S402 "Obtain the environmental information at the current moment according to the video data and the point cloud data" can be as Figure 7 shown, including:

[0100] S501. Perform first target detection processing on the video data to obtain a video detection result; the video detection result includes the first position information and target type of each traffic participant.

[0101] Generally, due to the mutual occlusion among multiple traffic participants, video data may not include all traffic participants within a preset range. Among them, the first object detection process can be a processing method for extracting video detection results from the above-mentioned video data. For example, the first object detection process can include deep learning-based object detection algorithms such as SSD, Faster RCNN, and YOLO. Through the first object detection process, the first position information and object type of traffic participants in the current frame of video data can be obtained. For example, through the first object detection process, the coordinate information of the object box containing traffic participants is output, and at the same time, the object type of the traffic participants in the object box is identified, such as category information of vehicles, pedestrians, etc.

[0102] S502. Perform a second object detection process on the point cloud data to obtain a point cloud detection result; the point cloud detection result includes the rotation speed, driving speed, and second position information of each traffic participant.

[0103] The point cloud data is information obtained based on the reflection of radar signals when encountering obstacles. Generally, the point cloud data of the current frame is accumulated based on the information obtained by reflecting multiple emitted radar signals, and it can indicate the motion state of an object, such as driving speed and rotation speed. Among them, the second object detection process can be a processing method for extracting a point cloud detection result from the above-mentioned point cloud data. For example, the second object detection process can include point cloud object detection algorithms based on deep learning such as second and PointRCNN. Through the second object detection process, the coordinate box information containing traffic participants is obtained from the point cloud data of the current frame, and at the same time, the second position information, driving speed, and rotation speed of the traffic participants in the object box are identified.

[0104] S503. Obtain the environmental information at the current moment according to the video detection result and the point cloud detection result.

[0105] In a possible situation, a possible implementation method of the above S503 "obtain the environmental information at the current moment according to the video detection result and the point cloud detection result" includes: registering the video detection result and the point cloud detection result based on the first position information and the second position information, and performing a fusion process on the video detection result and the point cloud detection result by using a fusion algorithm to obtain the environmental information at the current moment.

[0106] After performing first target detection processing on video data to obtain a video detection result and performing second target detection processing on point cloud data to obtain a point cloud detection result, based on the first position information indicating traffic participants in the video detection result and the second position information indicating the same traffic participants in the point cloud detection result, rotation and translation matrix registration can be performed on the video detection result and the point cloud detection result to unify the video detection result and the point cloud detection result into the same coordinate system. Subsequently, the target types of each traffic participant in the video detection result and the rotational speed and driving speed of each traffic participant in the point cloud detection result are further fused to obtain the environmental information at the current moment.

[0107] It should be understood that although the steps in the flowcharts in the above embodiments are sequentially shown according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least some of the steps in the flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same moment, but can be executed at different moments. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or sub-steps or stages of other steps.

[0108] Figure 8 It is a schematic structural diagram of an early warning device in an embodiment of the present application, as Figure 8 shown, including: an acquisition module 10, a first determination module 20, a second determination module 30, and an early warning module 40, where:

[0109] The acquisition module 10 is configured to acquire environmental information at the current moment within a preset range; the environmental information includes the rotational speed, position information, and driving speed of traffic participants within the preset range.

[0110] The first determination module 20 is configured to determine the predicted trajectory of the target traffic participant within a preset duration starting from the current moment according to the environmental information and the motion model corresponding to the target traffic participant; the target traffic participant is at least one traffic participant in the environmental information.

[0111] The second determination module 30 acquires the predicted trajectories of other traffic participants within a preset duration starting from the current moment; the other traffic participants are traffic participants in the environmental information other than the target traffic participant.

[0112] The early warning module 40 is configured to issue an early warning message when there is an intersection between the predicted trajectory of the target traffic participant and the predicted trajectories of other traffic participants.

[0113] In one embodiment, the warning device further includes: a third determination module 50, where:

[0114] The third determination module 50 is configured to determine the motion model corresponding to the target traffic participant according to whether the rotational speed of the target traffic participant is 0.

[0115] In one embodiment, the third determination module 50 is specifically configured to when the rotational speed of the target traffic participant is 0, the motion model corresponding to the target traffic participant is the first formula; the first formula includes:

[0116]

[0117] Where, represents the position of the traffic participant at the next moment, g(x(t)) represents the predicted position of the traffic participant at the current moment, v represents the speed of the traffic participant at the current moment, θ represents the circular motion angle of the traffic participant at the current moment, x(t) represents the coordinate of the traffic participant on the X-axis at the current moment, y(t) represents the coordinate of the traffic participant on the Y-axis at the current moment, ω represents the rotational speed of the traffic participant at the current moment, and Δt represents the time interval between the current moment and the next moment; when the rotational speed of the target traffic participant is not 0, the motion model corresponding to the target traffic participant is the second formula; the second formula includes:

[0118]

[0119] Where, represents the position of the traffic participant at the next moment, g(x(t)) represents the predicted position of the traffic participant at the current moment, v represents the speed of the traffic participant at the current moment, ω represents the rotational speed of the traffic participant at the current moment, θ represents the circular motion angle of the traffic participant at the current moment, x(t) represents the coordinate of the traffic participant on the X-axis at the current moment, y(t) represents the coordinate of the traffic participant on the Y-axis at the current moment, and Δt represents the time interval between the current moment and the next moment.

[0120] In one embodiment, the second determination module 30 is specifically configured to determine the predicted trajectory of other traffic participants within a preset duration starting from the current moment according to the environmental information and the motion models corresponding to other traffic participants.

[0121] In one embodiment, the above preset range includes a traffic intersection where traffic is indicated by traffic lights, and other traffic participants are vehicles on the target lane of the traffic intersection; the target lane is at least one lane among the lanes in each direction of the traffic intersection; the second determination module 30 is specifically configured to obtain the passing period of the target lane and the driving direction corresponding to the target lane; and determine the predicted trajectory of other traffic participants according to the driving direction corresponding to the target lane and the passing period of the target lane.

[0122] In one embodiment, the warning module 40 is specifically configured to determine a target traffic participant prohibited from driving according to the passing time period and environmental information of the target lane; when the predicted trajectory of the target traffic participant prohibited from driving intersects with the predicted trajectories of other traffic participants, a warning message is sent to the target traffic participant prohibited from driving.

[0123] In one embodiment, the acquisition module 10 is specifically configured to acquire video data and point cloud data within a preset range according to a preset sampling frequency; and acquire environmental information at the current moment according to the video data and the point cloud data.

[0124] In one embodiment, the acquisition module 10 is specifically configured to perform a first target detection process on the video data to obtain a video detection result; the video detection result includes the first position information and target type of each traffic participant; perform a second target detection process on the point cloud data to obtain a point cloud detection result; the point cloud detection result includes the rotation speed, driving speed and second position information of each traffic participant; and acquire environmental information at the current moment according to the video detection result and the point cloud detection result.

[0125] In one embodiment, the acquisition module 10 is specifically configured to register the video detection result and the point cloud detection result based on the first position information and the second position information, and perform a fusion process on the video detection result and the point cloud detection result by using a fusion algorithm to obtain environmental information at the current moment.

[0126] The warning device provided by the embodiment of the present application can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be elaborated here.

[0127] For the specific limitation of a warning device, reference can be made to the limitation of the warning method in the above text, which will not be elaborated here. Each module in the above warning device can be implemented in whole or in part by software, hardware and their combination. The above modules can be embedded in or independent of a processor in an electronic device in the form of hardware, or stored in a memory in the electronic device in the form of software, so as to facilitate the processor to call and execute the operations corresponding to the above respective modules.

[0128] In one embodiment, an electronic device is provided, and its internal structure diagram can be as Figure 9As shown. The electronic device includes a processor, a memory, a network interface, and an input device connected by a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the electronic device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it implements an early warning method.

[0129] Those skilled in the art can understand that Figure 9 the structure shown in is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the electronic device to which the solution of this application is applied. The specific electronic device may include more or fewer components than those shown in the figure, or combine some components, or have a different component arrangement.

[0130] It should be clear that the process of the processor executing the computer program in the embodiments of this application is consistent with the execution processes of the various steps in the above method. For specific details, please refer to the description above.

[0131] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, it can implement the early warning method provided in the above method embodiments of this application.

[0132] It should be clear that the process of the processor executing the computer program in the embodiments of this application is consistent with the execution processes of the various steps in the above method. For specific details, please refer to the description above.

[0133] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the various embodiments provided in the present application can include non-volatile and / or volatile memories. Non-volatile memories can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memories can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0134] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0135] The above-described embodiments merely represent several implementation manners of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation on the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the appended claims.

Claims

1. An early warning method, characterized in that, it includes: Obtain the environmental information at the current moment within a preset range; The environmental information includes the rotational speed, position information, and driving speed of traffic participants within the preset range; According to the environmental information and the motion model corresponding to the target traffic participant, determine the predicted trajectory of the target traffic participant within a preset duration starting from the current moment; the target traffic participant is at least one of the traffic participants in the environmental information; Obtain the predicted trajectories of other traffic participants within a preset duration starting from the current moment; the other traffic participants are the traffic participants in the environmental information except the target traffic participant; When there is an intersection between the predicted trajectory of the target traffic participant and the predicted trajectories of other traffic participants, send out an early warning message; Among them, before determining the motion trajectory of the target traffic participant within a preset duration starting from the current moment according to the environmental information and the motion model corresponding to the target traffic participant, the method further includes: When the rotational speed of the target traffic participant is 0, the motion model corresponding to the target traffic participant is the first formula; the first formula includes: Among them, represents the position of the traffic participant at the next moment, g(x(t)) represents the predicted position of the traffic participant at the current moment, v represents the driving speed of the traffic participant at the current moment, θ represents the circular motion angle of the traffic participant at the current moment, x(t) represents the coordinate of the traffic participant on the X-axis at the current moment, y(t) represents the coordinate of the traffic participant on the Y-axis at the current moment, ω represents the rotational speed of the traffic participant at the current moment, and Δt represents the time interval between the current moment and the next moment; When the rotational speed of the target traffic participant is not 0, the motion model corresponding to the target traffic participant is the second formula; the second formula includes: Among them, represents the position of the traffic participant at the next moment, g(x(t)) represents the predicted position of the traffic participant at the current moment, v represents the driving speed of the traffic participant at the current moment, ω represents the rotational speed of the traffic participant at the current moment, θ represents the circular motion angle of the traffic participant at the current moment, x(t) represents the coordinate of the traffic participant on the X-axis at the current moment, y(t) represents the coordinate of the traffic participant on the Y-axis at the current moment, and Δt represents the time interval between the current moment and the next moment.

2. The method according to claim 1, characterized in that, The obtaining of the predicted trajectories of other traffic participants within a preset duration starting from the current moment includes: According to the environmental information and the motion model corresponding to the other traffic participants, determine the predicted trajectories of the other traffic participants within a preset duration starting from the current moment.

3. The method according to claim 1, characterized in that, The preset range includes a traffic intersection where traffic is indicated by traffic lights, and the other traffic participants are vehicles on the target lanes of the traffic intersection; the target lanes are at least one lane in each direction of the traffic intersection; The obtaining of the predicted trajectories of other traffic participants within a preset duration starting from the current moment includes: Obtain the passing period of the target lane and the driving direction corresponding to the target lane; According to the driving direction corresponding to the target lane and the passing period of the target lane, determine the predicted trajectories of the other traffic participants.

4. The method according to claim 3, characterized in that, The step of sending out an early warning message when there is an intersection between the predicted trajectory of the target traffic participant and the predicted trajectories of other traffic participants includes: According to the passing period of the target lane and the environmental information, determine the target traffic participants prohibited from driving indicated by the traffic lights; When the predicted trajectory of the prohibited target traffic participant intersects with the predicted trajectories of the other traffic participants, send the early warning message to the prohibited target traffic participant.

5. The method according to claim 1, characterized in that, The obtaining of the environmental information at the current moment within a preset range includes: Obtain the video data and point cloud data within the preset range according to a preset sampling frequency; Obtain the environmental information at the current moment according to the video data and the point cloud data.

6. The method according to claim 5, wherein, the obtaining the environmental information at the current moment according to the video data and the point cloud data includes: Performing first target detection processing on the video data to obtain a video detection result; the video detection result includes the first position information and target types of each of the traffic participants; Performing second target detection processing on the point cloud data to obtain a point cloud detection result; the point cloud detection result includes the rotational speed, driving speed and second position information of each of the traffic participants; Obtain the environmental information at the current moment according to the video detection result and the point cloud detection result.

7. The method according to claim 6, wherein, the obtaining the environmental information at the current moment according to the video detection result and the point cloud detection result includes: Based on the first position information and the second position information, registering the video detection result and the point cloud detection result, and performing fusion processing on the video detection result and the point cloud detection result by using a fusion algorithm to obtain the environmental information at the current moment.

8. An early warning device, wherein, comprising: An acquisition module, configured to acquire the environmental information at the current moment within a preset range; The environmental information includes the rotational speed, position information and driving speed of traffic participants within the preset range; A first determination module, configured to determine the predicted trajectory of the target traffic participant within a preset duration starting from the current moment according to the environmental information and the motion model corresponding to the target traffic participant; the target traffic participant is at least one of the traffic participants in the environmental information; A second determination module, acquiring the predicted trajectories of other traffic participants within a preset duration starting from the current moment; the other traffic participants are traffic participants other than the target traffic participant in the environmental information; An early warning module, configured to issue an early warning message when there is an intersection between the predicted trajectory of the target traffic participant and the predicted trajectories of other traffic participants; A third determination module, when the rotational speed of the target traffic participant is 0, the motion model corresponding to the target traffic participant is the first formula; The first formula includes: Among them, represents the position of the traffic participant at the next moment, g(x(t)) represents the predicted position of the traffic participant at the current moment, v represents the driving speed of the traffic participant at the current moment, θ represents the circular motion angle of the traffic participant at the current moment, x(t) represents the coordinate of the traffic participant on the X-axis at the current moment, y(t) represents the coordinate of the traffic participant on the Y-axis at the current moment, ω represents the rotational speed of the traffic participant at the current moment, and Δt represents the time interval between the current moment and the next moment; When the rotational speed of the target traffic participant is not 0, the motion model corresponding to the target traffic participant is the second formula; the second formula includes: Among them, represents the position of the traffic participant at the next moment, g(x(t)) represents the predicted position of the traffic participant at the current moment, v represents the driving speed of the traffic participant at the current moment, ω represents the rotational speed of the traffic participant at the current moment, θ represents the circular motion angle of the traffic participant at the current moment, x(t) represents the coordinate of the traffic participant on the X-axis at the current moment, y(t) represents the coordinate of the traffic participant on the Y-axis at the current moment, and Δt represents the time interval between the current moment and the next moment.

9. An electronic device, wherein, comprising a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, the method according to any one of claims 1-7 is implemented.

10. A computer-readable storage medium, wherein, a computer program is stored thereon, and when the computer program is executed by a processor, the method according to any one of claims 1-7 is implemented.

Images