Early warning method, apparatus and system, computing device, and storage medium

Abstract

An early warning method, apparatus and system, a computing device, and a storage medium, relating to the technical field of image processing. The method comprises: acquiring position information of an unmanned aerial vehicle; on the basis of the position information, acquiring a scene video collected by the unmanned aerial vehicle in the flying process; and performing early warning identification on the scene video to obtain an identification result of the scene video, and performing early warning on the basis of the identification result. In this way, on the basis of the position of the unmanned aerial vehicle, a vehicle end uses different methods to acquire the scene video collected by the unmanned aerial vehicle, so that the scene video can be acquired regardless of whether the vehicle end and the unmanned aerial vehicle can communicate or not, avoiding the situation that early warning cannot be performed when communication cannot be performed; and the road condition in front of the vehicle is early warned by identifying the scene video, improving timeliness and comprehensiveness of early warning, thus improving driving safety.

Description

Early warning method, device, system, computing device and storage medium

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to Chinese patent application number 202410099105.0, filed on January 24, 2024, entitled “Early Warning Method, Apparatus, System, Computing Equipment and Storage Medium,” the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present invention relates to the field of image processing technology, and in particular to an early warning method, apparatus, system, computing device and storage medium. Background Art

[0004] In the current vehicle-mounted drone solution, the drone and the vehicle only have simple data communication, and the communication between the two is single and affected by geographical features and weather, as well as the coverage of communication signals. Communication may be impossible. In this case, the road conditions in front of the vehicle cannot be detected, and timely warnings cannot be issued, resulting in reduced vehicle driving safety. Summary of the Invention

[0005] To this end, the present invention proposes an early warning method, as well as an early warning system, a vehicle terminal, an unmanned aerial vehicle, a computing device and a computer-readable storage medium, aiming to at least to some extent solve the technical problem in related technologies that timely early warning cannot be provided during vehicle driving.

[0006] To achieve the above objectives, a first embodiment of the present invention provides an early warning method, which is applied to a vehicle side and includes:

[0007] Get the location information of the drone;

[0008] Based on the location information, obtain the scene video collected during the UAV flight;

[0009] Perform early warning recognition on the scene video, obtain the recognition result of the scene video, and issue an early warning based on the recognition result.

[0010] According to one embodiment of the present invention, obtaining a scene video captured during a drone flight based on location information includes:

[0011] If the drone is determined to be in the communication area based on the location information and the communication between the drone and the vehicle is normal, the scene video is obtained from the drone through the communication module on the vehicle.

[0012] According to one embodiment of the present invention, obtaining a scene video captured during a drone flight based on location information includes:

[0013] If it is determined based on the location information that the drone is not in the communication area, or the drone is in the communication area but the communication with the vehicle is abnormal, a scene video acquisition request is sent to the satellite, and the scene video is obtained from the satellite through the satellite module on the vehicle. The scene video is sent to the satellite by the drone when it cannot communicate with the vehicle.

[0014] According to one embodiment of the present invention, determining that communication between the drone and the vehicle is abnormal includes:

[0015] If no information is received from the drone within the preset time, it is determined that the communication between the drone and the vehicle is abnormal.

[0016] According to one embodiment of the present invention, after obtaining the location information of the drone, the method further includes:

[0017] If the drone is determined to be in a no-fly zone based on the location information, the drone is controlled to fly to a non-no-fly zone.

[0018] According to one embodiment of the present invention, after obtaining the location information of the drone, the method further includes:

[0019] If the drone is determined to be at the edge of a non-no-fly zone or a communication zone based on the location information, a warning message is generated and the flight path of the drone is adjusted.

[0020] According to one embodiment of the present invention, issuing an early warning based on the recognition result includes:

[0021] The vehicle-side processing module analyzes the recognition results to obtain the scene information and scene hazard level corresponding to the scene video;

[0022] If the scene is determined to be abnormal based on the scene danger level, corresponding early warning measures will be taken based on the scene information.

[0023] According to one embodiment of the present invention, taking corresponding warning measures according to the scenario information includes:

[0024] Adjusting the vehicle's route based on the scene information; and / or,

[0025] Generate warning information based on the scenario information and send it to the warning module and / or vehicle simulator on the vehicle side, so that the warning module and / or vehicle simulator performs a warning operation.

[0026] According to one embodiment of the present invention, after the vehicle-side processing module analyzes the recognition result and obtains the scene information and scene danger level of the scene corresponding to the scene video, the method further includes:

[0027] If it is determined that there is no abnormality in the scene based on the scene danger level, the scene video will be sent to the vehicle simulator on the vehicle side for display.

[0028] To achieve the above-mentioned object, a second embodiment of the present invention provides an early warning method applied to a drone, the method comprising:

[0029] During the flight, the vehicle collects video of the scene in front of it and sends the location information to the vehicle.

[0030] Based on the location information, the scene video is sent to the vehicle side so that the vehicle side can perform early warning recognition on the scene video, obtain the recognition result of the scene video, and issue an early warning based on the recognition result.

[0031] According to one embodiment of the present invention, sending a scene video to a vehicle based on location information includes:

[0032] If feedback from the vehicle side in response to the location information is received within a preset time period, the scene video is sent to the vehicle side;

[0033] If no feedback in response to the position information is received from the vehicle end within a preset time period, the scene video is sent to the satellite so that the satellite sends the scene video to the vehicle end.

[0034] To achieve the above-mentioned purpose, a third embodiment of the present invention provides an early warning device, which is applied to a vehicle, and includes:

[0035] The first acquisition module is used to obtain the location information of the drone;

[0036] The second acquisition module is used to obtain the scene video collected during the flight of the UAV based on the location information;

[0037] The early warning module is used to perform early warning recognition on the scene video, obtain the recognition result of the scene video, and issue an early warning based on the recognition result.

[0038] To achieve the above-mentioned objectives, a fourth embodiment of the present invention provides an early warning device for use with a drone, the device comprising:

[0039] The acquisition module is used to collect video of the scene in front of the vehicle during flight and send the location information to the vehicle end;

[0040] The sending module is used to send the scene video to the vehicle side based on the location information, so that the vehicle side can perform early warning recognition on the scene video, obtain the recognition result of the scene video, and issue an early warning based on the recognition result.

[0041] To achieve the above-mentioned object, a fifth embodiment of the present invention provides an early warning system, the system including a drone and a vehicle end;

[0042] Drones are used to collect video of the scene in front of the vehicle and send location information to the vehicle;

[0043] The vehicle side is used to obtain the scene video collected during the flight of the drone based on the location information, perform early warning recognition on the scene video, obtain the recognition result of the scene video, and issue an early warning based on the recognition result.

[0044] According to one embodiment of the present invention, the system further comprises a satellite;

[0045] Drones are also used to send scene videos to satellites when communication with the vehicle is unavailable;

[0046] The vehicle side is also used to send a scene video acquisition request to the satellite when it is determined based on the location information that the drone is not in the communication area or that the drone is in the communication area but the communication between the drone and the vehicle side is abnormal;

[0047] The satellite is used to send the scene video to the vehicle side in response to the scene video acquisition request.

[0048] To achieve the above-mentioned objectives, the sixth embodiment of the present invention provides a computing device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the early warning method as described in any one of the first or second aspects above.

[0049] To achieve the above-mentioned purpose, the seventh embodiment of the present invention provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, it implements the early warning method as described in any one of the first or second aspects above.

[0050] In the early warning method provided by an embodiment of the present invention, the vehicle obtains the location information of the drone and, based on the location information, obtains the scene video captured by the drone during flight; performs early warning identification on the scene video, obtains the scene video identification result, and issues an early warning based on the identification result. In this way, the vehicle obtains the scene video captured by the drone using different methods based on the drone's location. Regardless of whether the vehicle and the drone can communicate, the scene video can be obtained, avoiding the situation where the vehicle cannot issue an early warning when communication is impossible. By identifying the scene video, the vehicle issues an early warning of the road conditions ahead of the vehicle, improving the timeliness and comprehensiveness of the early warning, thereby improving driving safety.

[0051] Additional aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description which follows, or may be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] FIG1 is a flow chart of an early warning method provided according to an embodiment of the present invention;

[0053] FIG2 is a flow chart of another early warning method provided according to an embodiment of the present invention;

[0054] FIG3 is a schematic diagram of an early warning system provided according to an embodiment of the present invention;

[0055] FIG4 is a schematic structural diagram of an early warning device provided according to an embodiment of the present invention;

[0056] FIG5 is a schematic structural diagram of another early warning device provided according to an embodiment of the present invention;

[0057] FIG6 is a schematic structural diagram of a computing device provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0058] The following describes embodiments of the present invention in detail, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and are not to be construed as limiting the present invention.

[0059] With the development of drone technology, the application scenarios of vehicle-mounted drones are increasing. At the same time, there are several prominent problems in these scenarios. The first is the communication problem, the second is the early warning scenario, and the third is the drone chip computing power problem.

[0060] Communication issues: Current vehicle-mounted drones generally use 4G (4th generation, fourth generation mobile communication technology) or 4G+WIFI (wireless network communication technology) solutions. Data transmission using 4G network modules is affected by the environment and may be delayed, resulting in data reception timeouts and affecting user experience. Without 4G network modules, data transmission is mainly through short-distance communication, and the communication distance is affected by distance and location. If the communication distance is exceeded, the real-time data of the drone cannot be received. In areas without 4G signal coverage or beyond the communication range, the drone will be lost and unable to locate its position after getting lost.

[0061] Warning scenarios: Currently, drone warnings are not integrated with the vehicle side. Instead, they are fed back to the vehicle side through image transmission technology alone. The vehicle side uses rule-based judgment to determine whether a warning is needed. However, the current warning rules are limited, and some unknown scenarios cannot be judged, which may lead to warning errors.

[0062] Computing power issue: Currently, drones and vehicles use a separate design, that is, the drones and vehicles are designed separately. The computing power of the drones and the vehicle cannot be shared, which seriously wastes the computing power of the vehicle.

[0063] To this end, an embodiment of the present invention provides an early warning method that can solve the above technical problems. For its specific implementation, please refer to the description of each embodiment below.

[0064] The following describes the early warning method, apparatus, system, computing device, and storage medium proposed in the embodiments of the present invention with reference to the accompanying drawings.

[0065] FIG1 is a flow chart of an early warning method provided according to an embodiment of the present invention. The method is applied to a vehicle side and may include the following steps.

[0066] Step 101: Get the location information of the drone.

[0067] In an embodiment of the present invention, in order to know the road conditions ahead in advance and issue a warning during driving, or to know the road conditions ahead so as to take corresponding measures when problems occur during driving, the vehicle can send a flight mission to the drone and instruct the drone to inform its own location information in real time during the flight, and to collect scene video in real time and return it, so as to issue a timely warning in the event of an abnormal scene.

[0068] In a specific implementation, the vehicle can include a processing module and a communication module, while the drone can include a collection module and a communication module. The vehicle-side communication module and the drone's communication module can communicate data via Wi-Fi or other communication protocols. The vehicle-side processing module can plan routes based on road conditions and vehicle information, set warnings based on warning zones, and set geo-fences based on no-fly zones. The planned flight route, warning settings, and geo-fence settings are then included in the flight mission and transmitted to the drone's communication module via the vehicle-side communication module. The drone then conducts flight according to the planned flight route, warning settings, and geo-fence settings, and collects scene video in real time via the collection module. Furthermore, to obtain real-time information about the drone's operational status, the vehicle can communicate with the drone in real time via the communication module. The vehicle-side simulator can display the drone's location, route information, flight status, and scene video acquired through the communication module.

[0069] As an example, an early warning setting can ensure that drones do not operate within a warning zone (a dangerous area), and an electronic fence setting can ensure that drones operate within a non-no-fly zone. A non-no-fly zone is an area where drones can fly, while a no-fly zone is an area where drones are prohibited from flying, as opposed to a non-no-fly zone. These warning zones, no-fly zones, and non-no-fly zones can be pre-set.

[0070] In some embodiments, after obtaining the location information of the drone, the method further includes: if it is determined based on the location information that the drone is in a no-fly zone, controlling the drone to fly to a non-no-fly zone.

[0071] In other words, the vehicle can obtain the location information of the drone in real time, and when the drone is in a no-fly zone, adjust the flight route of the drone according to the range of the non-no-fly zone, so that the flight route avoids the non-no-fly zone, and then send the adjustment information to the drone. The drone will then fly according to the adjusted flight route and be able to fly out of the no-fly zone and into the non-no-fly zone.

[0072] In this way, when the drone flies into a no-fly zone, the vehicle can control the drone's flight route to ensure that the drone can always fly in a non-no-fly zone, thereby improving flight safety.

[0073] In other embodiments, after obtaining the location information of the drone, the method further includes: if it is determined based on the location information that the drone is at the edge of a non-no-fly zone or an edge of a communication zone, generating a warning message and adjusting the flight path of the drone.

[0074] The communication area refers to the area where the drone and the vehicle can communicate. It can be an area with a preset distance as the radius centered on the vehicle, and the range of the communication area may change as the vehicle moves.

[0075] That is to say, the vehicle can obtain the location information of the drone in real time, and when the drone is on the edge of a non-no-fly zone and is about to fly into a no-fly zone, or when the drone is on the edge of a communication zone and is about to lose communication with the vehicle, it will generate an early warning message to inform the user of the current situation and adjust the flight route of the drone according to the actual situation to ensure that the drone does not fly into a no-fly zone, or ensure that the drone does not lose communication with the vehicle.

[0076] In this way, if the drone is about to fly into a no-fly zone, the vehicle can adjust the drone's flight path in time to ensure that the drone always flies within the non-no-fly zone, thereby improving flight safety. Moreover, if the drone is about to leave the communication area, the drone's flight path can be adjusted in advance to control the drone to always fly within the communication area, ensuring that the drone and the vehicle can communicate normally.

[0077] In an embodiment of the present invention, the vehicle sends a flight mission to the drone. The drone collects scene video during the flight and reports its location information to the vehicle in real time. If the drone is in a no-fly zone or is about to fly into a no-fly zone, the drone's flight path can be adjusted to ensure that the drone always flies in a non-no-fly zone, thereby improving flight safety. If the drone is at the edge of the communication area, that is, the drone is about to lose communication with the vehicle, the drone's flight path can be adjusted so that the drone can always fly in the communication area, thereby ensuring that the drone and the vehicle can communicate normally.

[0078] Step 102: Based on the location information, obtain the scene video collected during the flight of the UAV.

[0079] In an embodiment of the present invention, the drone and vehicle can communicate in real time to ensure that the vehicle can obtain real-time information about the drone's operating status and avoid loss of the drone. However, since communication between the drone and the vehicle is affected by various conditions, such as communication distance, environmental factors, and failure of the drone or vehicle, if the communication distance between the drone and the vehicle is out of range, communication between the drone and the vehicle is interrupted due to environmental factors, or if at least one of the two fails, the drone and the vehicle can be considered unable to communicate, and the vehicle cannot directly obtain scene video from the drone. Therefore, the vehicle needs to use different methods to obtain scene video captured by the drone during flight, based on the drone's location information.

[0080] According to one embodiment of the present invention, based on location information, the specific implementation of obtaining scene video collected during the flight of a drone may include: if it is determined based on the location information that the drone is in a communication area and the communication between the drone and the vehicle is normal, then the scene video is obtained from the drone through the communication module on the vehicle.

[0081] In other words, if the drone is within the communication area, theoretically it can communicate normally with the vehicle. However, since communication between the vehicle and the drone is affected by various factors, communication may not be possible even within the communication area. Therefore, it is necessary to confirm whether the communication between the drone and the vehicle is normal. If so, the drone's communication module will send the scene video to the vehicle's communication module, and the vehicle can obtain the scene video. In this case, direct communication between the vehicle and the drone can improve communication efficiency.

[0082] According to another embodiment of the present invention, based on location information, the specific implementation of obtaining scene video collected during the flight of a drone may include: if it is determined based on the location information that the drone is not in the communication area, or the drone is in the communication area but the communication with the vehicle side is abnormal, then a scene video acquisition request is sent to the satellite, and the scene video is obtained from the satellite through the satellite module on the vehicle side, wherein the scene video is sent to the satellite by the drone when it is unable to communicate with the vehicle side.

[0083] That is to say, if the drone is not in the communication area, or the drone is in the communication area but cannot communicate normally with the vehicle, the vehicle cannot directly obtain the scene video from the drone. In this case, the drone can send the scene video to the satellite, and the vehicle can send a scene video acquisition request to the satellite. The satellite can send the scene video to the vehicle in response to the scene video acquisition request, and the vehicle can then obtain the scene video.

[0084] As an example, determining that communication between the drone and the vehicle is abnormal may include determining that communication between the drone and the vehicle is abnormal if no information is received from the drone within a preset time period. The preset time period may be pre-set or adjusted based on actual circumstances and is not limited in this embodiment of the present invention.

[0085] In the specific implementation, since the drone and the vehicle can communicate in real time, if the communication between the drone and the vehicle is abnormal, the drone will not be able to receive any information from the vehicle for a long time, and the vehicle will not be able to receive any information from the drone for a long time. In this case, the drone can send the scene video to the satellite, and the vehicle can send a scene video acquisition request to the satellite. In this way, the scene video can be acquired through the satellite, avoiding the situation where the warning cannot be issued due to the lack of communication.

[0086] In an embodiment of the present invention, both the vehicle and the drone can include satellite modules. If the drone does not receive any information from the vehicle within a preset time, it will deem that the communication with the vehicle is abnormal and send the scene video to the satellite through its own satellite module. If the drone is not in the communication area or does not receive any information from the drone within a preset time, the vehicle will send a scene video acquisition request to the satellite. In response to the scene video acquisition request, the satellite will send the scene video to the ground station, and the ground station will send the scene video to the satellite module on the vehicle. The vehicle can then obtain the scene video. In this way, if the drone and the vehicle cannot communicate normally, the scene video can be acquired through the satellite, ensuring that the vehicle can also receive the scene video even if the drone and the vehicle cannot communicate normally, thereby avoiding the impact of communication interruption on early warning.

[0087] In some embodiments, in order to ensure the security and efficiency of data transmission, the drone can encrypt and / or compress the collected scene video, and then send the encrypted and / or compressed scene video to the vehicle via satellite or directly. This not only reduces the amount of data transmitted and improves data transmission efficiency, but also encrypts the video to improve the security of data transmission. In a specific implementation, the drone can also include a data processing module, and encrypt and / or compress the scene data through the data processing module. In this case, what the vehicle receives is the processed scene video. Before performing warning recognition, the processed scene video can be decrypted and / or decompressed to obtain the scene video, and then the scene video can be used for warning recognition. This can not only improve the security of the scene video, but also improve the accuracy of warning recognition.

[0088] In an embodiment of the present invention, if the drone and the vehicle can communicate normally, the scene video is sent to the communication module of the vehicle through the communication module of the drone; if the drone and the vehicle cannot communicate normally, the drone can send the scene video to the satellite through the satellite module, and then the satellite sends the scene video to the satellite module of the vehicle, ensuring that the vehicle can receive the scene video under any circumstances, avoiding the impact of communication interruption on the early warning, and thus ensuring driving safety.

[0089] Step 103: Perform early warning recognition on the scene video, obtain the recognition result of the scene video, and issue an early warning based on the recognition result.

[0090] In an embodiment of the present invention, the vehicle side may include a processing module that performs early warning recognition on the scene video to determine whether the scene corresponding to the scene video contains any abnormality and obtain a recognition result. In addition, the processing module may have a built-in AI (Artificial Intelligence) computing power chip.

[0091] According to one embodiment of the present invention, the specific implementation of performing early warning recognition on scene video and obtaining the recognition result of the scene video may include: using a vehicle-side processing module to perform early warning recognition on the scene video based on a pre-built scene recognition model to obtain the recognition result of the scene video.

[0092] In a specific implementation, a scene recognition model can be pre-built and stored in a processing module on the vehicle side. After obtaining the scene video, the scene video is input into the pre-built scene recognition model. The scene recognition model can identify scene information such as scene type, location, time, and the degree of impact of the scene on driving (i.e., the scene hazard level), and generate corresponding recognition results based on this information.

[0093] As an example, if the scene video is an accident video, the recognition result may include the scene type (accident type), accident time, accident location, accident participants, traffic congestion caused by the accident, and the impact of the accident on driving (i.e., scene hazard level). If the scene video is a traffic jam video, the recognition result may include the scene type (congestion), congestion distance, congestion location, congestion duration, congestion degree, and the impact of congestion on driving (i.e., scene hazard level). If the scene video is a natural disaster video, the recognition result may include the scene type (natural disaster type), occurrence time, location, and natural disaster hazard level (i.e., scene hazard level).

[0094] In an embodiment of the present invention, the processing module on the vehicle side uses artificial intelligence to perform early warning recognition on the scene video through a pre-built scene recognition model to obtain recognition results, thereby improving the recognition accuracy, thereby achieving accurate early warning and improving driving safety.

[0095] In some embodiments, a pre-built scene recognition model is a model constructed using various preset inference algorithms for identifying scene videos. It can identify scene information such as scene type, location, and time, as well as the impact of the scene on driving (i.e., the scene hazard level) in the scene video to obtain corresponding recognition results. For example, the scene recognition model can be any model capable of scene recognition, such as an image segmentation model, an image recognition model, an image classification model, or a neural network model.

[0096] Therefore, before performing early warning recognition on the scene video and obtaining the recognition result of the scene video, it also includes: obtaining a sample training set, wherein the sample training set includes sample videos of multiple different scenes and the real recognition results corresponding to each sample video; using the sample videos of multiple different scenes as the input of the initial recognition model, using the real recognition results as labels, iteratively training the initial recognition model, and obtaining a scene recognition model.

[0097] In a specific implementation, sample videos of multiple different scenes and the actual recognition results of each sample video can be obtained from the scene video library. In addition, the sample videos can be processed in a unified format so that the size and format of the sample videos of multiple different scenes are unified, which is convenient for batch processing. The processed sample videos of multiple different scenes are then input into the initial recognition model, and the sample recognition results of each sample video are determined by the initial recognition model. A loss value is determined based on the sample recognition results and the actual recognition results. If the loss value is greater than a preset threshold, the model parameters of the initial recognition model are adjusted until the loss value is less than the preset threshold or the number of iterations reaches a preset number. The initial recognition model can be considered to have been trained and a scene recognition model is obtained.

[0098] As an example, sample videos include but are not limited to vehicle accidents (speeding, driving in the wrong direction, illegal parking or temporary parking, etc.), traffic accidents between vehicles (chain rear-end collisions, forced overtaking, head-on collisions, side collisions, cornering collisions, etc.), traffic accidents between vehicles and pedestrians (jaywalking accidents, vehicles entering the sidewalk, etc.), traffic congestion (caused by rain, traffic light failure or road construction, etc.), natural disasters (sandstorms, mudslides, avalanches, etc.) and other scene videos.

[0099] In an embodiment of the present invention, sample videos of multiple different scenes are used as input to the initial recognition model, and the corresponding real recognition results are used as output of the initial recognition model. The initial recognition model is iteratively trained to obtain a scene recognition model. Then, based on the scene recognition model, warning recognition is performed on the scene video, thereby improving the accuracy of warning recognition based on the scene recognition model.

[0100] Furthermore, after obtaining the scene video recognition results through early warning recognition, different measures can be taken to issue early warnings based on the recognition results. In addition, it can be set that if the scene danger level is 0, it means that there is no abnormality in the scene and no warning is required. If the scene danger level is not 0, it means that there is an abnormality in the scene and a warning is required for the user.

[0101] In some embodiments, the specific implementation of early warning based on the recognition results may include: parsing the recognition results through the processing module on the vehicle side to obtain the scene information and scene hazard level of the scene corresponding to the scene video; if it is determined that the scene is abnormal based on the scene hazard level, then taking corresponding early warning measures based on the scene information.

[0102] In a specific implementation, after obtaining the scene video recognition results through the scene recognition model, the recognition results can be analyzed to obtain scene information and scene danger level. If the scene danger level is not 0, it is determined that the scene is abnormal and the corresponding warning measures are determined based on the scene information. This not only provides a warning function, but also allows the implementation of corresponding warning measures based on the actual situation, thereby improving driving safety.

[0103] In some embodiments, the specific implementation of taking corresponding warning measures based on scene information may include: adjusting the vehicle's route based on the scene information; and / or generating warning information based on the scene information and sending it to the warning module and / or vehicle simulator on the vehicle side, so that the warning module and / or vehicle simulator performs warning operations.

[0104] The vehicle simulator can be an application installed on the vehicle side or a DHU APP (Desktop Head Unit Application).

[0105] In specific implementations, the decision to wait, continue driving, or reroute the vehicle can be made based on the scenario information, including the scenario type, time, and specific circumstances, to avoid road conditions affecting driving. Alternatively, a warning message can be generated and sent to the vehicle's warning module and / or vehicle simulator. The warning module can control the vehicle's lights to flash or the vehicle's horn to warn other vehicles of the danger ahead, and / or the vehicle simulator can display the scenario information so that the user can determine how to operate the vehicle next.

[0106] For example, if a sandstorm occurs in front of a vehicle, the drone can collect sandstorm video. After analysis on the vehicle side, the vehicle's route can be replanned, and the vehicle can be moved in time before the sandstorm reaches the vehicle's current location, thereby improving driving safety.

[0107] As an example, after the vehicle-side processing module generates a warning message, it is sent to the vehicle-side warning module, or to the vehicle-side vehicle simulator, or to both the vehicle-side warning module and the vehicle simulator. The warning module can issue a warning prompt through the vehicle screen, audio system, lighting, etc., and the specific warning method can be set according to user needs. After receiving the warning message, the vehicle simulator can display the corresponding warning information or adopt different alarm methods based on the warning information.

[0108] In this embodiment of the present invention, the vehicle-side system can reroute the vehicle based on scenario information to avoid dangerous road conditions and ensure driving safety. The processing module sends the warning information to the warning module and / or vehicle simulator, so that different measures can be taken to warn the user and improve driving safety.

[0109] Furthermore, after the vehicle-side processing module parses the recognition results and obtains the scene information and scene danger level of the scene corresponding to the scene video, it also includes: if it is determined that there is no abnormality in the scene based on the scene danger level, the scene video is sent to the vehicle-side vehicle simulator for display.

[0110] In the specific implementation, after obtaining the recognition result of the scene video through the scene recognition model, the recognition result can be analyzed to obtain the scene information and the scene danger level. When the scene danger level is 0, it is determined that there is no abnormality in the scene and no early warning is required. However, in order to allow users to understand the on-site situation in a timely manner, the scene video can be sent to the car simulator for display, so that users can download or subscribe to the scene at any time and take corresponding measures according to actual needs.

[0111] The early warning method provided by an embodiment of the present invention utilizes a vehicle-side communication module to communicate with the drone's communication module to obtain scene video for early warning identification when the drone and vehicle are communicating normally. If the drone and vehicle are unable to communicate, the scene video is transmitted to the vehicle via satellite, ensuring normal transmission of the scene video and avoiding the impact of communication interruptions on early warnings, thereby ensuring driving safety. Furthermore, early warning identification of scene video is performed through artificial intelligence analysis, improving the accuracy of early warning identification. This is achieved on the vehicle side without requiring the drone to perform early warning identification, reducing the drone's computational workload, lowering its cost, and improving its overall computing power. Furthermore, this solution uses a vehicle-mounted drone to provide early warnings of both natural disasters in the wild and dangerous road conditions in cities. Therefore, the vehicle can be used not only in cities but also in the wild, improving its applicability and thereby enhancing driving safety.

[0112] FIG2 is a flow chart of another early warning method provided according to an embodiment of the present invention. The method is applied to a UAV and may include the following steps.

[0113] Step 201: During the flight, the scene video in front of the vehicle is collected and the location information is sent to the vehicle end.

[0114] In an embodiment of the present invention, the drone includes a collection module, and uses the collection module to collect a video of the scene in front of the vehicle. The collection module can be a camera or any module capable of video collection.

[0115] Step 202: Based on the location information, the scene video is sent to the vehicle side so that the vehicle side can perform early warning recognition on the scene video, obtain the recognition result of the scene video, and issue an early warning based on the recognition result.

[0116] According to one embodiment of the present invention, the specific implementation of sending scene video to the vehicle side based on location information may include: if feedback from the vehicle side in response to the location information is received within a preset time length, the scene video is sent to the vehicle side; if feedback from the vehicle side in response to the location information is not received within the preset time length, the scene video is sent to the satellite, so that the satellite sends the scene video to the vehicle side.

[0117] In the specific implementation, after the drone sends the location information to the vehicle, the vehicle can send a scene video acquisition request to the drone. If the drone receives feedback from the vehicle in response to the location information within the preset time, it can be determined that the communication between the drone and the vehicle is normal, and the drone can send the scene video to the vehicle. However, if no feedback from the vehicle in response to the location information is received within the preset time, it may be that the vehicle determines that the drone is not in the communication area based on the location information or the communication between the two is abnormal. Therefore, the drone can send the scene video to the satellite so that the satellite sends the scene video to the vehicle.

[0118] It should be noted that the implementation process of step 201-step 202 is the same as the relevant steps of the above embodiment. Therefore, the specific implementation of step 201-step 202 can refer to the relevant description of the above embodiment, and this embodiment will not be repeated here.

[0119] In the early warning method provided by an embodiment of the present invention, a drone collects scene video in front of a vehicle. When the drone communicates normally with the vehicle, the drone sends the scene video to the vehicle. When the drone cannot communicate normally with the vehicle, the drone sends the scene video to the vehicle via a satellite. The vehicle performs early warning identification on the scene video and generates early warning information. This ensures that even when the vehicle and the drone cannot communicate, the drone can still provide an early warning of the road conditions in front of the vehicle, thereby improving the timeliness of the early warning and thereby improving driving safety.

[0120] FIG3 is a schematic diagram of an early warning system according to an embodiment of the present invention, wherein the early warning system includes a drone 301 and a vehicle 302 .

[0121] The drone 301 is used to collect video of the scene in front of the vehicle and send location information to the vehicle 302;

[0122] The vehicle end 302 is used to obtain the scene video collected by the drone 301 during the flight based on the location information, perform early warning recognition on the scene video, obtain the recognition result of the scene video, and issue an early warning based on the recognition result.

[0123] According to one embodiment of the present invention, the early warning system further includes a satellite 303;

[0124] The drone 301 is also used to send scene video to the satellite 303 when it is unable to communicate with the vehicle 302;

[0125] The vehicle end 302 is further configured to send a scene video acquisition request to the satellite 303 when it is determined based on the location information that the drone 301 is not in the communication area or that the drone 301 is in the communication area but the communication between the drone 301 and the vehicle end 302 is abnormal;

[0126] Satellite 303 is used to send the scene video to vehicle end 302 in response to the scene video acquisition request.

[0127] In some embodiments, the drone 301, as a data collector, may include a satellite module, a communication module, a collection module, and a data processing module. The satellite module may include a built-in satellite terminal (Beidou / Tiantong / GPS), allowing the drone 301 to communicate with satellite 303 via the satellite module. The communication module transmits and communicates with the vehicle 302 in real time via a communication protocol. The data processing module processes the scene video captured by the collection module. The drone 301 may also include sensors, such as related hardware sensors, a flight control processing unit, and the like.

[0128] In some embodiments, the vehicle-side 302, serving as the core for data processing and analysis, may include a satellite module, a communication module, a processing module, an early warning module, and a vehicle simulator. The vehicle simulator may be an application on the vehicle's large screen, enabling operations such as viewing, playing, and deleting videos and files. Furthermore, the vehicle simulator may be used to plan routes for drones, set routes, set early warnings, and establish electronic fences. The vehicle-side 302 may also include a remote control for playing and viewing videos.

[0129] It should be noted that the operations performed by the drone, vehicle and satellite in the early warning system can be found in the relevant descriptions of the above embodiments, and will not be repeated in this embodiment.

[0130] In an embodiment of the present invention, a drone collects scene video through a collection module and compresses and encrypts the scene video through a data processing module. If the drone can communicate normally with the vehicle, the drone sends the scene video to the vehicle. If the drone cannot communicate normally with the vehicle, the drone sends the processed scene video to a satellite through a satellite module. The satellite then sends the processed scene video to the vehicle. The satellite module on the vehicle receives the scene video and sends it to the processing module. The processing module on the vehicle decompresses and decrypts the processed scene video to obtain the scene video. The scene video is then subjected to early warning recognition to obtain a recognition result. Based on the recognition result, the vehicle's route is adjusted to avoid the impact of road conditions on driving, or a warning message is generated and sent to the early warning module and the vehicle simulator. The early warning module takes different measures to issue a warning, and the vehicle simulator displays the warning message. If the drone can communicate normally with the vehicle, the drone sends the processed scene video to the vehicle's communication module through the communication module for subsequent processing.

[0131] In this way, regardless of whether the drone and the vehicle can communicate normally, the vehicle can obtain the scene video, avoiding the situation where there is no warning when communication is impossible. The vehicle can also perform warning identification on the scene video, which improves the timeliness and comprehensiveness of the warning, thereby improving driving safety.

[0132] FIG4 is a schematic diagram of the structure of an early warning device provided according to an embodiment of the present invention, which is applied to a vehicle. The early warning device may include:

[0133] The first acquisition module 401 is used to obtain the location information of the UAV;

[0134] The second acquisition module 402 is used to acquire scene videos collected during the flight of the UAV based on the location information;

[0135] The early warning module 403 is used to perform early warning recognition on the scene video, obtain the recognition result of the scene video, and issue an early warning based on the recognition result.

[0136] According to one embodiment of the present invention, the second acquisition module 402 is further configured to:

[0137] If the drone is determined to be in the communication area based on the location information and the communication between the drone and the vehicle is normal, the scene video is obtained from the drone through the communication module on the vehicle.

[0138] According to one embodiment of the present invention, the second acquisition module 402 is further configured to:

[0139] If it is determined based on the location information that the drone is not in the communication area, or the drone is in the communication area but the communication with the vehicle is abnormal, a scene video acquisition request is sent to the satellite, and the scene video is obtained from the satellite through the satellite module on the vehicle. The scene video is sent to the satellite by the drone when it cannot communicate with the vehicle.

[0140] According to one embodiment of the present invention, the second acquisition module 402 is further configured to:

[0141] If no information is received from the drone within the preset time, it is determined that the communication between the drone and the vehicle is abnormal.

[0142] According to one embodiment of the present invention, the device further includes a control module, configured to:

[0143] If the drone is determined to be in a no-fly zone based on the location information, the drone is controlled to fly to a non-no-fly zone.

[0144] According to one embodiment of the present invention, the control module is further configured to:

[0145] If the drone is determined to be at the edge of a non-no-fly zone or a communication zone based on the location information, a warning message is generated and the flight path of the drone is adjusted.

[0146] According to one embodiment of the present invention, the early warning module 403 is further configured to:

[0147] The vehicle-side processing module analyzes the recognition results to obtain the scene information and scene hazard level corresponding to the scene video;

[0148] If the scene is determined to be abnormal based on the scene danger level, corresponding early warning measures will be taken based on the scene information.

[0149] According to one embodiment of the present invention, the early warning module 403 is further configured to:

[0150] Adjusting the vehicle's route based on the scene information; and / or,

[0151] Generate warning information based on the scenario information and send it to the warning module and / or vehicle simulator on the vehicle side, so that the warning module and / or vehicle simulator performs a warning operation.

[0152] According to one embodiment of the present invention, the early warning module 403 is further configured to:

[0153] If it is determined that there is no abnormality in the scene based on the scene danger level, the scene video will be sent to the vehicle simulator on the vehicle side for display.

[0154] By applying the early warning method provided in the embodiment of the present invention, the vehicle side adopts different methods to obtain the scene video collected by the drone according to the position of the drone. Therefore, regardless of whether the vehicle side and the drone can communicate, the scene video can be obtained, avoiding the situation where no early warning can be issued when communication cannot be achieved. By identifying the scene video, an early warning is issued for the road conditions ahead of the vehicle, thereby improving the timeliness and comprehensiveness of the early warning and thereby improving driving safety.

[0155] The above is a schematic diagram of a vehicle-side warning device according to an embodiment of the present invention. It should be noted that the technical solution of this vehicle-side warning device and the technical solution of the vehicle-side warning method described above are based on the same concept. For details not described in detail in the technical solution of the vehicle-side warning device, please refer to the description of the technical solution of the vehicle-side warning method described above.

[0156] FIG5 is a schematic diagram of the structure of another early warning device provided in an embodiment of the present invention, which is applied to a drone. The early warning device may include:

[0157] The acquisition module 501 is used to collect the scene video in front of the vehicle during the flight and send the location information to the vehicle end;

[0158] The sending module 502 is used to send the scene video to the vehicle side based on the location information, so that the vehicle side can perform early warning recognition on the scene video, obtain the recognition result of the scene video, and issue an early warning according to the recognition result.

[0159] According to one embodiment of the present invention, the sending module 502 is further configured to:

[0160] If feedback from the vehicle side in response to the location information is received within a preset time period, the scene video is sent to the vehicle side;

[0161] If no feedback in response to the position information is received from the vehicle end within a preset time period, the scene video is sent to the satellite so that the satellite sends the scene video to the vehicle end.

[0162] By applying the early warning method provided in the embodiment of the present invention, the vehicle side adopts different methods to obtain the scene video collected by the drone according to the position of the drone. Therefore, regardless of whether the vehicle side and the drone can communicate, the scene video can be obtained, avoiding the situation where no early warning can be issued when communication cannot be achieved. By identifying the scene video, an early warning is issued for the road conditions ahead of the vehicle, thereby improving the timeliness and comprehensiveness of the early warning and thereby improving driving safety.

[0163] The above is a schematic diagram of an early warning device for use with a drone according to an embodiment of the present invention. It should be noted that the technical solution of this early warning device for use with a drone shares the same concept as the technical solution of the early warning method for use with a drone described above. For details not described in detail in the technical solution of the early warning device for use with a drone, please refer to the description of the technical solution of the early warning method for use with a drone described above.

[0164] Figure 6 is a schematic diagram of the structure of a computing device provided according to an embodiment of the present invention. The computing device 600 includes: a memory 601, a processor 602, and a computer program stored in the memory 601 and executable on the processor 602. When the processor 602 executes the computer program, it implements an early warning method as provided in any of the above embodiments.

[0165] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements an early warning method as proposed in any of the above embodiments.

[0166] It should be noted that the logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device), or in conjunction with such instruction execution system, apparatus, or device. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transmit a program for use by an instruction execution system, apparatus, or device, or in conjunction with such instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering, or processing in another suitable manner as necessary, and then stored in a computer memory.

[0167] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0168] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that a specific feature, structure, material, or characteristic described in conjunction with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0169] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0170] In the present invention, unless otherwise specified or limited, the terms "installed," "connected," "connect," "fixed," etc. should be understood in a broad sense. For example, they can refer to fixed connection, detachable connection, or integration; mechanical connection, electrical connection; direct connection, or indirect connection through an intermediate medium; internal communication between two components, or interaction between two components, unless otherwise specified. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0171] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A warning method, characterized in that, Applied to the vehicle end, the method includes: Obtain the position information of the drone; Based on the position information, obtain the scene video collected during the flight of the drone; Perform early warning recognition on the scene video to obtain the recognition result of the scene video, and issue an early warning according to the recognition result.

2. The method according to claim 1, wherein The step of obtaining the scene video collected during the flight of the drone based on the position information includes: If it is determined based on the position information that the drone is in the communication area and the communication between the drone and the vehicle end is normal, obtain the scene video from the drone through the communication module of the vehicle end.

3. The method according to claim 1, characterized in that, The step of obtaining the scene video collected during the flight of the drone based on the position information includes: If it is determined based on the position information that the drone is not in the communication area, or the drone is in the communication area but the communication with the vehicle end is abnormal, send a scene video acquisition request to the satellite, and obtain the scene video from the satellite through the satellite module of the vehicle end, where the scene video is sent to the satellite by the drone when it cannot communicate with the vehicle end.

4. The method according to claim 2 or 3, characterized in that Determining that the communication between the drone and the vehicle end is abnormal includes: If no information from the drone is received within a preset time period, determine that the communication between the drone and the vehicle end is abnormal.

5. The method according to any one of claims 1 to 3, characterized in that After obtaining the position information of the drone, it further includes: If it is determined based on the position information that the drone is in a no-fly zone, control the drone to fly to a non-no-fly zone.

6. The method according to any one of claims 1 to 3, characterized in that After obtaining the position information of the drone, it further includes: If it is determined based on the position information that the drone is at the edge of the non-no-fly zone or the edge of the communication area, generate a warning message and adjust the flight route of the drone.

7. The method according to any one of claims 1 to 3, characterized in that The step of issuing an early warning according to the recognition result includes: Parse the recognition result through the processing module of the vehicle end to obtain the scene information and the scene danger level of the scene corresponding to the scene video; If it is determined according to the scene danger level that there is an abnormality in the scene, take corresponding warning measures according to the scene information.

8. The method according to claim 7, wherein The step of taking corresponding warning measures according to the scene information includes: Adjust the driving route of the vehicle according to the scene information; and / or, Generate a warning message according to the scene information and send it to the warning module and / or the in-vehicle simulator of the vehicle end, so that the warning module and / or the in-vehicle simulator perform warning operations.

9. The method according to claim 7, wherein After parsing the recognition result through the processing module of the vehicle end to obtain the scene information and the scene danger level of the scene corresponding to the scene video, it further includes: If it is determined according to the scene danger level that there is no abnormality in the scene, send the scene video to the in-vehicle simulator of the vehicle end for display.

10. A warning method, characterized in that, Applied to the drone, the method includes: Collect the scene video in front of the vehicle during flight and send the position information to the vehicle end; Based on the position information, send the scene video to the vehicle end, so that the vehicle end performs early warning recognition on the scene video to obtain the recognition result of the scene video, and issues an early warning according to the recognition result.

11. The method according to claim 10, wherein The step of sending the scene video to the vehicle end based on the position information includes: If the feedback from the vehicle terminal in response to the location information is received within the preset duration, send the scenario video to the vehicle terminal; If the feedback from the vehicle terminal in response to the location information is not received within the preset duration, send the scenario video to the satellite so that the satellite sends the scenario video to the vehicle terminal.

12. An early warning device, characterized in that, Applied to the vehicle terminal, the device includes: A first acquisition module, configured to acquire the location information of the drone; A second acquisition module, configured to acquire the scenario video collected during the flight of the drone based on the location information; An early warning module, configured to perform early warning identification on the scenario video, obtain the identification result of the scenario video, and perform early warning according to the identification result.

13. An early warning device, characterized in that, Applied to the drone, the device includes: An acquisition module, configured to collect the scenario video in front of the vehicle during flight and send the location information to the vehicle terminal; A sending module, configured to send the scenario video to the vehicle terminal based on the location information, so that the vehicle terminal performs early warning identification on the scenario video, obtains the identification result of the scenario video, and performs early warning according to the identification result.

14. An early warning system, characterized in that, The system includes a drone and a vehicle terminal; The drone is configured to collect the scenario video in front of the vehicle during driving and send the location information to the vehicle terminal; The vehicle terminal is configured to acquire the scenario video collected during the flight of the drone based on the location information, perform early warning identification on the scenario video, obtain the identification result of the scenario video, and perform early warning according to the identification result.

15. The system according to claim 14, wherein The system further includes a satellite; The drone is further configured to send the scenario video to the satellite when it is unable to communicate with the vehicle terminal; The vehicle terminal is further configured to send a scenario video acquisition request to the satellite when it is determined based on the location information that the drone is not in the communication area or although it is in the communication area but the communication between the drone and the vehicle terminal is abnormal; The satellite is configured to send the scenario video to the vehicle terminal in response to the scenario video acquisition request.

16. A computing device, characterized in that, Includes: A memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the early warning method according to any one of claims 1-9 or any one of claims 10-11 is implemented.

17. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, the early warning method according to any one of claims 1-9 or any one of claims 10-11 is implemented.

Images