Vehicle-based high-altitude object throwing monitoring method, system, device and storage medium

Abstract

The present invention discloses a vehicle-based method, system, device, and storage medium for monitoring objects dropped from a height. The method comprises the following steps: obtaining depth image information of the area above a target vehicle, inputting the depth image information into a pre-trained object dropping recognition model to determine whether objects are dropped from a height; when objects are dropped from a height, determining a first motion trajectory of the object dropped from a height based on multiple frames of continuous depth image information, and predicting the starting and landing positions of the object dropped from a height based on the first motion trajectory; judging whether the object dropped from a height poses a threat to the target vehicle based on the landing position and the type of the object dropped from a height, and if so, controlling the target vehicle to take body protection measures; generating evidence information for objects dropped from a height based on the starting and landing positions and the depth image information, and transmitting the information to an alarm platform or the owner of the target vehicle. The present invention can monitor the behavior of objects dropped from a height in real time and determine the starting and landing positions for real-time evidence, thereby reducing the risks brought by objects dropped from a height, and can be applied to the field of vehicle safety detection technology.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle safety monitoring, and in particular to a vehicle-based high-altitude object throwing monitoring method, system, device and storage medium. Background Art

[0002] In recent years, the phenomenon of objects being thrown from high places in residential areas has occurred frequently, posing a huge safety hazard to pedestrians and vehicles. In the existing technology, most of the methods are to install high-altitude cameras around the buildings to shoot and monitor the area above. This type of method cannot accurately inform pedestrians and vehicles of the risk of objects being thrown from high places in real time. It can often only be used for retrospective purposes and cannot guarantee the safety of pedestrians and vehicles. In related technologies, high-altitude cameras are installed on the roof to monitor and warn of objects being thrown from high places above the vehicle. However, this method cannot take timely countermeasures even if objects are detected when the owner is not in the car. On the other hand, it cannot quickly lock the throwing location of the objects thrown from high places and provide evidence. In addition, it also ignores the reminders to pedestrians, and likewise cannot guarantee the safety of pedestrians and vehicles. Summary of the Invention

[0003] The purpose of the present invention is to solve one of the technical problems existing in the prior art to at least a certain extent.

[0004] To this end, one purpose of an embodiment of the present invention is to provide a vehicle-based method for monitoring objects thrown from high places, which reduces the risks brought by objects thrown from high places, ensures the safety of pedestrians and vehicles to a certain extent, and can provide real-time evidence of objects thrown from high places.

[0005] Another object of an embodiment of the present invention is to provide a vehicle-based high-altitude object throwing monitoring system.

[0006] In order to achieve the above technical objectives, the technical solutions adopted by the embodiments of the present invention include:

[0007] In a first aspect, an embodiment of the present invention provides a vehicle-based method for monitoring objects thrown from a height, comprising the following steps:

[0008] Acquire depth image information of the area above the target vehicle in real time, input the depth image information into a pre-trained high-altitude object recognition model, and determine whether there is a high-altitude object based on the high-altitude object recognition result;

[0009] When there is a high-altitude object, determining a first motion trajectory of the high-altitude object based on multiple frames of depth image information, and predicting a take-off position and a landing position of the high-altitude object based on the first motion trajectory;

[0010] Determining whether the object thrown from a high altitude poses a threat to the target vehicle based on the landing position and the type of the object thrown from a high altitude, and if so, controlling the target vehicle to take vehicle body protection measures;

[0011] Evidence information of objects thrown from high places is generated according to the throwing position and the depth image information, and the evidence information of objects thrown from high places is sent to a preset alarm platform or the owner of the target vehicle.

[0012] Furthermore, in one embodiment of the present invention, the step of acquiring the depth image information of the area above the target vehicle in real time is specifically as follows:

[0013] When the target vehicle is in a parked state, a binocular depth camera installed on the roof of the target vehicle is started, and the depth image information of the area above the target vehicle is continuously acquired through the binocular depth camera.

[0014] Furthermore, in one embodiment of the present invention, the method for monitoring objects thrown from a height further includes the step of pre-training the object thrown from a height recognition model, which specifically includes:

[0015] Acquire a plurality of preset high-altitude parabolic object depth images, and determine the high-altitude parabolic object type label corresponding to each of the high-altitude parabolic object depth images;

[0016] Convert the high-altitude parabolic depth image into three-dimensional point cloud sample data, and construct a training data set based on the three-dimensional point cloud sample data and the corresponding high-altitude parabolic type label;

[0017] The training data set is input into a pre-built convolutional neural network for training to obtain the trained high-altitude projectile recognition model.

[0018] Furthermore, in one embodiment of the present invention, the step of inputting the training data set into a pre-built convolutional neural network for training to obtain the trained high-altitude object recognition model specifically includes:

[0019] Inputting the training data set into the convolutional neural network to obtain a high-altitude object prediction result;

[0020] Determining a loss value of the convolutional neural network according to the high-altitude object throwing prediction result and the high-altitude object throwing type label;

[0021] Updating the model parameters of the convolutional neural network through a back-propagation algorithm according to the loss value, and returning to the step of inputting the training data set into the convolutional neural network;

[0022] When the loss value reaches a preset first threshold or the number of iterations reaches a preset second threshold, the training is stopped to obtain a trained high-altitude projectile recognition model.

[0023] Furthermore, in one embodiment of the present invention, the step of determining the first motion trajectory of the high-altitude object based on multiple consecutive frames of depth image information, and predicting the take-off position and landing position of the high-altitude object based on the first motion trajectory specifically includes:

[0024] Performing differential processing on the depth image information of the current frame and the depth image information of the previous frame to obtain foreground image information of the high-altitude parabolic object corresponding to the depth image information of the current frame, and determining the first position coordinates of the high-altitude parabolic object based on the foreground image information;

[0025] Determine a first motion trajectory of the high-altitude object according to the first position coordinates corresponding to multiple consecutive frames of depth image information;

[0026] Determining three-dimensional spatial position information of a plurality of high-altitude buildings based on the depth image information;

[0027] The launching position is determined according to the first motion trajectory and the three-dimensional spatial position information, and the landing position is determined according to the first motion trajectory and the horizontal plane where the binocular depth camera is located.

[0028] Furthermore, in one embodiment of the present invention, the step of determining whether the object thrown from a high altitude poses a threat to the target vehicle based on the landing position and the type of the object thrown from a high altitude, and if so, controlling the target vehicle to take vehicle body protection measures, specifically includes:

[0029] Determining whether the high-altitude object will fall on the current position of the target vehicle based on the landing position;

[0030] Determining the type of the object thrown from high altitude according to the high altitude object recognition result, and determining the danger level of the object thrown from high altitude according to the type of the object;

[0031] When the object thrown from a high altitude will fall on the current position of the target vehicle and the danger level is greater than or equal to a preset third threshold, it is determined that the object thrown from a high altitude poses a threat to the target vehicle;

[0032] When it is determined that the object thrown from a high altitude poses a threat to the target vehicle, the target vehicle is controlled to activate an airbag protection device arranged on the roof of the target vehicle.

[0033] Furthermore, in one embodiment of the present invention, the method for monitoring objects thrown from a high altitude further comprises the following steps:

[0034] When there is a risk of objects being thrown from a high altitude, a voice broadcast device installed on the target vehicle is used to remind people around the target vehicle that there is a risk of objects being thrown from a high altitude.

[0035] In a second aspect, an embodiment of the present invention provides a vehicle-based high-altitude object throwing monitoring system, comprising:

[0036] A high-altitude object recognition module is used to obtain depth image information of the area above the target vehicle in real time, input the depth image information into a pre-trained high-altitude object recognition model, and determine whether there is a high-altitude object based on the high-altitude object recognition result;

[0037] A motion trajectory determination module is used to determine a first motion trajectory of the high-altitude object based on multiple frames of depth image information when there is a high-altitude object, and predict the take-off position and landing position of the high-altitude object based on the first motion trajectory;

[0038] a vehicle body protection control module, configured to determine whether the object thrown from a high altitude poses a threat to the target vehicle based on the landing position and the type of the object thrown from a high altitude, and if so, control the target vehicle to take vehicle body protection measures;

[0039] The high-altitude object dropping evidence module is used to generate high-altitude object dropping evidence information based on the throwing position and the depth image information, and send the high-altitude object dropping evidence information to a preset alarm platform or the owner of the target vehicle.

[0040] In a third aspect, an embodiment of the present invention provides a vehicle-based high-altitude object throwing monitoring device, comprising:

[0041] at least one processor;

[0042] at least one memory for storing at least one program;

[0043] When the at least one program is executed by the at least one processor, the at least one processor implements the above-mentioned vehicle-based high-altitude object throwing monitoring method.

[0044] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium storing a program executable by a processor, wherein the program executable by the processor is used to execute the above-mentioned vehicle-based high-altitude object throwing monitoring method when executed by the processor.

[0045] The advantages and benefits of the present invention will be described in part in the following description and will become apparent from the following description or learned through practice of the present invention:

[0046] The embodiment of the present invention acquires depth image information of the area above the target vehicle in real time, inputs the depth image information into a pre-trained high-altitude object throwing recognition model, and determines whether there is high-altitude object throwing based on the high-altitude object throwing recognition result. If there is high-altitude object throwing, the first motion trajectory of the high-altitude object throwing is determined based on the continuous multi-frame depth image information, and the starting and landing positions of the high-altitude object throwing are predicted based on the first motion trajectory. Then, based on the landing position and the type of object thrown from high altitude, it is determined whether the high-altitude object throwing poses a threat to the target vehicle. If so, the target vehicle is controlled to take body protection measures. In addition, high-altitude object throwing evidence information is generated based on the starting and landing positions and the depth image information, and the high-altitude object throwing evidence information is sent to a preset alarm platform or the owner of the target vehicle. The embodiment of the present invention can monitor the high-altitude object throwing behavior in real time and determine the starting and landing positions for real-time evidence through the acquisition of depth image information and model recognition, which facilitates the timely locating of the perpetrators, reduces the risks brought by high-altitude object throwing, and ensures the safety of pedestrians and vehicles to a certain extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following introduction is made to the drawings required for use in the embodiments of the present invention. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative work.

[0048] Figure 1 A flowchart of a vehicle-based method for monitoring objects thrown from a height provided by an embodiment of the present invention;

[0049] Figure 2 A structural block diagram of a vehicle-based high-altitude object throwing monitoring system provided in an embodiment of the present invention;

[0050] Figure 3 This is a structural block diagram of a vehicle-based high-altitude object throwing monitoring device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0051] The embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention and are not to be construed as limiting the present invention. The step numbers in the following embodiments are provided for ease of explanation only and do not limit the order of the steps. The order of execution of the steps in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0052] In the description of the present invention, "a plurality" means two or more. The terms "first" and "second" are used solely to distinguish technical features and are not to be construed as indicating or implying relative importance, or as implicitly indicating the number of the indicated technical features, or as implicitly indicating the order of the indicated technical features. Furthermore, unless otherwise defined, all technical and scientific terms used herein have the same meanings as commonly understood by those skilled in the art.

[0053] Reference Figure 1 The embodiment of the present invention provides a vehicle-based high-altitude object dropping monitoring method, which specifically includes the following steps:

[0054] S101. Acquire depth image information of the area above the target vehicle in real time, input the depth image information into a pre-trained high-altitude object recognition model, and determine whether high-altitude object recognition exists based on the high-altitude object recognition result.

[0055] Specifically, in computer vision systems, three-dimensional scene information provides more possibilities for various computer vision applications such as image segmentation, target detection, and object tracking, and depth images (Depth map) have been widely used as a universal way to express three-dimensional scene information. The grayscale value of each pixel in the depth image can be used to characterize the distance between a certain point in the scene and the camera. In an embodiment of the present invention, by acquiring the depth image information of the area above the target vehicle, the distance between the high-altitude projectile and the camera device can be accurately perceived, thereby accurately determining the actual position of the high-altitude projectile.

[0056] As an optional implementation, the step of acquiring the depth image information of the area above the target vehicle in real time is specifically as follows:

[0057] When the target vehicle is in a parked state, the binocular depth camera installed on the roof of the target vehicle is started, and the depth image information of the area above the target vehicle is continuously obtained through the binocular depth camera.

[0058] Specifically, the embodiment of the present invention uses a binocular depth camera to obtain a depth image of the area above the target vehicle. In some optional embodiments, a laser radar can also be used for depth imaging.

[0059] As an optional embodiment, the method for monitoring objects thrown from a height may further include the step of pre-training a model for identifying objects thrown from a height, which specifically includes:

[0060] A1. Obtain multiple preset high-altitude object depth images and determine the high-altitude object type label corresponding to each high-altitude object depth image;

[0061] A2. Convert the high-altitude parabolic depth image into 3D point cloud sample data, and construct a training dataset based on the 3D point cloud sample data and the corresponding high-altitude parabolic type labels;

[0062] A3. Input the training dataset into the pre-built convolutional neural network for training to obtain a trained high-altitude object recognition model.

[0063] Specifically, a sufficient number of depth images of high-altitude objects of different object types are obtained, and the label information of each depth image of high-altitude objects is determined according to the corresponding object type, such as the label of paper is (1), the label of metal products is (2), and so on; in addition, multiple sample images without high-altitude objects can also be obtained, and the label can be set to (0).

[0064] The high-altitude parabolic depth image is converted into three-dimensional point cloud sample data to facilitate the extraction of image features; a training data set can be formed based on the obtained three-dimensional point cloud sample data and the corresponding labels.

[0065] As an optional implementation, the step of inputting the training data set into a pre-built convolutional neural network for training to obtain a trained high-altitude object recognition model specifically includes:

[0066] A31. Input the training data set into the convolutional neural network to obtain the high-altitude object prediction results;

[0067] A32. Determine the loss value of the convolutional neural network based on the high-altitude object throwing prediction result and the high-altitude object throwing type label;

[0068] A33. Update the model parameters of the convolutional neural network through the back propagation algorithm according to the loss value, and return to the step of inputting the training data set into the convolutional neural network;

[0069] A34. When the loss value reaches a preset first threshold or the number of iterations reaches a preset second threshold, the training is stopped to obtain a trained high-altitude object recognition model.

[0070] Specifically, after inputting the data from the training dataset into the initialized convolutional neural network model, the model outputs a recognition result, namely a prediction result for high-altitude objects. The accuracy of the model's prediction can be evaluated based on the high-altitude objects prediction result and the aforementioned label information, thereby updating the model's parameters. For the high-altitude objects recognition model, the accuracy of the model's prediction result can be measured using a loss function. The loss function is defined on a single training data point and is used to measure the prediction error of a training data point. Specifically, the loss value of a training data point is determined by the label of the single training data point and the model's prediction result for that training data point. In actual training, a training dataset contains a lot of training data, so a cost function is generally used to measure the overall error of the training dataset. The cost function is defined on the entire training dataset and is used to calculate the average prediction error of all training data points, which can better measure the model's prediction effect. For general machine learning models, the aforementioned cost function, plus a regularization term to measure the model's complexity, can be used as the training objective function. Based on this objective function, the loss value of the entire training dataset can be calculated. There are many types of commonly used loss functions, such as 0-1 loss function, square loss function, absolute loss function, logarithmic loss function, cross entropy loss function, etc., which can all be used as loss functions of machine learning models, which will not be elaborated here one by one. In an embodiment of the present invention, any one of the loss functions can be selected to determine the loss value of training. Based on the loss value of training, the parameters of the model are updated using the back propagation algorithm, and a trained high-altitude parabolic recognition model can be obtained by iterating for several rounds. Specifically, the number of iterations can be pre-set, or the training is considered to be completed when the test set meets the accuracy requirements.

[0071] When identifying the depth image information of the area above the target vehicle, the depth image information is also first converted into the corresponding three-dimensional point cloud, and then input into the high-altitude object throwing recognition model to obtain the high-altitude object throwing recognition results output by the model, such as paper-type high-altitude objects, metal product-type high-altitude objects, no high-altitude objects, etc.

[0072] S102: When there is a high-altitude object, determine a first motion trajectory of the high-altitude object based on multiple frames of depth image information, and predict a take-off position and a landing position of the high-altitude object based on the first motion trajectory.

[0073] As an optional embodiment, determining a first motion trajectory of a high-altitude object based on continuous multi-frame depth image information, and predicting a take-off position and a landing position of the high-altitude object based on the first motion trajectory specifically includes:

[0074] S1021, performing differential processing on the depth image information of the current frame and the depth image information of the previous frame to obtain foreground image information of the high-altitude projectile corresponding to the depth image information of the current frame, and determining the first position coordinates of the high-altitude projectile based on the foreground image information;

[0075] S1022: Determine a first motion trajectory of a high-altitude object according to first position coordinates corresponding to a plurality of consecutive frames of depth image information;

[0076] S1023. Determine three-dimensional spatial position information of several high-altitude buildings based on the depth image information;

[0077] S1024: Determine a launching position according to the first motion trajectory and the three-dimensional spatial position information, and determine a landing position according to the first motion trajectory and the horizontal plane where the binocular depth camera is located.

[0078] Specifically, when high-altitude objects are detected for the first time, it means that high-altitude objects appear for the first time in the current frame depth image. At this time, the previous frame depth image can be used as the background image of the current frame depth image; through differential processing, the foreground image of the high-altitude objects in the current frame depth image can be obtained quickly and accurately, thereby determining the position coordinates of the high-altitude objects; when high-altitude objects are detected again subsequently, since the previous depth images have already determined the foreground image and location of the high-altitude objects, differential processing can also be used to quickly obtain the foreground image of the high-altitude objects in the current frame depth image, thereby determining its position coordinates.

[0079] After determining a certain number of first position coordinates, the first motion trajectory of the high-altitude projectile can be determined based on the relevant mechanical model; at the same time, the depth image information also records the depth information of the surrounding high-altitude buildings, and their three-dimensional spatial position information can be obtained through three-dimensional reconstruction technology; the intersection of the first motion trajectory and the three-dimensional spatial position information can be determined as the take-off position, and the intersection of the first motion trajectory and the horizontal plane where the binocular depth camera is located can be determined as the landing position.

[0080] S103: Determine whether the object thrown from high altitude poses a threat to the target vehicle based on the landing location and the type of the object thrown from high altitude. If so, control the target vehicle to take vehicle body protection measures.

[0081] Specifically, when the landing position is at the location of the target vehicle and the type of object thrown from a high altitude has a high risk level, it can be determined that there is a threat to the target vehicle and vehicle protection measures can be taken. Step S103 specifically includes the following steps:

[0082] S1031. Determine, based on the landing position, whether the high-altitude thrown object will fall on the current position of the target vehicle;

[0083] S1032. Determine the type of the object thrown from high altitude based on the result of the high altitude object recognition, and determine the danger level of the object thrown from high altitude based on the type of the object;

[0084] S1033: When the object thrown from high altitude will fall on the current position of the target vehicle and the danger level is greater than or equal to a preset third threshold, it is determined that the object thrown from high altitude poses a threat to the target vehicle;

[0085] S1034. When it is determined that objects thrown from a high altitude pose a threat to the target vehicle, the target vehicle is controlled to activate an airbag protection device installed on the roof of the target vehicle.

[0086] Specifically, different types of items can be pre-set with different danger levels, such as the danger level of paper can be set to 0, the danger level of metal products can be set to 5, the danger level of plastic products can be set to 3, and so on; when it is determined that objects thrown from high altitude will fall at the current position of the target vehicle, and the danger level of the objects thrown from high altitude is greater than or equal to a third threshold value (such as 2), it can be determined that the objects thrown from high altitude pose a threat to the target vehicle; the airbag protection device on the roof of the target vehicle is activated to cover the body of the target vehicle to prevent objects thrown from high altitude from causing damage to the target vehicle.

[0087] S104: Generate evidence information of objects thrown from high places based on the throwing position and depth image information, and send the evidence information of objects thrown from high places to a preset alarm platform or the owner of the target vehicle.

[0088] Specifically, after identifying the origin of an object, regardless of whether it poses a threat to the target vehicle, the origin and corresponding multiple depth images are sent as evidence to the alarm platform or the vehicle owner's user terminal, facilitating rapid identification of the perpetrator. For example, the origin can be annotated in the depth image to provide evidence.

[0089] As an optional embodiment, the method for monitoring objects thrown from a high altitude further includes the following steps:

[0090] When there is a risk of objects being thrown from a high altitude, a voice broadcast device installed on the target vehicle will be used to remind people around the target vehicle that there is a risk of objects falling from a high altitude.

[0091] Specifically, after identifying objects thrown from high places, regardless of whether the objects thrown from high places will pose a threat to the target vehicle, a voice reminder will be given through the voice broadcast device installed on the target vehicle to prevent the objects thrown from high places from causing harm to surrounding people, further ensuring the safety of pedestrians.

[0092] The above describes the method steps of the embodiment of the present invention. It can be appreciated that the embodiment of the present invention, through the acquisition of depth image information and model recognition, can monitor the behavior of throwing objects from high places in real time and determine the location of the throwing object for real-time evidence, thereby facilitating the timely identification of the perpetrator, reducing the risks posed by throwing objects from high places, and ensuring the safety of pedestrians and vehicles to a certain extent.

[0093] Reference Figure 2 The embodiment of the present invention provides a vehicle-based high-altitude object throwing monitoring system, comprising:

[0094] The high-altitude object recognition module is used to obtain the depth image information of the area above the target vehicle in real time, input the depth image information into the pre-trained high-altitude object recognition model, and determine whether there is a high-altitude object based on the high-altitude object recognition result;

[0095] A motion trajectory determination module is used to determine a first motion trajectory of a high-altitude object based on continuous multi-frame depth image information when there is a high-altitude object, and to predict a take-off position and a landing position of the high-altitude object based on the first motion trajectory;

[0096] The vehicle protection control module is used to determine whether the high-altitude objects pose a threat to the target vehicle based on the landing location and the type of objects thrown from high altitude. If so, it controls the target vehicle to take body protection measures;

[0097] The high-altitude object dropping evidence module is used to generate high-altitude object dropping evidence information based on the throwing position and depth image information, and send the high-altitude object dropping evidence information to a preset alarm platform or the owner of the target vehicle.

[0098] The contents of the above method embodiments are all applicable to the present system embodiments. The functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0099] Reference Figure 3 The embodiment of the present invention provides a vehicle-based high-altitude object throwing monitoring device, comprising:

[0100] at least one processor;

[0101] at least one memory for storing at least one program;

[0102] When the at least one program is executed by the at least one processor, the at least one processor implements the vehicle-based high-altitude object throwing monitoring method.

[0103] The contents of the above method embodiments are all applicable to the present device embodiments. The functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0104] An embodiment of the present invention also provides a computer-readable storage medium, which stores a program executable by a processor. When the program is executed by the processor, it is used to execute the above-mentioned vehicle-based high-altitude object throwing monitoring method.

[0105] A computer-readable storage medium according to an embodiment of the present invention can execute a vehicle-based high-altitude object throwing monitoring method provided by an embodiment of the method of the present invention, can execute any combination of implementation steps of the method embodiment, and has the corresponding functions and beneficial effects of the method.

[0106] The embodiment of the present invention also discloses a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs Figure 1 The method shown.

[0107] In some optional embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the above-mentioned boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided in an exemplary manner for the purpose of providing a more comprehensive understanding of the technology. The disclosed method is not limited to the operation and logic flow presented herein. Optional embodiments are contemplated in which the order of the various operations is changed and the sub-operations described as a part of a larger operation are performed independently.

[0108] In addition, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the above-mentioned functions and / or features can be integrated into a single physical device and / or software module, or one or more functions and / or features can be implemented in separate physical devices or software modules. It is also understood that a detailed discussion of the actual implementation of each module is not necessary for understanding the present invention. More specifically, given the properties, functions, and internal relationships of the various functional modules in the devices disclosed herein, the actual implementation of the module will be understood within the routine skills of an engineer. Therefore, a person skilled in the art can implement the present invention set forth in the claims using ordinary skills without undue experimentation. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0109] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the above methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program code.

[0110] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0111] More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable media on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, deciphering, or processing in another suitable manner as necessary, and then stored in a computer memory.

[0112] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0113] In the above description of this specification, reference to the terms "one embodiment / example," "another embodiment / example," or "certain embodiments / examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0114] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to the embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the claims and their equivalents.

[0115] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A vehicle-based method for monitoring objects thrown from a high altitude, characterized in that: The following steps are involved: Acquire depth image information of the area above the target vehicle in real time, input the depth image information into a pre-trained high-altitude object recognition model, and determine whether there is a high-altitude object based on the high-altitude object recognition result; When there is a high-altitude object, determining a first motion trajectory of the high-altitude object based on multiple frames of depth image information, and predicting a take-off position and a landing position of the high-altitude object based on the first motion trajectory; Determining whether the object thrown from a high altitude poses a threat to the target vehicle based on the landing position and the type of the object thrown from a high altitude, and if so, controlling the target vehicle to take vehicle body protection measures; Generate high-altitude object throwing evidence information based on the throwing location and the depth image information, and send the high-altitude object throwing evidence information to a preset alarm platform or the owner of the target vehicle; The step of acquiring the depth image information of the area above the target vehicle in real time is specifically as follows: When the target vehicle is in a parked state, starting a binocular depth camera installed on the roof of the target vehicle, and continuously acquiring the depth image information of the area above the target vehicle through the binocular depth camera; The step of determining a first motion trajectory of the high-altitude object based on multiple consecutive frames of depth image information, and predicting a take-off position and a landing position of the high-altitude object based on the first motion trajectory specifically includes: Performing differential processing on the depth image information of the current frame and the depth image information of the previous frame to obtain foreground image information of the high-altitude parabolic object corresponding to the depth image information of the current frame, and determining the first position coordinates of the high-altitude parabolic object based on the foreground image information; Determine a first motion trajectory of the high-altitude object according to the first position coordinates corresponding to multiple consecutive frames of depth image information; Determining three-dimensional spatial position information of a plurality of high-altitude buildings based on the depth image information; The launching position is determined according to the first motion trajectory and the three-dimensional spatial position information, and the landing position is determined according to the first motion trajectory and the horizontal plane where the binocular depth camera is located.

2. A vehicle-based high-altitude object throwing monitoring method according to claim 1, characterized in that: The method for monitoring objects thrown from a height further includes the step of pre-training the object thrown from a height recognition model, which specifically includes: Acquire a plurality of preset high-altitude parabolic object depth images, and determine the high-altitude parabolic object type label corresponding to each of the high-altitude parabolic object depth images; Convert the high-altitude parabolic depth image into three-dimensional point cloud sample data, and construct a training data set based on the three-dimensional point cloud sample data and the corresponding high-altitude parabolic type label; The training data set is input into a pre-built convolutional neural network for training to obtain the trained high-altitude projectile recognition model.

3. A vehicle-based high-altitude object throwing monitoring method according to claim 2, characterized in that: The step of inputting the training data set into a pre-built convolutional neural network for training to obtain the trained high-altitude object recognition model specifically includes: Inputting the training data set into the convolutional neural network to obtain a high-altitude object prediction result; Determining a loss value of the convolutional neural network according to the high-altitude object throwing prediction result and the high-altitude object throwing type label; Updating the model parameters of the convolutional neural network through a back-propagation algorithm according to the loss value, and returning to the step of inputting the training data set into the convolutional neural network; When the loss value reaches a preset first threshold or the number of iterations reaches a preset second threshold, the training is stopped to obtain a trained high-altitude projectile recognition model.

4. The vehicle-based high-altitude object throwing monitoring method according to claim 1 is characterized in that: The step of determining whether the object thrown from a high altitude poses a threat to the target vehicle based on the landing position and the type of the object thrown from a high altitude, and if so, controlling the target vehicle to take vehicle body protection measures, specifically includes: Determining whether the high-altitude object will fall on the current position of the target vehicle based on the landing position; Determining the type of the object thrown from high altitude according to the high altitude object recognition result, and determining the danger level of the object thrown from high altitude according to the type of the object; When the object thrown from a high altitude will fall on the current position of the target vehicle and the danger level is greater than or equal to a preset third threshold, it is determined that the object thrown from a high altitude poses a threat to the target vehicle; When it is determined that the object thrown from a high altitude poses a threat to the target vehicle, the target vehicle is controlled to activate an airbag protection device arranged on the roof of the target vehicle.

5. A vehicle-based high-altitude object throwing monitoring method according to any one of claims 1 to 4, characterized in that: The method for monitoring objects thrown from a height also includes the following steps: When there is a risk of objects being thrown from a high altitude, a voice broadcast device installed on the target vehicle is used to remind people around the target vehicle that there is a risk of objects being thrown from a high altitude.

6. A vehicle-based high-altitude object throwing monitoring system, characterized in that: include: A high-altitude object recognition module is used to obtain depth image information of the area above the target vehicle in real time, input the depth image information into a pre-trained high-altitude object recognition model, and determine whether there is a high-altitude object based on the high-altitude object recognition result; A motion trajectory determination module is used to determine a first motion trajectory of the high-altitude object based on multiple frames of depth image information when there is a high-altitude object, and predict the take-off position and landing position of the high-altitude object based on the first motion trajectory; a vehicle body protection control module, configured to determine whether the object thrown from a high altitude poses a threat to the target vehicle based on the landing position and the type of the object thrown from a high altitude, and if so, control the target vehicle to take vehicle body protection measures; A high-altitude object dropping evidence module is used to generate high-altitude object dropping evidence information based on the throwing location and the depth image information, and send the high-altitude object dropping evidence information to a preset alarm platform or the owner of the target vehicle; The real-time acquisition of depth image information of the area above the target vehicle is specifically as follows: When the target vehicle is in a parked state, starting a binocular depth camera installed on the roof of the target vehicle, and continuously acquiring the depth image information of the area above the target vehicle through the binocular depth camera; The determining of a first motion trajectory of the high-altitude object according to the depth image information of the continuous multiple frames, and predicting a take-off position and a landing position of the high-altitude object according to the first motion trajectory, specifically includes: Performing differential processing on the depth image information of the current frame and the depth image information of the previous frame to obtain foreground image information of the high-altitude parabolic object corresponding to the depth image information of the current frame, and determining the first position coordinates of the high-altitude parabolic object based on the foreground image information; Determine a first motion trajectory of the high-altitude object according to the first position coordinates corresponding to multiple consecutive frames of depth image information; Determining three-dimensional spatial position information of a plurality of high-altitude buildings based on the depth image information; The launching position is determined according to the first motion trajectory and the three-dimensional spatial position information, and the landing position is determined according to the first motion trajectory and the horizontal plane where the binocular depth camera is located.

7. A vehicle-based high-altitude object throwing monitoring device, characterized in that: include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the vehicle-based high-altitude object throwing monitoring method as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a program executable by a processor, characterized in that: The program executable by the processor is used to execute the vehicle-based high-altitude object throwing monitoring method as described in any one of claims 1 to 5 when executed by the processor.

Images