Risk early warning method and device and electronic equipment

Abstract

The invention discloses a risk early warning method and device and electronic equipment. The method comprises the following steps: acquiring a first thermal imaging image and an environment image of a target area at a target moment; under the condition that the image of the moving object exists in the first thermal imaging image, the position relation between the moving object and the target reference object is determined; generating a three-dimensional point cloud image of the target area at the target moment, and determining a target coordinate of the moving object in the three-dimensional point cloud image according to the position relation; generating a first predicted moving track of the moving object in the three-dimensional point cloud image according to the historical coordinates and the target coordinates of the moving object; and determining a second predicted movement track of the target vehicle in the three-dimensional point cloud image, and determining a collision risk of the target vehicle and the moving object according to the first predicted movement track and the second predicted movement track. Through the risk early warning method and device, the problem that the accuracy of risk early warning through single equipment such as a camera in the prior art is low is solved.

Description

Technical Field

[0001] The present application relates to the field of automation, and in particular to a risk warning method, device and electronic equipment. Background Art

[0002] With the acceleration of urbanization and the improvement of people's living standards, the number of vehicles continues to increase, and the road traffic environment is becoming increasingly complex, resulting in an increasingly severe traffic safety situation. Therefore, it is very important to monitor the environment around the vehicle and issue risk warnings while the vehicle is driving.

[0003] Traditional environmental monitoring and risk warning are achieved through single devices such as cameras, but they have certain limitations in complex traffic environments. For example, when a vehicle is driving, pedestrians or other moving objects suddenly appear behind an obstacle on the road. Due to the blind spot, the camera cannot capture the situation behind the obstacle, resulting in the vehicle being unable to avoid it in time, which poses a very serious safety hazard.

[0004] Currently, no effective solution has been proposed to address the problem of low accuracy in risk warnings using single devices such as cameras in related technologies. Summary of the invention

[0005] The present application provides a risk warning method, device and electronic device to solve the problem of low accuracy of risk warning through a single device such as a camera in the related art.

[0006] According to one aspect of the present application, a risk warning method is provided. The method includes: acquiring a first thermal imaging image of a target area at a target time, and acquiring an environmental image of the target area at the target time through an image acquisition device, wherein the target area is located in front of a target vehicle, and the environmental image is an image of the environment in the target area; judging whether there is an image of a moving object in the first thermal imaging image according to the environmental image, and if there is an image of a moving object in the first thermal imaging image, determining the positional relationship between the moving object and the target reference object according to the first thermal imaging image and the environmental image, wherein the target reference object is located in the environmental image, and the target reference object is located between the target vehicle and the moving object; generating a three-dimensional point cloud image of the target area at the target time through a radar laser, and determining the coordinates of the moving object in the three-dimensional point cloud image according to the positional relationship to obtain the target coordinates; acquiring the coordinate information of the moving object at the last acquisition time of the target time, obtaining the historical coordinates, and generating a first predicted moving trajectory of the moving object in the three-dimensional point cloud image according to the historical coordinates and the target coordinates; determining the second predicted moving trajectory of the target vehicle in the three-dimensional point cloud image, and determining the collision risk between the target vehicle and the moving object according to the first predicted moving trajectory and the second predicted moving trajectory.

[0007] Optionally, judging whether there is an image of a moving object in the first thermal imaging image based on the environmental image includes: judging whether there is an abnormal area in the image area of ​​the first thermal imaging image whose temperature value is greater than a preset temperature value; if there is an abnormal area, generating a second thermal imaging image corresponding to the environmental image based on the image content contained in the environmental image; comparing the first thermal imaging image and the second thermal imaging image, and determining that there is an image of a moving object in the first thermal imaging image if the first thermal imaging image and the second thermal imaging image are inconsistent.

[0008] Optionally, determining the positional relationship between the mobile object and the target reference object based on the first thermal imaging image and the environmental image includes: acquiring size information of the target reference object from the environmental image; determining the proportional relationship between the target reference object and the mobile object based on the first thermal imaging image; calculating the distance and angle between the target reference object and the mobile object based on the proportional relationship and the size information, and determining the distance and angle as the positional relationship.

[0009] Optionally, determining the coordinates of the moving object in the three-dimensional point cloud image based on the positional relationship to obtain the target coordinates includes: determining the coordinate information of the target reference object in the three-dimensional point cloud image to obtain reference coordinate information; calculating the coordinate information of the moving object based on the distance, angle and reference coordinate information between the target reference object and the mobile object to obtain the target coordinates.

[0010] Optionally, generating a first predicted movement trajectory of a moving object in a three-dimensional point cloud image based on historical coordinates and target coordinates includes: obtaining a first time difference between a previous acquisition moment of the target moment and the target moment; calculating a distance difference between the historical coordinates and the target coordinates; calculating a first moving speed of the moving object based on the first time difference and the distance difference; calculating coordinate information of the moving object at each acquisition moment after the target moment based on the first moving speed and the first time difference to obtain multiple first predicted coordinates; and generating a first predicted movement trajectory based on the multiple first predicted coordinates.

[0011] Optionally, determining a second predicted movement trajectory of the target vehicle in the three-dimensional point cloud image includes: obtaining a second moving speed of the target vehicle, and calculating the coordinate information of the target vehicle at each collection time after the target time based on the second moving speed and the first time difference, to obtain multiple second predicted coordinates; and generating a second predicted movement trajectory based on the multiple second predicted coordinates.

[0012] Optionally, determining the collision risk between the target vehicle and the moving object based on the first predicted moving trajectory and the second predicted moving trajectory includes: determining the coordinates of the overlapping point of the first predicted moving trajectory and the second predicted moving trajectory; obtaining the arrival time corresponding to the overlapping point coordinates in the first predicted moving trajectory to obtain the first arrival time; obtaining the arrival time corresponding to the overlapping point coordinates in the second predicted moving trajectory to obtain the second arrival time; judging whether the second time difference between the first arrival time and the second arrival time is greater than the preset time difference; if the second time difference is greater than the preset time difference, determining that there is no collision risk between the target vehicle and the moving object; if the second time difference is less than or equal to the preset time difference, determining that there is a collision risk between the target vehicle and the moving object, and displaying a warning message on the display device of the target vehicle, wherein the warning message is used to prompt the driver that there is a moving object in the target area.

[0013] According to another aspect of the present application, a risk warning device is provided. The device comprises: an acquisition unit, which is used to acquire a first thermal imaging image of a target area at a target moment, and to acquire an environmental image of the target area at the target moment through an image acquisition device, wherein the target area is located in front of a target vehicle, and the environmental image is an image of the environment in the target area; a judgment unit, which is used to judge whether there is an image of a moving object in the first thermal imaging image according to the environmental image, and when there is an image of the moving object in the first thermal imaging image, determine the positional relationship between the moving object and a target reference object according to the first thermal imaging image and the environmental image, wherein the target reference object is located in the environmental image, and the target reference object is located between the target vehicle and the moving object; a first determination unit, which is used to generate a three-dimensional point cloud image of the target area at the target moment through a radar laser, and determine the coordinates of the moving object in the three-dimensional point cloud image according to the positional relationship to obtain the target coordinates; a second determination unit, which is used to acquire the coordinate information of the moving object at the last acquisition moment of the target moment, obtain the historical coordinates, and generate the first predicted moving trajectory of the moving object in the three-dimensional point cloud image according to the historical coordinates and the target coordinates; a third determination unit, which is used to determine the second predicted moving trajectory of the target vehicle in the three-dimensional point cloud image, and determine the collision risk between the target vehicle and the moving object according to the first predicted moving trajectory and the second predicted moving trajectory.

[0014] According to another aspect of the present invention, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, a risk warning method provided in the aforementioned embodiment of the present application is implemented.

[0015] According to another aspect of the present invention, an electronic device is also provided, comprising one or more processors and a memory; the memory stores computer-readable instructions, and the processor is used to execute the computer-readable instructions, wherein when the computer-readable instructions are executed, a risk warning method provided in the aforementioned embodiment is executed.

[0016] Through the present application, the following steps are adopted: obtaining a first thermal imaging image of a target area at a target moment, and obtaining an environmental image of the target area at the target moment through an image acquisition device, wherein the target area is located in front of a target vehicle, and the environmental image is an image of the environment within the target area; judging whether there is an image of a moving object in the first thermal imaging image according to the environmental image, and in the case where there is an image of a moving object in the first thermal imaging image, determining the positional relationship between the moving object and a target reference object according to the first thermal imaging image and the environmental image, wherein the target reference object is located in the environmental image, and the target reference object is located between the target vehicle and the moving object; generating a three-dimensional point cloud image of the target area at the target moment through a radar laser, and determining the coordinates of the moving object in the three-dimensional point cloud image according to the positional relationship to obtain the target coordinates; obtaining the coordinate information of the moving object at the previous acquisition moment of the target moment to obtain the historical coordinates, and generating a first predicted moving trajectory of the moving object in the three-dimensional point cloud image according to the historical coordinates and the target coordinates; determining a second predicted moving trajectory of the target vehicle in the three-dimensional point cloud image, and determining the collision risk between the target vehicle and the moving object according to the first predicted moving trajectory and the second predicted moving trajectory. By combining thermal imaging images and environmental images to determine the positional relationship between the moving object and the target reference object, and then combining the target coordinates of the moving object obtained from the three-dimensional point cloud image, the moving trajectory and the coordinate information of the coincidence point of the target vehicle and the moving object are predicted, thereby determining the collision risk between the target vehicle and the moving object, solving the problem of low accuracy of risk warning through a single device such as a camera in related technologies. This has achieved the technical effect of improving the accuracy of risk warnings and reducing the risk of traffic accidents. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0018] Figure 1 is a flow chart of a risk warning method provided according to an embodiment of the present application;

[0019] Figure 2 is a flow chart of an optional risk warning method provided according to an embodiment of the present application;

[0020] Figure 3 is a schematic diagram of a risk warning device provided according to an embodiment of the present application;

[0021] Figure 4 It is a schematic diagram of an electronic device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0022] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0023] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0024] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0025] It should be noted that the risk warning method, device and electronic equipment determined in the present disclosure can be used in the field of automation, and can also be used in any field other than the field of automation. The application field of the risk warning method, device and electronic equipment determined in the present disclosure is not limited.

[0026] It should be noted that the collected information, user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) used in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of relevant data are in compliance with the relevant laws, regulations and standards of the relevant regions, necessary confidentiality measures are taken, and public order and good customs are not violated. Corresponding operation portals are provided for users to choose to authorize or refuse to use. For example, an interface is set up between this system and relevant users or organizations. Before obtaining relevant information, it is necessary to send an acquisition request to the aforementioned user or organization through the interface, and obtain relevant information after receiving the consent information fed back by the aforementioned user or organization.

[0027] The embodiments or examples of the present disclosure are not exhaustive, but are only illustrative of some embodiments or examples, and are not intended to be specific limitations on the scope of protection of the present disclosure. In the absence of contradiction, each step in a certain embodiment or example can be implemented as an independent example, and the steps can be combined arbitrarily. For example, the scheme after removing some steps in a certain embodiment or example can also be implemented as an independent example, and the order of the steps in a certain embodiment or example can be arbitrarily exchanged. In addition, the optional methods or optional examples in a certain embodiment or example can be combined arbitrarily; in addition, the various embodiments or examples can be combined arbitrarily, for example, some or all steps of different embodiments or examples can be combined arbitrarily, and a certain embodiment or example can be combined arbitrarily with the optional methods or optional examples of other embodiments or examples.

[0028] For the convenience of description, some nouns or terms involved in the embodiments of the present application are explained below:

[0029] Thermal imaging image: An image obtained by a thermal imaging sensor that shows the heat distribution of an object. Thermal imaging images usually use different colors to represent the temperature of different parts of an object, so they can be used to detect heat distribution, heat leakage, faults and other problems of an object.

[0030] Three-dimensional point cloud image: an image used to represent a set of points in three-dimensional space. Each point has specific coordinate information in three-dimensional space, and the shape and structure of the entire three-dimensional space can be constructed through the coordinates of these points.

[0031] World coordinate system: A coordinate system used to describe the position and orientation of an object, usually expressed in a three-dimensional Cartesian coordinate system. In a world coordinate system, the origin is usually located at a fixed reference point, and the coordinate axes extend along fixed directions. By using a world coordinate system, the position and orientation of an object in space can be accurately described, making it easier to perform calculations and analysis.

[0032] According to an embodiment of the present application, a risk early warning method is provided.

[0033] Figure 1 is a flow chart of a risk warning method provided according to an embodiment of the present application. Figure 1 As shown, the method comprises the following steps:

[0034] Step S101, acquiring a first thermal imaging image of a target area at a target moment, and acquiring an environmental image of the target area at the target moment by an image acquisition device, wherein the target area is located in front of a target vehicle, and the environmental image is an image of the environment within the target area.

[0035] Specifically, a first thermal imaging image of the area in front of the vehicle at the target moment can be obtained through a thermal imaging sensor, wherein the thermal imaging image displays the shape and characteristics of an object by capturing the heat emitted by it. By analyzing whether the first thermal imaging image contains areas with abnormal temperature, it can be determined whether there are moving objects suspected to be pedestrians, animals, etc. At the same time, an environmental image of the area in front of the vehicle at the target moment can be acquired through an image acquisition device such as a camera. The camera can capture unobstructed pedestrians, animals or vehicles, etc. By comparing the first thermal imaging image with the environmental image, it can be determined whether there is an image of a moving object in the first thermal imaging image.

[0036] Step S102, judging whether there is an image of a moving object in the first thermal imaging image based on the environmental image, and if there is an image of a moving object in the first thermal imaging image, determining the positional relationship between the moving object and the target reference object based on the first thermal imaging image and the environmental image, wherein the target reference object is located in the environmental image and is located between the target vehicle and the mobile object.

[0037] Specifically, by comparing whether there is a difference between the area with abnormal temperature in the first thermal imaging image and the environmental image, it can be determined whether there is an image of a moving object in the first thermal imaging image. For example, if there is an image of a partial area in the area with abnormal temperature in the first thermal imaging image that does not exist in the corresponding environmental image, then the image of this area is the image of the moving object, and the moving object may be a pedestrian or an animal, etc.; if the comparison result shows that there is no image of the moving object in the first thermal imaging image, it can be determined that the road condition in the area in front of the target vehicle at the target time is relatively safe and the vehicle can be driven normally.

[0038] Furthermore, when it is determined that there is an image of a moving object in the first thermal imaging, the obstacle between the moving object and the target vehicle can be determined as the target reference object. The proportional relationship between the image of the moving object and the image of the target reference object can be calculated based on the first thermal imaging image. At the same time, the actual size information of the target reference object can be obtained through the environmental image. Then, the distance, angle and other position information between the moving object and the target reference object can be determined by combining the information of the first thermal imaging image and the environmental image. By determining the positional relationship between the two, it can help predict the movement trajectory of the moving object.

[0039] Step S103, generating a three-dimensional point cloud image of the target area at the target time through radar laser, and determining the coordinates of the moving object in the three-dimensional point cloud image according to the position relationship to obtain the target coordinates.

[0040] Specifically, since thermal imaging sensors and cameras only obtain the approximate position information range of the moving object, the specific coordinates of the moving object can be determined through the point cloud. First, the lidar can be used as a sensing device to obtain three-dimensional point cloud data by emitting a laser beam and measuring its reflection time, and then a three-dimensional point cloud image of the area in front of the vehicle at the target time can be generated.

[0041] Furthermore, a coordinate system, such as a world coordinate system, can be established based on the three-dimensional point cloud image. First, a certain position of the target vehicle, target reference object and mobile object in the three-dimensional point cloud image can be selected to represent their coordinate points in the coordinate system. For example, the lowest center point on the left side of the target vehicle, target reference object and mobile object image can be selected to establish the coordinate point. First, the coordinate information of the target reference object can be determined in the world coordinate system. Since the position information such as the distance and angle of the target reference object relative to the target vehicle can be obtained by radar laser, the coordinate information of the target vehicle can be determined. Then, based on the position information between the mobile object and the target reference object determined by combining the information of the first thermal imaging image and the environmental image, the coordinate information of the mobile object in the coordinate system can be determined to obtain the target coordinates of the mobile object in the three-dimensional point cloud image.

[0042] Step S104, obtaining coordinate information of the mobile object at the previous acquisition time of the target time, obtaining historical coordinates, and generating a first predicted movement trajectory of the mobile object in the three-dimensional point cloud image according to the historical coordinates and the target coordinates.

[0043] Specifically, to predict the movement trajectory of a moving object, the coordinate information of the moving object in the world coordinate system at another moment can be obtained first. The other moment can be the moment of the last captured image before the target moment. Then, the coordinate information of the moving object at two different moments, the acquired historical coordinates and the aforementioned target coordinates, can be used to calculate the moving distance and moving time of the moving object. Then, the moving speed of the moving object can be calculated, and the coordinate information of the moving object at each moment of the captured image after the target moment can be predicted based on the moving speed of the moving object. By connecting all the predicted coordinate points, a moving trajectory can be formed, which is the first predicted moving trajectory of the moving object in the three-dimensional point cloud image.

[0044] For example, the coordinates of the moving object at the target time t0 are P0 (x0, y0, z0), and the coordinates at the previous acquisition time t1 are P1 (x1, y1, z1). Then the moving distance ΔP and moving time ΔT of the moving object can be calculated as follows:

[0045] ΔP=√((x1-x0) 2 +(y1-y0) 2 +(z1-z0) 2 );

[0046] ΔT = t0 - t1;

[0047] The moving speed V of the moving object can be calculated based on the moving distance ΔP and the moving time ΔT:

[0048] V = ΔP / ΔT;

[0049] The next acquisition time t can be calculated by calculating the speed V N The moving distance is calculated, and then the coordinates of the moving object at that moment are obtained. Finally, the first predicted moving trajectory of the moving object in the three-dimensional point cloud image is formed through each coordinate, so as to help determine whether the target vehicle has the risk of colliding with the moving object.

[0050] Step S105 , determining a second predicted movement trajectory of the target vehicle in the three-dimensional point cloud image, and determining a collision risk between the target vehicle and the moving object according to the first predicted movement trajectory and the second predicted movement trajectory.

[0051] Specifically, similar to the method for determining the first predicted movement trajectory of the moving object, the second predicted movement trajectory of the target vehicle in the three-dimensional point cloud image can be determined by obtaining the coordinate information of the target vehicle in the world coordinate system at the last acquisition moment of the target moment, and then calculating the moving distance, moving time and moving speed of the target vehicle based on the coordinate information of the target vehicle at two different moments, and generating the second predicted movement trajectory of the target vehicle in the three-dimensional point cloud image by predicting the coordinate information of the target vehicle at each acquisition moment after the target moment.

[0052] Furthermore, the coordinates of the overlapping point of the two trajectories can be determined through the first predicted moving trajectory and the second predicted moving trajectory, and then the moments when the moving object and the target vehicle reach the overlapping point coordinates can be obtained respectively according to their respective moving speeds. By calculating the interval between the two moments, it is determined whether the time difference is within a safe range, and then it can be determined whether there is a risk of collision between the target vehicle and the moving object.

[0053] The risk warning method provided by the embodiment of the present application obtains a first thermal imaging image of the target area at the target moment, and obtains an environmental image of the target area at the target moment through an image acquisition device, wherein the target area is located in front of a target vehicle, and the environmental image is an image of the environment within the target area; determines whether there is an image of a moving object in the first thermal imaging image according to the environmental image, and when there is an image of the moving object in the first thermal imaging image, determines the positional relationship between the moving object and a target reference object according to the first thermal imaging image and the environmental image, wherein the target reference object is located in the environmental image, and the target reference object is located between the target vehicle and the mobile object; generates a three-dimensional point cloud image of the target area at the target moment through a radar laser, and determines the coordinates of the moving object in the three-dimensional point cloud image according to the positional relationship, and obtains the target reference object. coordinates; obtain the coordinate information of the moving object at the last acquisition time of the target time, obtain the historical coordinates, and generate the first predicted moving trajectory of the moving object in the three-dimensional point cloud image according to the historical coordinates and the target coordinates; determine the second predicted moving trajectory of the target vehicle in the three-dimensional point cloud image, and determine the collision risk between the target vehicle and the moving object according to the first predicted moving trajectory and the second predicted moving trajectory, and determine the positional relationship between the moving object and the target reference object by combining the thermal imaging image and the environmental image, and then predict the moving trajectory and the coordinate information of the coincident point of the target vehicle and the moving object in combination with the target coordinates of the moving object obtained according to the three-dimensional point cloud image, so as to determine the collision risk between the target vehicle and the moving object, and solve the problem of low accuracy of risk warning through a single device such as a camera in the related technology. And then achieve the technical effect of improving the accuracy of risk warning and reducing the risk of traffic accidents.

[0054] Optionally, in the risk warning method provided in an embodiment of the present application, judging whether there is an image of a moving object in the first thermal imaging image based on the environmental image includes: judging whether there is an abnormal area in the image area of ​​the first thermal imaging image whose temperature value is greater than a preset temperature value; if there is an abnormal area, generating a second thermal imaging image corresponding to the environmental image based on the image content contained in the environmental image; comparing the first thermal imaging image and the second thermal imaging image, and determining that there is an image of a moving object in the first thermal imaging image if the first thermal imaging image and the second thermal imaging image are inconsistent.

[0055] It should be noted that thermal imaging images display the shape and characteristics of objects by capturing the heat emitted by them. Objects of different temperatures appear at different temperatures and brightnesses in thermal imaging images. The heat emitted by pedestrians is higher than the heat emitted by most parts of the vehicle. Therefore, it is possible to observe whether there are abnormal areas in the first thermal imaging image with a temperature value greater than a preset temperature value. For example, if there is an abnormal area in the first thermal imaging image with a temperature value greater than a preset temperature value of 30°C, it can be determined that there are suspected moving objects such as pedestrians in the area.

[0056] Furthermore, since other non-pedestrians and other moving objects in the environment may also have a higher temperature, such as vehicle engines, etc., they may interfere with the judgment result obtained based on the first thermal imaging image, resulting in an erroneous judgment that there are pedestrians in front. Therefore, when there is an abnormal area in the first thermal imaging image area, the camera can be used to obtain the environmental image of the area in front of the target vehicle, and the image content therein can be used to generate a second thermal imaging image. By comparing the first thermal imaging image and the second thermal imaging image to see if there is inconsistency in the abnormal area, it is determined whether there is an image of a moving object. If the image of a part of the abnormal area of ​​the first thermal imaging image does not exist in the second thermal imaging image, it can be determined that there is an image of a moving object in the first thermal imaging image, thereby reducing the probability of misidentification of the moving object.

[0057] This embodiment compares the first thermal imaging image in which the abnormal area exists with the second thermal imaging image generated using the environmental image, thereby accurately determining whether there is an image of a moving object in the first thermal imaging image, thereby ensuring the accuracy of the risk warning.

[0058] Optionally, in the risk warning method provided in an embodiment of the present application, determining the positional relationship between the moving object and the target reference object based on the first thermal imaging image and the environmental image includes: obtaining size information of the target reference object from the environmental image; determining the proportional relationship between the target reference object and the moving object based on the first thermal imaging image; calculating the distance and angle between the target reference object and the moving object based on the proportional relationship and the size information, and determining the distance and angle as the positional relationship.

[0059] It should be noted that when there is a moving object in the first thermal imaging image, the environmental image obtained by the camera can only capture the obstacle between the target vehicle and the moving object, that is, the target reference object. The size information of the target reference object can be obtained from the environmental image through a variety of methods. For example, the actual size of the vehicle can be calculated by measuring the number of pixels of the target reference object in the environmental image and combining the pixel storage and shooting distance of the camera.

[0060] Furthermore, the proportional relationship between the target reference object and the moving object can be measured based on the first thermal imaging image. Based on the proportional relationship between the two and the size information of the aforementioned target reference object, the distance and angle between the target reference object and the moving object can be calculated, thereby determining the positional relationship between the target reference object and the moving object.

[0061] This embodiment determines the positional relationship between the target reference object and the moving object by acquiring the size information of the target reference object and the proportional relationship between the target reference object and the moving object, thereby ensuring the accuracy of the coordinate information of the moving object determined according to the positional relationship.

[0062] Optionally, in the risk warning method provided in the embodiment of the present application, the coordinates of the moving object in the three-dimensional point cloud image are determined according to the positional relationship, and obtaining the target coordinates includes: determining the coordinate information of the target reference object in the three-dimensional point cloud image to obtain reference coordinate information; calculating the coordinate information of the moving object according to the distance, angle and reference coordinate information between the target reference object and the mobile object to obtain the target coordinates.

[0063] It should be noted that in order to more accurately predict the moving trajectory of the mobile object and the target vehicle, a laser radar can be used as a sensing device to generate a three-dimensional point cloud image of the area in front of the target vehicle at the target moment. Then, a world coordinate system can be established based on the three-dimensional point cloud image, and the coordinate information of the target vehicle, the target reference object and the moving object can be determined in the coordinate system. The lowest center point on the left side of the target vehicle, the target reference object and the moving object image can be selected as the coordinate point. First, the coordinate information of the target vehicle is determined. Then, based on the positional relationship such as the distance and angle between the target reference object and the target vehicle obtained by the laser radar, the coordinate information of the target reference object can be determined. Then, based on the distance and angle between the target reference object and the moving object and the coordinate information of the target reference object, the coordinate information of the moving object can be determined by calculation, and the target coordinates of the moving object in the three-dimensional point cloud image can be obtained.

[0064] For example, assuming that the coordinates of the target vehicle a are (4, 3), the distance between the target vehicle a and the target reference object b is 5√2 meters, and the angle is 45°, after calculation, the coordinates of the target reference object b can be obtained as (9, 8). When the distance between the target reference object b and the mobile object c is 2 meters and the angle is 30°, the coordinates of the mobile object are calculated to be (9+√3, 9). Therefore, the target coordinates can be determined, and then the first predicted moving trajectory of the mobile object can be predicted through the target coordinates.

[0065] This embodiment establishes a coordinate system through a three-dimensional point cloud image to determine the coordinate information of the moving object and the target vehicle, thereby ensuring the accuracy of predicting the moving trajectory of the moving object and the target vehicle.

[0066] Optionally, in the risk warning method provided in the embodiment of the present application, generating a first predicted movement trajectory of the moving object in the three-dimensional point cloud image based on the historical coordinates and the target coordinates includes: obtaining a first time difference between the previous acquisition moment of the target moment and the target moment; calculating the distance difference between the historical coordinates and the target coordinates; calculating a first moving speed of the moving object based on the first time difference and the distance difference; calculating the coordinate information of the moving object at each acquisition moment after the target moment based on the first moving speed and the first time difference, and obtaining multiple first predicted coordinates; and generating a first predicted movement trajectory based on the multiple first predicted coordinates.

[0067] It should be noted that the predicted moving trajectory of a moving object can be obtained by calculating the coordinate information of the moving object at two different moments. First, another moment different from the target moment is selected, for example, it can be the moment of the last captured image of the target moment, and then the first time difference between the target moment and the last captured moment is calculated. At the same time, through coordinate calculation, the distance difference between the historical coordinates of the moving object at the last captured moment and the target coordinates is obtained. The first moving speed of the moving object can be obtained by dividing the distance difference and the first time difference. Then, the coordinates of the moving object at each captured moment after the target moment can be predicted by calculation to obtain multiple first predicted coordinates. The first predicted moving trajectory of the moving object can be obtained by connecting each first predicted coordinate point.

[0068] For example, assuming that the coordinates of the moving object at the target time 9:30 are (15, 18), the acquisition device acquires an image every second, and obtains the coordinates of the moving object one second ago (14, 19) as the historical coordinates. It can be calculated that the distance difference of the target vehicle at the two times is √2 meters, and the first time difference is 1 second. Therefore, the first moving speed of the target vehicle can be obtained as √2m / s. According to the first moving speed, the coordinate information of the target vehicle at each acquisition time after the target time 9:30 can be calculated, and multiple first predicted coordinates can be obtained. The second predicted moving trajectory can be obtained by connecting each first predicted coordinate point.

[0069] This embodiment obtains a first predicted moving trajectory of the moving object through coordinate calculation, so that the coordinates of the coincidence point of the moving object and the target vehicle can be predicted according to the moving trajectory, thereby ensuring the accuracy of predicting the collision risk between the target vehicle and the moving object.

[0070] Optionally, in the risk warning method provided in an embodiment of the present application, determining the second predicted movement trajectory of the target vehicle in the three-dimensional point cloud image includes: obtaining a second moving speed of the target vehicle, and calculating the coordinate information of the target vehicle at each collection time after the target time based on the second moving speed and the first time difference, to obtain multiple second predicted coordinates; and generating a second predicted movement trajectory based on the multiple second predicted coordinates.

[0071] It should be noted that after obtaining the first predicted moving trajectory of the moving object, the same method can be used to generate the second moving trajectory of the target vehicle. The coordinate information of the target vehicle at the target time and the previous collection time can be obtained, and the moving speed of the target vehicle can be calculated by calculating the time difference between the two times and the distance difference between the two coordinates to obtain the second moving speed of the target vehicle. The second predicted coordinates of multiple target vehicles can be obtained by multiplying the second moving speed with the time difference between each collection time after the target time and the target time. The second predicted moving trajectory of the target vehicle can be obtained by connecting each second predicted coordinate point.

[0072] For example, assuming that the coordinates of the target vehicle at the target time 9:30 are (10, 10), and the previous collection time at the target time is one second ago, at which the coordinates of the target vehicle are (5, 8), it can be calculated that the distance difference of the target vehicle at the two times is √29 meters, and the time difference is 1 second. Therefore, the second moving speed of the target vehicle can be obtained as √29 m / s. Based on the second moving speed, the coordinate information of the target vehicle at each collection time after the target time 9:30 can be calculated, and multiple second predicted coordinates can be obtained. Connecting each second predicted coordinate point can generate a second predicted moving trajectory.

[0073] This embodiment obtains the second predicted moving trajectory of the target vehicle through coordinate calculation, so that the coordinate information of the coincidence point of the moving object and the target vehicle can be determined in combination with the first moving trajectory of the moving object, thereby ensuring the accuracy of predicting the collision risk between the target vehicle and the moving object.

[0074] Optionally, in the risk warning method provided in the embodiment of the present application, determining the collision risk between the target vehicle and the moving object according to the first predicted moving trajectory and the second predicted moving trajectory includes: determining the coordinates of the overlapping point of the first predicted moving trajectory and the second predicted moving trajectory; obtaining the arrival time corresponding to the overlapping point coordinates in the first predicted moving trajectory, and obtaining the first arrival time; obtaining the arrival time corresponding to the overlapping point coordinates in the second predicted moving trajectory, and obtaining the second arrival time; judging whether the second time difference between the first arrival time and the second arrival time is greater than the preset time difference; when the second time difference is greater than the preset time difference, determining that there is no collision risk between the target vehicle and the moving object; when the second time difference is less than or equal to the preset time difference, determining that there is a collision risk between the target vehicle and the moving object, and displaying a warning message on the display device of the target vehicle, wherein the warning message is used to prompt the driver that there is a moving object in the target area.

[0075] It should be noted that after obtaining the first predicted moving trajectory and the second predicted moving trajectory, the coordinates of their overlapping points in the three-dimensional point cloud image can be determined based on the two trajectories. According to the coordinates of the overlapping points and the coordinates of the target vehicle and the moving object at the target time, the distances between the target vehicle and the moving object and the coordinates of the overlapping points can be calculated, and then the respective distances divided by the moving speeds can be calculated. Finally, the times when the target vehicle and the moving object arrive at the coordinates of the overlapping points can be calculated, that is, the first arrival time and the second arrival time.

[0076] Furthermore, after obtaining the first arrival time and the second arrival time, the time difference between the first arrival time and the second arrival time can be calculated, that is, the second time difference. When there is a safe time range for the target vehicle and the moving object to arrive at the overlapping point, the safe time range is determined as the preset time difference. For example, the preset time difference can be 1 minute, which means that when the event of the target vehicle and the moving object arriving at the overlapping point is greater than or equal to one minute, it can be judged that there is no risk of collision between the two.

[0077] Furthermore, when the time difference between the target vehicle and the moving object reaching the coordinates of the overlapping point is less than the preset time difference, it is judged that there is a risk of collision between the two, and therefore a warning needs to be issued to the driver to avoid a safety accident. The warning information can be displayed on the display device of the target vehicle to remind the driver that there is a moving object in the area in front of the target vehicle, and the driver can then take measures such as slowing down to reduce the risk of collision with the moving object.

[0078] This implementation case determines the coordinates of the overlapping point of the first moving trajectory and the second moving trajectory, and then calculates the time difference between the moving object and the target vehicle reaching the overlapping point coordinates, thereby accurately predicting the risk of collision between the two and ensuring safety through warning information.

[0079] Figure 2 is a flow chart of an optional risk warning method provided according to an embodiment of the present application, such as Figure 2 As shown, the process includes the following steps:

[0080] After acquiring the image of the area in front of the vehicle at the target moment through the acquisition device, an initial detection can be performed first. By identifying and detecting the thermal imaging image, it is determined whether the thermal imaging image contains an area with abnormal temperature based on the detection result. The area with abnormal temperature can be regarded as a moving object suspected to be a pedestrian or animal. If the detection result shows that there is no abnormal area, the next frame of thermal imaging image can be detected; if the detection result indicates that there is an abnormal area, the thermal imaging image can be subjected to 2D perception fusion detection.

[0081] In the process of 2D perception fusion detection, by fusing and comparing the thermal imaging image with the 2D environmental image taken by the camera, it is possible to determine whether the temperature abnormality area detected by the initial detection step is the engine of an obstacle vehicle (target reference object) or other interfering object information. After determining that there are moving objects such as pedestrians in the abnormal area, the distance, angle and other position information between the moving object and the target reference object can be determined based on the thermal imaging image and the 2D environmental image, and then this information can be input into the point cloud perception fusion detection module for image data fusion.

[0082] In the process of point cloud perception fusion detection, the laser radar is used as a perception device to capture 3D environmental information, and a three-dimensional point cloud image of the area in front of the target vehicle at the target time can be generated. By fusing it with the position information output by the 2D perception fusion detection, the position data of the object can be corrected, and the coordinate information corresponding to the target vehicle, target reference object and mobile object can be generated in the three-dimensional point cloud image. The moving trajectory of the mobile object and the target vehicle is predicted through coordinate calculation, and the coordinates of the coincidence point between the two are determined. Then, by calculating the time difference between the mobile object and the target vehicle reaching the coincidence point coordinates, the risk of collision between the mobile object and the target vehicle is determined.

[0083] The coordinate information and risk prediction results are input into the planning module. When the prediction results show that there is a risk of collision between the moving object and the target vehicle, the planning module will use risk avoidance methods such as deceleration, detour or parking based on the coordinate information, thereby reducing the risk of the vehicle colliding with moving objects in the blind spot during operation and ensuring the safe driving of the vehicle.

[0084] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0085] The embodiment of the present application also provides a risk warning device. It should be noted that the risk warning device of the embodiment of the present application can be used to execute the risk warning method provided in the embodiment of the present application. The risk warning device provided in the embodiment of the present application is introduced below.

[0086] Figure 3 Schematic diagram of a risk warning device provided according to an embodiment of the present application. Figure 3 As shown, the device comprises:

[0087] The acquisition unit 31 is used to acquire a first thermal imaging image of the target area at the target moment, and to acquire an environmental image of the target area at the target moment through an image acquisition device, wherein the target area is located in front of the target vehicle, and the environmental image is an image of the environment within the target area.

[0088] The judgment unit 32 is used to judge whether there is an image of a moving object in the first thermal imaging image according to the environmental image, and when there is an image of a moving object in the first thermal imaging image, determine the positional relationship between the moving object and the target reference object according to the first thermal imaging image and the environmental image, wherein the target reference object is located in the environmental image and is located between the target vehicle and the mobile object.

[0089] The first determination unit 33 is used to generate a three-dimensional point cloud image of the target area at the target time through radar laser, and determine the coordinates of the moving object in the three-dimensional point cloud image according to the position relationship to obtain the target coordinates.

[0090] The second determining unit 34 is used to obtain the coordinate information of the mobile object at the previous acquisition moment of the target moment, obtain the historical coordinates, and generate a first predicted movement trajectory of the mobile object in the three-dimensional point cloud image according to the historical coordinates and the target coordinates.

[0091] The third determining unit 35 is used to determine a second predicted moving trajectory of the target vehicle in the three-dimensional point cloud image, and determine the collision risk between the target vehicle and the moving object according to the first predicted moving trajectory and the second predicted moving trajectory.

[0092] The risk warning device provided by the embodiment of the present application acquires a first thermal imaging image of a target area at a target time through an acquisition unit 31, and acquires an environmental image of the target area at the target time through an image acquisition device, wherein the target area is located in front of a target vehicle, and the environmental image is an image of the environment in the target area; the judgment unit 32 judges whether there is an image of a moving object in the first thermal imaging image according to the environmental image, and when there is an image of a moving object in the first thermal imaging image, determines the positional relationship between the moving object and the target reference object according to the first thermal imaging image and the environmental image, wherein the target reference object is located in the environmental image, and the target reference object is located between the target vehicle and the moving object; the first determination unit 33 generates a three-dimensional point cloud image of the target area at the target time through a radar laser, and determines the coordinates of the moving object in the three-dimensional point cloud image according to the positional relationship to obtain the target coordinates; the second determination unit 34 acquires the coordinate information of the moving object at the previous acquisition time of the target time, obtains the historical coordinates, and generates a first predicted moving trajectory of the moving object in the three-dimensional point cloud image according to the historical coordinates and the target coordinates; the third determination unit 35 determines the second predicted moving trajectory of the target vehicle in the three-dimensional point cloud image, and determines the collision risk between the target vehicle and the moving object according to the first predicted moving trajectory and the second predicted moving trajectory. The positional relationship between the moving object and the target reference object is determined by combining the thermal imaging image and the environmental image, and the moving trajectory and the coordinate information of the coincidence point of the target vehicle and the moving object are predicted by combining the target coordinates of the moving object obtained from the three-dimensional point cloud image, thereby determining the collision risk between the target vehicle and the moving object, solving the problem of low accuracy of risk warning through a single device such as a camera in related technologies. This has achieved the technical effect of improving the accuracy of risk warning and reducing the risk of traffic accidents.

[0093] Optionally, in the risk warning device provided in the embodiment of the present application, the judgment unit 32 includes: a first judgment module, used to judge whether there is an abnormal area in the image area of ​​the first thermal imaging image whose temperature value is greater than a preset temperature value; a first generation module, used to generate a second thermal imaging image corresponding to the environmental image according to the image content contained in the environmental image when an abnormal area exists; a first determination module, used to compare the first thermal imaging image and the second thermal imaging image, and when the first thermal imaging image and the second thermal imaging image are inconsistent, determine that there is an image of a moving object in the first thermal imaging image.

[0094] Optionally, in the risk warning device provided in the embodiment of the present application, the judgment unit 32 includes: a first acquisition module, used to acquire size information of the target reference object from the environmental image; a second determination module, used to determine the proportional relationship between the target reference object and the moving object based on the first thermal imaging image; a first calculation module, used to calculate the distance and angle between the target reference object and the moving object based on the proportional relationship and size information, and determine the distance and angle as a positional relationship.

[0095] Optionally, in the risk warning device provided in the embodiment of the present application, the first determination unit 33 includes: a third determination module, used to determine the coordinate information of the target reference object in the three-dimensional point cloud image to obtain reference coordinate information; a second calculation module, used to calculate the coordinate information of the moving object based on the distance, angle and reference coordinate information between the target reference object and the moving object to obtain the target coordinates.

[0096] Optionally, in the risk warning device provided in the embodiment of the present application, the second determination unit 34 includes: a second acquisition module, used to acquire a first time difference between the previous collection time of the target time and the target time; a third calculation module, used to calculate the distance difference between the historical coordinates and the target coordinates; a fourth calculation module, used to calculate the first moving speed of the moving object according to the first time difference and the distance difference; a fifth calculation module, used to calculate the coordinate information of the moving object at each collection time after the target time according to the first moving speed and the first time difference, and obtain multiple first predicted coordinates; a second generation module, used to generate a first predicted movement trajectory based on multiple first predicted coordinates.

[0097] Optionally, in the risk warning device provided in the embodiment of the present application, the third determination unit 35 includes: a sixth calculation module, used to obtain the second moving speed of the target vehicle, and calculate the coordinate information of the target vehicle at each collection time after the target time according to the second moving speed and the first time difference, to obtain multiple second predicted coordinates; a third generation module, used to generate a second predicted movement trajectory based on the multiple second predicted coordinates.

[0098] Optionally, in the risk warning device provided in the embodiment of the present application, the third determination unit 35 includes: a fourth determination module, used to determine the coordinates of the overlapping point of the first predicted moving trajectory and the second predicted moving trajectory; a third acquisition module, used to obtain the arrival time corresponding to the overlapping point coordinates in the first predicted moving trajectory, and obtain the first arrival time; a fourth acquisition module, used to obtain the arrival time corresponding to the overlapping point coordinates in the second predicted moving trajectory, and obtain the second arrival time; a second judgment module, used to judge whether the second time difference between the first arrival time and the second arrival time is greater than the preset time difference; the fourth determination module, used to determine that there is no collision risk between the target vehicle and the moving object when the second time difference is greater than the preset time difference; a fifth determination module, used to determine that there is a collision risk between the target vehicle and the moving object when the second time difference is less than or equal to the preset time difference, and display a warning message on the display device of the target vehicle, wherein the warning message is used to prompt the driver that there is a moving object in the target area.

[0099] The risk warning device provided in the embodiment of the present application determines the positional relationship between the moving object and the target reference object by combining the thermal imaging image and the environmental image, and then predicts the moving trajectory and the coordinate information of the coincidence point of the target vehicle and the moving object by combining the target coordinates of the moving object obtained from the three-dimensional point cloud image, thereby determining the collision risk between the target vehicle and the moving object, solving the problem of low accuracy of risk warning by a single device such as a camera in the related technology. Thus, the technical effect of improving the accuracy of risk warning and reducing the risk of traffic accidents is achieved.

[0100] The above-mentioned risk warning device includes a processor and a memory. The above-mentioned acquisition unit 31, judgment unit 32, first determination unit 33, second determination unit 34, third determination unit 35, etc. are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize corresponding functions.

[0101] The processor includes a kernel, which retrieves the corresponding program unit from the memory. One or more kernels can be set, and the kernel parameters can be adjusted to achieve the technical effect of improving the accuracy of risk warning and reducing the risk of traffic accidents.

[0102] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0103] An embodiment of the present invention provides a computer-readable storage medium on which a program is stored. When the program is executed by a processor, the risk warning method is implemented.

[0104] An embodiment of the present invention provides a processor, which is used to run a program, wherein the risk warning method is executed when the program is running.

[0105] Figure 4 is a schematic diagram of an electronic device provided according to an embodiment of the present application, such as Figure 4 As shown, an embodiment of the present invention provides an electronic device, the electronic device 40 includes a processor, a memory, and a program stored in the memory and executable on the processor, and the processor implements the following steps when executing the program: acquiring a first thermal imaging image of a target area at a target moment, and acquiring an environmental image of the target area at the target moment through an image acquisition device, wherein the target area is located in front of a target vehicle, and the environmental image is an image of the environment within the target area; judging whether there is an image of a moving object in the first thermal imaging image according to the environmental image, and if there is an image of a moving object in the first thermal imaging image, determining the distance between the moving object and the target reference object according to the first thermal imaging image and the environmental image; The positional relationship between the target reference object and the moving object, wherein the target reference object is located in the environment image, and the target reference object is located between the target vehicle and the moving object; a three-dimensional point cloud image of the target area at the target time is generated by radar laser, and the coordinates of the moving object in the three-dimensional point cloud image are determined according to the positional relationship to obtain the target coordinates; the coordinate information of the moving object at the last acquisition time of the target time is obtained to obtain the historical coordinates, and the first predicted moving trajectory of the moving object in the three-dimensional point cloud image is generated according to the historical coordinates and the target coordinates; the second predicted moving trajectory of the target vehicle in the three-dimensional point cloud image is determined, and the collision risk between the target vehicle and the moving object is determined according to the first predicted moving trajectory and the second predicted moving trajectory. The device in this article can be a server, PC, PAD, mobile phone, etc.

[0106] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program that is initialized with the following method steps: acquiring a first thermal imaging image of a target area at a target moment, and acquiring an environmental image of the target area at the target moment through an image acquisition device, wherein the target area is located in front of a target vehicle, and the environmental image is an image of the environment within the target area; judging whether there is an image of a moving object in the first thermal imaging image based on the environmental image, and determining a positional relationship between the moving object and a target reference object based on the first thermal imaging image and the environmental image when there is an image of the moving object in the first thermal imaging image, wherein the target reference object is located in the environmental image, and the target reference object is located between the target vehicle and the moving object; generating a three-dimensional point cloud image of the target area at the target moment through a radar laser, and determining the coordinates of the moving object in the three-dimensional point cloud image based on the positional relationship to obtain the target coordinates; acquiring the coordinate information of the moving object at the last acquisition moment of the target moment to obtain the historical coordinates, and generating a first predicted moving trajectory of the moving object in the three-dimensional point cloud image based on the historical coordinates and the target coordinates; determining a second predicted moving trajectory of the target vehicle in the three-dimensional point cloud image, and determining the collision risk between the target vehicle and the moving object based on the first predicted moving trajectory and the second predicted moving trajectory.

[0107] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0108] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0109] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0110] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0111] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0112] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0113] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0114] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0115] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A risk early warning method, characterized in that: include: Acquire a first thermal imaging image of a target area at a target moment, and acquire an environmental image of the target area at the target moment by an image acquisition device, wherein the target area is located in front of a target vehicle, and the environmental image is an image of the environment within the target area; Determining whether there is an image of a moving object in the first thermal imaging image according to the environment image, and if there is an image of the moving object in the first thermal imaging image, determining a positional relationship between the moving object and a target reference object according to the first thermal imaging image and the environment image, wherein the target reference object is located in the environment image and is located between the target vehicle and the moving object; Generate a three-dimensional point cloud image of the target area at the target time by radar laser, and determine the coordinates of the moving object in the three-dimensional point cloud image according to the positional relationship to obtain the target coordinates; Acquire coordinate information of the mobile object at a previous acquisition moment of the target moment to obtain historical coordinates, and generate a first predicted movement trajectory of the mobile object in the three-dimensional point cloud image according to the historical coordinates and the target coordinates; A second predicted movement trajectory of the target vehicle in the three-dimensional point cloud image is determined, and a collision risk between the target vehicle and the moving object is determined based on the first predicted movement trajectory and the second predicted movement trajectory.

2. The method according to claim 1, characterized in that Judging whether there is an image of a moving object in the first thermal imaging image according to the environment image includes: Determine whether there is an abnormal area in the image area of ​​the first thermal imaging image whose temperature value is greater than a preset temperature value; In the case where the abnormal area exists, generating a second thermal imaging image corresponding to the environmental image according to the image content contained in the environmental image; The first thermal imaging image is compared with the second thermal imaging image, and when the first thermal imaging image and the second thermal imaging image are inconsistent, it is determined that the image of the moving object exists in the first thermal imaging image.

3. The method according to claim 1, characterized in that Determining the positional relationship between the moving object and the target reference object according to the first thermal imaging image and the environment image includes: Acquiring size information of the target reference object from the environment image; determining a proportional relationship between the target reference object and the moving object according to the first thermal imaging image; The distance and angle between the target reference object and the moving object are calculated according to the proportional relationship and the size information, and the distance and the angle are determined as the positional relationship.

4. The method according to claim 3, characterized in that Determining the coordinates of the moving object in the three-dimensional point cloud image according to the positional relationship, and obtaining the target coordinates includes: Determine the coordinate information of the target reference object in the three-dimensional point cloud image to obtain reference coordinate information; The coordinate information of the moving object is calculated according to the distance, angle and reference coordinate information between the target reference object and the moving object to obtain the target coordinates.

5. The method according to claim 1, characterized in that: Generating a first predicted movement trajectory of the moving object in the three-dimensional point cloud image according to the historical coordinates and the target coordinates includes: Acquire a first time difference between a previous acquisition time of the target time and the target time; Calculating the distance difference between the historical coordinates and the target coordinates; Calculating a first moving speed of the moving object according to the first time difference and the distance difference; Calculating coordinate information of the moving object at each collection time after the target time according to the first moving speed and the first time difference to obtain a plurality of first predicted coordinates; The first predicted movement trajectory is generated according to the multiple first predicted coordinates.

6. The method according to claim 5, characterized in that Determining a second predicted moving trajectory of the target vehicle in the three-dimensional point cloud image includes: Acquire a second moving speed of the target vehicle, and calculate the coordinate information of the target vehicle at each collection time after the target time according to the second moving speed and the first time difference, to obtain a plurality of second predicted coordinates; The second predicted movement trajectory is generated according to the plurality of second predicted coordinates.

7. The method according to claim 1, characterized in that Determining the collision risk between the target vehicle and the moving object according to the first predicted moving trajectory and the second predicted moving trajectory includes: Determining coordinates of a point where the first predicted movement trajectory and the second predicted movement trajectory overlap; Obtaining the arrival time corresponding to the coordinates of the coincident point in the first predicted movement trajectory to obtain a first arrival time; Obtaining the arrival time corresponding to the coordinates of the coincident point in the second predicted movement trajectory to obtain a second arrival time; Determining whether a second time difference between the first arrival time and the second arrival time is greater than a preset time difference; When the second time difference is greater than the preset time difference, determining that there is no collision risk between the target vehicle and the moving object; When the second time difference is less than or equal to the preset time difference, it is determined that there is a collision risk between the target vehicle and the moving object, and a warning message is displayed on the display device of the target vehicle, wherein the warning message is used to prompt the driver that the moving object exists in the target area.

8. A risk warning device, characterized in that: include: an acquisition unit, configured to acquire a first thermal imaging image of a target area at a target moment, and acquire an environmental image of the target area at the target moment through an image acquisition device, wherein the target area is located in front of a target vehicle, and the environmental image is an image of the environment within the target area; a judging unit, configured to judge whether there is an image of a moving object in the first thermal imaging image according to the environment image, and, if there is an image of the moving object in the first thermal imaging image, determine a positional relationship between the moving object and a target reference object according to the first thermal imaging image and the environment image, wherein the target reference object is located in the environment image and is located between the target vehicle and the moving object; A first determining unit is used to generate a three-dimensional point cloud image of the target area at the target time by radar laser, and determine the coordinates of the moving object in the three-dimensional point cloud image according to the position relationship to obtain the target coordinates; a second determining unit, configured to obtain coordinate information of the mobile object at a previous acquisition moment of the target moment, obtain historical coordinates, and generate a first predicted movement trajectory of the mobile object in the three-dimensional point cloud image according to the historical coordinates and the target coordinates; The third determining unit is used to determine a second predicted moving trajectory of the target vehicle in the three-dimensional point cloud image, and determine the collision risk between the target vehicle and the moving object according to the first predicted moving trajectory and the second predicted moving trajectory.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored executable program, wherein when the executable program is running, the device where the computer-readable storage medium is located is controlled to execute the risk warning method described in any one of claims 1 to 7.

10. An electronic device, characterized in that: include: A memory storing an executable program; A processor is used to run the program, wherein the risk warning method described in any one of claims 1 to 7 is executed when the program is run.

Images