Vehicle chassis collision warning methods, devices, computer equipment and storage media

Abstract

This application relates to a vehicle chassis collision warning method, device, computer equipment, and storage medium. The method includes: acquiring target images from different virtual perspectives in front of the vehicle using a vehicle-based image acquisition device; inputting the target images from different virtual perspectives into a deep neural network model for inference to obtain target prediction information of the target obstacle in the original target image; generating warning information of the target obstacle at the target position based on the target prediction information and acquired vehicle state information; and performing a warning operation based on the warning information. This application solves the problem in related technologies that detection can only be performed after a collision occurs and lacks predictive capability. It accurately detects the target obstacle's prediction information in advance, enabling early warning operations, reducing the occurrence of collisions or scrapes, and improving driving safety.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automobiles, and in particular to a vehicle chassis collision early warning method and device, a computer device, and a storage medium. BACKGROUND

[0002] With the development of the automobile industry, additional requirements are proposed for automobiles in addition to basic driving functions. Among them, chassis collision early warning requirements are one of the additional requirements. The current implementation scheme is to use sensors installed on the chassis to generate electrical signals through collisions, send them to the battery management domain control, and then provide early warning information to the driver. However, this scheme has the disadvantage that detection can only be achieved after a collision occurs and does not have predictive capabilities.

[0003] To address the issue of only being able to detect after a collision occurs and not having predictive capabilities in related technologies, no effective solutions have been proposed. SUMMARY

[0004] A vehicle chassis collision early warning method, device, computer device, and storage medium are provided in this embodiment to address the issue of only being able to detect after a collision occurs and not having predictive capabilities in related technologies.

[0005] In a first aspect, a vehicle chassis collision early warning method is provided in this embodiment, comprising:

[0006] An image acquisition device on the vehicle acquires target images at different virtual perspectives in front of the vehicle;

[0007] The target images at different virtual perspectives are input into a deep neural network model for inference to obtain target prediction information of target obstacles in the original image of the target image;

[0008] According to the target prediction information and the acquired vehicle state information, early warning information of the target obstacles at the target position is generated, and a warning operation is performed based on the early warning information.

[0009] In some embodiments, the image acquisition device on the vehicle acquires target images at different virtual perspectives of the vehicle, comprising:

[0010] An image acquisition device on the vehicle acquires an original image in the target image of the vehicle and intrinsic and extrinsic parameters of the image acquisition device;

[0011] According to the intrinsic and extrinsic parameters, the original image is calculated to obtain a depth map and a columnar expansion map in the target image.

[0012] In some embodiments, the inputting the target image under different virtual perspectives into a deep neural network model for inference to obtain target prediction information of a target obstacle in an original image of the target image comprises:

[0013] inputting the original image and the depth map of the target image into a first network model of the deep neural network model for inference to obtain a corresponding feature map; the feature map comprises first prediction information of an obstacle;

[0014] inputting the feature map and a cylindrical unwrapping map of the target image into a second network model of the deep neural network model for inference to obtain second prediction information of the obstacle;

[0015] based on a post-processing module of the deep neural network model, fusing and judging the first prediction information and the second prediction information to obtain the target prediction information of the target obstacle in the original image.

[0016] In some embodiments, the first network model comprises a first convolution module and a multi-layer encoder-decoder; each layer of the encoder-decoder comprises an encoder and a decoder; the first convolution module serves as a top layer; the output of the encoder of the current layer serves as the input of the encoder of the next layer and the decoder of the current layer; the output of the decoder of the current layer serves as the input of the decoder of the next layer.

[0017] In some embodiments, the second network model comprises a second convolution module, a pooling module and a fully connected layer;

[0018] the second convolution module is connected with the pooling module and is configured to perform feature extraction on the feature map and the cylindrical unwrapping map to obtain a feature sequence;

[0019] the pooling module is connected with the fully connected layer and is configured to perform average pooling on the feature sequence to obtain a feature vector;

[0020] the fully connected layer is configured to transform the feature vector to obtain the second prediction information of the obstacle.

[0021] In some embodiments, the fusing and judging the first prediction information and the second prediction information to obtain the target prediction information of the target obstacle in the original image comprises:

[0022] comparing the first prediction information and the second prediction information to determine whether the obstacle is at a local extreme point in the depth map;

[0023] if the obstacle is not at the local extreme point in the depth map, the obstacle is discarded;

[0024] If the obstacle is a local extreme point in the depth map, the obstacle is taken as the target obstacle, and corresponding target prediction information is output.

[0025] In some embodiments, the pre-warning operation based on the pre-warning information includes:

[0026] Based on the pre-warning information, the screen of the vehicle is controlled to display an alarm icon, a prompt, or a warning light.

[0027] Alternatively, based on the pre-warning information, the horn of the vehicle is controlled to play prompt information.

[0028] Alternatively, based on the pre-warning information, the suspension height of the vehicle is increased.

[0029] In a second aspect, a vehicle chassis bump pre-warning device is provided in the present embodiment, which includes an acquisition module, an inference module, and a pre-warning module.

[0030] The acquisition module is configured to acquire target images under different virtual perspectives in front of the vehicle based on an image acquisition device of the vehicle.

[0031] The inference module is configured to input the target images under different virtual perspectives into a deep neural network model for inference to obtain target prediction information of a target obstacle in an original image of the target images.

[0032] The pre-warning module is configured to generate pre-warning information of the target obstacle at a target position according to the target prediction information and acquired vehicle state information, and perform a pre-warning operation based on the pre-warning information.

[0033] In a third aspect, a computer device is provided in the present embodiment, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the vehicle chassis bump pre-warning method of the first aspect when executing the computer program.

[0034] In a fourth aspect, a storage medium is provided in the present embodiment, which stores a computer program executable by a processor to implement the vehicle chassis bump pre-warning method of the first aspect.

[0035] Compared with related technologies, the vehicle chassis collision warning method, device, computer equipment, and storage medium provided in this embodiment acquire target images from different virtual perspectives in front of the vehicle through a vehicle-based image acquisition device; input the target images from different virtual perspectives into a deep neural network model for inference to obtain target prediction information of the target obstacle in the original image; generate warning information of the target obstacle at the target position based on the target prediction information and the acquired vehicle state information; and perform warning operations based on the warning information. This solves the problem in related technologies that can only detect after a collision and lacks predictive capability, accurately detects the target obstacle's target prediction information in advance, and performs warning operations earlier to reduce the occurrence of collisions or scrapes and improve driving safety.

[0036] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0037] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0038] Figure 1 This is a hardware structure block diagram of a 360° surround view system provided in an embodiment of this application;

[0039] Figure 2 This is a flowchart of a vehicle chassis collision warning method provided in an embodiment of this application;

[0040] Figure 3 This is a flowchart of step S210;

[0041] Figure 4 This is a schematic diagram of the structure of a deep neural network model provided in an embodiment of this application;

[0042] Figure 5 This is a schematic diagram of the structure of a first network model provided in an embodiment of this application;

[0043] Figure 6 This is a schematic diagram of the encoder provided in one embodiment of this application;

[0044] Figure 7 This is a schematic diagram of the structure of the first convolution module provided in an embodiment of this application;

[0045] Figure 8 This is a schematic diagram of the decoder provided in one embodiment of this application;

[0046] Figure 9This is a schematic diagram of the structure of a fully connected conditional random field provided in an embodiment of this application;

[0047] Figure 10 This is a schematic diagram illustrating the computational relationship between small regions provided in an embodiment of this application;

[0048] Figure 11 This is a schematic diagram of the structure of a unary arithmetic unit provided in an embodiment of this application;

[0049] Figure 12 This is a schematic diagram of the network structure of the second network model provided in an embodiment of this application;

[0050] Figure 13 This is a structural block diagram of a vehicle chassis collision warning device provided in one embodiment of this application.

[0051] In the diagram: 10, image acquisition device; 20, domain controller; 30, alarm; 210, acquisition module; 220, inference module; 230, early warning module. Detailed Implementation

[0052] To better understand the purpose, technical solution, and advantages of this application, the application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0053] Unless otherwise defined, the technical or scientific terms used in this application shall have the general meaning as understood by one of ordinary skill in the art to which this application pertains. Words such as “a,” “an,” “an,” “the,” “the,” and “these,” used in this application, do not indicate quantitative limitation and may be singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include steps or modules (units) not listed, or may include other steps or modules (units) inherent to such processes, methods, products, or devices. The terms “connected,” “linked,” and “coupled,” used in this application, are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. The term “multiple” used in this application refers to two or more. The "and / or" operator describes the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: A alone, A and B simultaneously, and B alone. Typically, the character " / " indicates that the objects before and after it are in an "or" relationship. The terms "first," "second," and "third," etc., used in this application are merely for distinguishing similar objects and do not represent a specific ordering of the objects.

[0054] The method embodiments provided in this example can be executed in a domain controller 20, terminal, computer, or similar computing device within a 360° surround view system. For example... Figure 1 As shown, the 360° surround view system may include an image acquisition device 10, a domain controller 20, and an alarm 30 connected in sequence. The image acquisition device 10 is used to acquire target images from different virtual perspectives in front of the vehicle. The domain controller 20 may be an autonomous driving domain controller or a cockpit domain controller, etc., used to run various method embodiments; the alarm 30 may be a central control screen, horn, warning light, vibrator in the seat, etc., used to perform specific alarm operations.

[0055] For example, the method embodiment operates on a domain controller 20, which may include one or more processors and a memory for storing data. The processors may include, but are not limited to, processing devices such as microprocessors (MCUs) or programmable logic devices (FPGAs). The domain controller 20 may also include transmission devices for communication functions and input / output devices. Those skilled in the art will understand that the domain controller 20 may also include more or fewer components.

[0056] The memory can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the vehicle chassis collision warning method in this embodiment. The processor executes various functional applications and data processing by running the computer program stored in the memory, thereby implementing the above-described method. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the domain controller 20 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0057] The transmission device is used to receive or send data via a network. This network includes a wireless network provided by the communication provider of the domain controller 20. In one example, the transmission device includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0058] This embodiment provides a vehicle chassis collision warning method. Figure 2 This is a flowchart of the vehicle chassis collision warning method in this embodiment, as follows:Figure 2 As shown, the process includes the following steps:

[0059] Step S210: Based on the vehicle image acquisition device, acquire target images from different virtual perspectives in front of the vehicle;

[0060] Step S220: Input the target images from different virtual perspectives into the deep neural network model for inference to obtain the target prediction information of the target obstacles in the original image of the target image;

[0061] Step S230: Based on the target prediction information and the acquired vehicle status information, generate early warning information about the target obstacle at the target location; and perform early warning operations based on the early warning information.

[0062] Specifically, image acquisition devices include, but are not limited to, cameras and video cameras. Installed on a vehicle, they can capture target images from different virtual perspectives in front of the vehicle; or, capture the original image in front of the vehicle; and then process the original image to obtain target images from different virtual perspectives. These different virtual perspectives can be the original perspective (generally, a frontal view is used as the original perspective due to the installation of the image acquisition device), a top-down view, and a columnar view. For example, three cameras with corresponding perspectives can be set up to acquire images corresponding to the original perspective, the top-down view, and the columnar view, respectively. Alternatively, a camera with the original perspective can be set up to acquire the original image of the original perspective, and then images from the other two perspectives can be obtained based on the processing of the original image. There are no restrictions on the specific methods used to acquire target images from different virtual perspectives.

[0063] The deep neural network model is pre-built and fully trained. It can employ supervised learning neural networks, unsupervised learning neural networks, or reinforcement learning neural networks, without any restrictions. The target images from different virtual perspectives are then input into the deep neural network model for inference. By fully utilizing relevant feature information from the target images under different virtual perspectives (such as the positional information of the depth map and the height information of the histogram), accurate target prediction information of the obstacles in the original image is obtained. Target prediction information includes, but is not limited to, height information and positional information.

[0064] Vehicle status information includes, but is not limited to, steering wheel angle, chassis height, and driving speed, which can be provided by various sensors, autonomous driving domain controllers, or cockpit domain controllers within the vehicle. Based on the target prediction information and vehicle status information, it can be calculated whether a target obstacle will collide with the vehicle on the current driving path, generating corresponding warning information; the point on the current driving path where a collision is likely is the target location. Furthermore, based on this warning information, early warning actions can be taken, thereby reducing the occurrence of collisions or scrapes and improving driving safety.

[0065] The current solution involves using sensors mounted on the chassis to generate electrical signals upon collision, which are then sent to the battery management domain controller and subsequently used to provide a warning to the driver. However, this solution has the drawback of only detecting collisions after they occur, lacking predictive capabilities. Furthermore, with the development of autonomous driving, more and more sensors (cameras, millimeter-wave radar, ultrasonic radar, lidar, etc.) are being installed around the vehicle. However, millimeter-wave radar and lidar detect distant targets, while millimeter-wave radar detects close-range targets, and these radars have a high false detection rate; camera images, on the other hand, provide a wealth of information within a reasonable range. In the more prevalent 360-degree surround view system, ultra-wide-angle cameras (fisheye cameras) are included. Due to their extremely large field of view, they undertake more perception tasks during parking and low-speed driving. However, fisheye cameras capture two-dimensional images, and how to use the two-dimensional information in the images to estimate the depth and height of low vertical obstacles on the road in front of the vehicle is a significant technical challenge. This application utilizes a vehicle-based image acquisition device to acquire target images from different virtual perspectives in front of the vehicle; inputs these target images from different virtual perspectives into a deep neural network model for inference to obtain target prediction information of the target obstacle in the original target image; generates warning information of the target obstacle at the target position based on the target prediction information and the acquired vehicle state information; and performs warning operations based on the warning information. This solves the problem in related technologies that can only detect obstacles after a collision and lack predictive capabilities, accurately detecting target prediction information of the target obstacle in advance to perform warning operations earlier, reduce the occurrence of collisions or scrapes, and improve driving safety.

[0066] The steps described above are explained in detail below:

[0067] In some of these embodiments, such as Figure 3 As shown, step S210, based on the image acquisition device on the vehicle, acquires target images from different virtual perspectives of the vehicle, including the following steps:

[0068] Step S211: Based on the image acquisition device on the vehicle, acquire the original image in the target image of the vehicle and the intrinsic and extrinsic parameters of the image acquisition device;

[0069] Step S212: Calculate the original image based on the intrinsic and extrinsic parameters to obtain the depth map and histogram unfolded in the target image.

[0070] Specifically, extrinsic parameters describe the position and orientation of the image acquisition device in the world coordinate system, that is, they determine the transformation relationship between the image acquisition device's coordinate system and the world coordinate system. These include, but are not limited to, translation and rotation matrices. Intrinsic parameters describe the internal imaging characteristics of the image acquisition device, reflecting the inherent properties of the lens and the geometric relationship between the imaging plane and the lens. These include, but are not limited to, focal length, principal point coordinates, and distortion parameters.

[0071] The original image refers to the image captured by the image acquisition device from a fixed position on the vehicle. Then, by calculating the viewpoint transformation of the original image based on intrinsic and extrinsic parameters, the depth map and histogram unfolded in the target image can be obtained. The depth map can be an image with depth (position) information, such as a top view or right view; the histogram unfolded can be an unfolded image with height information, such as a left view, right view, or front view. Furthermore, to simplify the computational workload of subsequent processing, the resolution of the original image, depth map, and histogram unfolded can be unified, with a resolution of 512 or higher. For example, a resolution of 512×512×3, etc. In other embodiments, it can also be implemented using a well-trained model, a parallel LUT scheme, or by calling third-party components; there are no limitations on this.

[0072] In this embodiment, by using an image acquisition device and combining intrinsic and extrinsic parameters, depth maps and bar charts can be obtained quickly and accurately, reducing the loss of effective data.

[0073] In some embodiments, step S220, which involves inputting target images from different virtual perspectives into a deep neural network model for inference to obtain target prediction information of the target obstacle in the original image, includes the following steps:

[0074] Step S221: Input the original image and depth map of the target image into the first network model of the deep neural network model for inference to obtain the corresponding feature map; the feature map includes the first prediction information of the obstacle;

[0075] Step S222: Input the feature map and the histogram of the target image into the second network model of the deep neural network model for reasoning to obtain the second prediction information of the obstacle;

[0076] Step S223: The post-processing module based on the deep neural network model performs a fusion judgment based on the first prediction information and the second prediction information to obtain the target prediction information of the target obstacle in the original image.

[0077] Specifically, the target image includes the original image, the depth map, and the histogram unfolded image. For example... Figure 4As shown, the deep neural network model consists of three parts: the first network model (PosNet), the second network model (HeightNet), and the post-processing module. It can be understood that these three parts are pre-built and fully trained.

[0078] The first network model is connected to the second network module and the post-processing module to perform inference on the input original image and depth map to obtain corresponding feature maps. That is, the original image and depth map each have feature maps generated after inference; each feature map includes the first prediction information of obstacles, that is, the feature map that identifies the location of obstacles with depth anomalies in the original image and depth map; thus knowing whether there are obstacles in the original image and where these obstacles are located.

[0079] The second network module is connected to the post-processing module and is used to infer the input feature map and histogram unfolded map to obtain the second predicted information of the obstacle. The second predicted information includes the second position information (coordinates), position confidence ppos, height h, and height confidence ph.

[0080] The post-processing module is used to perform fusion judgment based on the first and second prediction information, compare the positional relationship of obstacles from different perspectives, the relevant confidence level, etc., so as to remove interference and obtain accurate target prediction information of the target obstacles in the original image.

[0081] In this embodiment, target prediction information of the target obstacle can be obtained quickly and accurately, and interference from non-target obstacles can be effectively eliminated.

[0082] The following is a detailed explanation of deep neural network models:

[0083] In some embodiments, the first network model includes a first convolutional module and a multi-layer encoder-decoder; each layer encoder-decoder includes an encoder and a decoder; the first convolutional module serves as the top layer; the output of the encoder of this layer serves as the input of the encoder of the next layer and the decoder of this layer; the output of the decoder of this layer serves as the input of the decoder of the next layer.

[0084] Specifically, such as Figure 5 As shown, the first network model has a U-shaped structure, including a top-level first convolutional module (TopConv) and multiple layers of encoders and decoders; each layer of encoders and decoders includes an encoder and a decoder; the output of the encoder of this layer serves as the input of the encoder of the next layer and the decoder of this layer; the output of the decoder of this layer serves as the input of the decoder of the next layer.

[0085] Among them, such as Figure 6As shown, each layer of the encoder has the same structure and employs a multi-head self-attention mechanism to acquire feature maps at different resolutions. For example, if the input image resolution is 512×512×6 (including a depth map of 512×512×3 and the original image of 512×512×3, so the third dimension is 6), after passing through the first layer encoder, the resolution is reduced to 128×128×N1. The output of this layer encoder is used as the input of the next layer encoder. After passing through 4 layers of encoders, the first convolutional module at the top layer is reached.

[0086] Among them, such as Figure 7 As shown, the first convolutional module primarily extracts high-level semantic information. The input feature map, after being downscaled, passes through several layers of convolutional modules (conv, 3×3 convolutions) and residual connections of an average pooling module (AvgPool). Then, a concatenation module combines all the extracted semantic feature maps together to obtain a 16×16×N feature map. top The feature map sequence is then input into the decoder connected to it.

[0087] Among them, such as Figure 8 As shown and Figure 9 As shown, the decoder in this layer accepts the output of the decoder in the previous layer and the output of the corresponding encoder as input. Each layer's decoder includes windowed fully connected conditional random fields (window FC-CRFs), a rearrangement module, and an upampling block. The fully connected conditional random fields divide the feature map into several small regions (called windows), and then calculate the energy E within each window. The energy E can be seen as the contribution to the final position (depth) information, and the formula for calculating energy E is as follows:

[0088] E(x) = ∑ i ψ u ( i ) + ∑ ij ψ p (x i , j (1);

[0089] In Formula 1, x i x represents the predicted value for each point (i) on the feature map; j ψ represents the predicted value for each point (j) on the feature map; u This represents the univariate potential function, which is predicted based on the point values ​​on each feature map.

[0090] ψ p The calculation formula is as follows:

[0091] ψ p= μ(x i , j f(x) i , j )g(I i , j )k(p i , j (2);

[0092] In Formula 2, μ represents the difference between different feature points: μ = 1 if they are not the same feature point, and μ = 0 if they are the same feature point; i The color of point i on the feature map is represented by I. j The color of point j on the feature map is represented by p. i p represents the first position information of point i on the feature map. j This represents the first position information of point j on the feature map; color and position are used to guide the appropriate results; f is a binary potential function. g and k are parameters.

[0093] In this embodiment, the entire feature map is divided into h×ω small regions (windows). This division into multiple windows is primarily to reduce computational complexity. The total potential complexity estimate for each iteration is as follows:

[0094] Ω(Window FC) = hω*Ω(ψ) u ) + hω(N 2 -1)* Ω(ψ p (3);

[0095] In Equation 3, N is the size of the window, and Ω(ψu) and Ω(ψp) are the complexities of calculating the univariate and bivariate potential functions, respectively. Furthermore, although dividing the feature map into multiple windows theoretically severs the computation of connections between feature points in different windows, it is necessary to shuffle the window divisions within the feature map; therefore, a rearrangement module is introduced.

[0096] like Figure 10 As shown, the left figure shows the calculation relationship between points (red) and other points (blue) within each window, and the right figure is a schematic diagram of the reorganized module window.

[0097] ψ u (x i )= μ (I,x i ), (4);

[0098] like Figure 11As shown, a unaryNet (unary operational unit) is used to calculate the potential energy of a point on the feature map. In Equation 4, θ is the parameter of the unaryNet, which means a small neural network is used to predict the potential energy at point i. For binary potential energy calculation ψ... p (x i,j The main purpose is to calculate the connection between two points and fuse color and position information. Specifically, a transformer mechanism can be used to handle the potential energy calculation between two points. The process is as follows: First, a query vector q and a key value vector K are calculated from the feature map of each window. The q and k of all windows form a Q matrix and a K matrix. The dot product of Q and K is calculated to obtain the prediction parameter matrix W. Then, X (including input information such as feature values, color, and position) is multiplied by the W matrix. Here, a relative embedding position encoding module P is introduced to provide the relative position of each point in the window. Since the relative positions between points are explicitly provided, the calculation of the binary potential energy is as follows: Finally, the optimized X is obtained through an Optimize net. ′ As input to the next layer.

[0099] This embodiment enables the processing of the input image to obtain a feature map that identifies the location of obstacles with depth anomalies in the original image and the depth image, i.e., the first prediction information containing the obstacles.

[0100] In some of these embodiments, such as Figure 12 As shown, the second network model includes a second convolutional module, a pooling module, and a fully connected layer;

[0101] The second convolutional module, connected to the pooling module, is used to extract features from the feature map and the histogram unfolded map to obtain the feature sequence.

[0102] The pooling module, connected to the fully connected layer, is used to perform average pooling on the feature sequence to obtain the feature vector;

[0103] The fully connected layer is used to transform the feature vector to obtain the second prediction information of the obstacle.

[0104] Specifically, the second network model is mainly used to detect the height of obstacles. The second convolutional module can use a ResNet18 convolutional network to extract features from the feature map and histogram unfolded map, obtaining a feature sequence with reduced resolution. Then, through an average pooling module, the feature sequence is averaged to obtain a 512*1*1 dimensional feature vector. Finally, after passing through a fully connected layer (FC(10,5)), the feature vector is transformed to obtain a 10*5 vector; that is, the second prediction information of five obstacles is obtained. This second prediction information includes the second position information (coordinates), position confidence ppos, height h, and height confidence ph.

[0105] This embodiment enables accurate detection of obstacle height, reducing the complexity of detection.

[0106] In some embodiments, step S323, which involves fusing the first and second prediction information to obtain the target prediction information of the target obstacle in the original image, includes the following steps:

[0107] The first and second prediction information are compared to determine whether the obstacle is located at a local extremum point in the depth map.

[0108] If an obstacle is not located at a local extremum in the depth map, the obstacle will be discarded.

[0109] If an obstacle is located at a local extremum point in the depth map, the obstacle will be treated as a target obstacle, and the corresponding target prediction information will be output.

[0110] In this embodiment, the results of the first network model and the second network model are integrated. First, the second prediction information of the second network model is used to verify whether the obstacle in the first prediction information of the first network model is located at a local extremum point (i.e., at the point of maximum potential energy) in the depth map. If the obstacle is not located at a local extremum point in the depth map, it is discarded. If the obstacle is located at a local extremum point in the depth map, it is taken as the target obstacle, and the height and position corresponding to the target obstacle are output as target prediction information. Then, combined with vehicle status information, warning information of the target obstacle at the target position can be generated; and warning operations can be performed based on the warning information.

[0111] This embodiment can fuse the results of the first network model and the second network model, thereby improving the prediction accuracy of target prediction information and reducing the occurrence of collisions or scrapes.

[0112] In some embodiments, the warning operation based on the warning information in step S230 includes the following steps:

[0113] Based on the warning information, control the vehicle's screen to display alarm icons, prompts, or warning lights;

[0114] Alternatively, based on the warning information, control the vehicle's horn to play a prompt message;

[0115] Alternatively, based on the warning information, the vehicle's suspension height can be increased.

[0116] In this embodiment, various warning operations are provided by combining various devices on the vehicle (control screen, horn, warning lights, vibrators in the seats, etc.) to further reduce the occurrence of collisions or scrapes. For example: based on the warning information, the vehicle's screen is controlled to display an alarm icon, prompt message, or warning light. Another example: based on the warning information, the vehicle's horn is controlled to play a warning message. Yet another example: based on the warning information, the vehicle's suspension height is increased, etc.

[0117] It should be noted that the steps shown in the above process or in the flowchart of the accompanying figures can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0118] This embodiment also provides a vehicle chassis collision warning device, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. The terms "module," "unit," "subunit," etc., used below refer to combinations of software and / or hardware that achieve a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0119] Figure 13 This is a structural block diagram of the vehicle chassis collision warning device in this embodiment, as shown below. Figure 13 As shown, the device includes: an acquisition module 210, an inference module 220, and an early warning module 230;

[0120] The acquisition module 210 is used for a vehicle-based image acquisition device to acquire target images from different virtual perspectives in front of the vehicle.

[0121] The inference module 220 is used to input target images from different virtual perspectives into a deep neural network model for inference, and obtain target prediction information of target obstacles in the original image of the target image;

[0122] The early warning module 230 is used to generate early warning information about the target obstacle at the target location based on the target prediction information and the acquired vehicle status information; and to perform early warning operations based on the early warning information.

[0123] The aforementioned device solves the problem in related technologies that detection can only be performed after a collision occurs and lacks predictive capability. It accurately detects target obstacle information in advance, enabling early warning operations, reducing the occurrence of collisions or scrapes, and improving driving safety.

[0124] In some embodiments, the acquisition module 210 is further configured to acquire the original image in the target image of the vehicle and the intrinsic and extrinsic parameters of the image acquisition device based on the image acquisition device on the vehicle.

[0125] Based on intrinsic and extrinsic parameters, the original image is calculated to obtain the depth map and histogram of the target image.

[0126] In some embodiments, the inference module 220 is further configured to input the original image and depth image of the target image into the first network model of the deep neural network model for inference to obtain the corresponding feature map; the feature map includes the first prediction information of the obstacle;

[0127] The feature map and the target image's histogram are input into the second network model of the deep neural network model for inference to obtain the second prediction information of the obstacle;

[0128] The post-processing module based on the deep neural network model fuses the first and second prediction information to obtain the target prediction information of the target obstacle in the original image.

[0129] In some embodiments, the first network model includes a first convolutional module and a multi-layer encoder-decoder; each layer encoder-decoder includes an encoder and a decoder; the first convolutional module serves as the top layer; the output of the encoder of this layer serves as the input of the encoder of the next layer and the decoder of this layer; the output of the decoder of this layer serves as the input of the decoder of the next layer.

[0130] In some of these embodiments, the second network model includes a second convolutional module, a pooling module, and a fully connected layer;

[0131] The second convolutional module, connected to the pooling module, is used to extract features from the feature map and the histogram unfolded map to obtain the feature sequence.

[0132] The pooling module, connected to the fully connected layer, is used to perform average pooling on the feature sequence to obtain the feature vector;

[0133] The fully connected layer is used to transform the feature vector to obtain the second prediction information of the obstacle.

[0134] In some embodiments, the inference module 220 is further configured to compare the first prediction information and the second prediction information to determine whether the obstacle is located at a local extremum point in the depth map.

[0135] If an obstacle is not located at a local extremum in the depth map, the obstacle will be discarded.

[0136] If an obstacle is located at a local extremum point in the depth map, the obstacle will be treated as a target obstacle, and the corresponding target prediction information will be output.

[0137] In some embodiments, the warning module 230 is also used to control the vehicle's screen to display alarm icons, prompts, or warning lights based on the warning information;

[0138] Alternatively, based on the warning information, control the vehicle's horn to play a prompt message;

[0139] Alternatively, based on the warning information, the vehicle's suspension height can be increased.

[0140] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can reside in the same processor; or the above modules can be located in different processors in any combination.

[0141] This embodiment also provides a computer device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0142] Optionally, the computer device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0143] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0144] S1, a vehicle-based image acquisition device, acquires target images from different virtual perspectives in front of the vehicle;

[0145] S2, input the target images from different virtual perspectives into the deep neural network model for inference, and obtain the target prediction information of the target obstacles in the original image of the target image;

[0146] S3 generates warning information about the target obstacle at the target location based on the target prediction information and the acquired vehicle status information; and performs warning operations based on the warning information.

[0147] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated in this embodiment.

[0148] Furthermore, in conjunction with the vehicle chassis collision warning method provided in the above embodiments, this embodiment can also provide a storage medium for implementation. This storage medium stores a computer program; when executed by a processor, the computer program implements any of the vehicle chassis collision warning methods described in the above embodiments.

[0149] It should be noted that all information and data involved in this application are authorized by the user or fully authorized by all parties and will be used legally.

[0150] It should be understood that the specific embodiments described herein are merely illustrative of the application and not intended to limit it. All other embodiments derived by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0151] Obviously, the accompanying drawings are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar situations based on these drawings without any creative effort. Furthermore, it is understood that although the work done in this development process may be complex and lengthy, for those skilled in the art, certain design, manufacturing, or production modifications made based on the technical content disclosed in this application are merely conventional technical means and should not be considered as insufficient disclosure of this application.

[0152] The term "embodiment" in this application refers to a specific feature, structure, or characteristic described in connection with an embodiment that may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily imply the same embodiment, nor does it imply that it is mutually exclusive with or independent of other embodiments. It will be clearly or implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0153] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of patent protection. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the appended claims.

Claims

1. A method of underbody impact warning for a vehicle, characterized by, The method comprises the following steps: An image acquisition device on a vehicle acquires target images of different virtual perspectives in front of the vehicle; The target images of different virtual perspectives are input into a deep neural network model for inference to obtain target prediction information of a target obstacle in an original image of the target images, which comprises: The original image and a depth map of the target images are input into a first network model of the deep neural network model for inference to obtain corresponding feature maps; the feature maps comprise first prediction information of an obstacle; The feature maps and a columnar expansion image of the target images are input into a second network model of the deep neural network model for inference to obtain second prediction information of the obstacle; A post-processing module of the deep neural network model is used to fuse and judge the first prediction information and the second prediction information to obtain the target prediction information of the target obstacle in the original image; According to the target prediction information and acquired vehicle state information, warning information of the target obstacle at a target position is generated, and a warning operation is performed based on the warning information.

2. The vehicle chassis bumping pre-warning method according to claim 1, characterized in that, The image acquisition device on the vehicle acquires target images of different virtual perspectives of the vehicle, which comprises: An image acquisition device on the vehicle acquires an original image in the target images of the vehicle and internal and external parameters of the image acquisition device; According to the internal and external parameters, the original image is calculated to obtain a depth map and a columnar expansion image in the target images.

3. The vehicle chassis bump warning method according to claim 1, characterized by, The first network model comprises a first convolution module and a plurality of layers of encoders and decoders; each layer of the encoders and decoders comprises an encoder and a decoder; the first convolution module serves as a top layer; the output of the encoder of the current layer serves as the input of the encoder of the next layer and the decoder of the current layer; and the output of the decoder of the current layer serves as the input of the decoder of the next layer.

4. The vehicle chassis bump warning method according to claim 1, characterized by, The second network model comprises a second convolution module, a pooling module and a fully connected layer; The second convolution module is connected with the pooling module and is configured to extract features from the feature maps and the columnar expansion image to obtain a feature sequence; The pooling module is connected with the fully connected layer and is configured to perform average pooling on the feature sequence to obtain a feature vector; The fully connected layer is configured to transform the feature vector to obtain the second prediction information of the obstacle.

5. The vehicle chassis bump warning method according to claim 1, characterized by, The fusion judgment based on the first prediction information and the second prediction information to obtain the target prediction information of the target obstacle in the original image comprises: The first prediction information and the second prediction information are compared to determine whether the obstacle is at a local extreme point in the depth map; If the obstacle is not at the local extreme point in the depth map, the obstacle is discarded; If the obstacle is at the local extreme point in the depth map, the obstacle is taken as the target obstacle, and corresponding target prediction information is output.

6. The vehicle chassis bump warning method according to claim 1, characterized by, The warning operation based on the warning information comprises: Based on the warning information, a screen of the vehicle is controlled to display an alarm icon, a prompt or a warning light. Or, based on the early warning information, control the horn of the vehicle to play prompt information; Or, based on the early warning information, increase the suspension height of the vehicle.

7. A vehicle chassis bumping pre-warning device, characterized by, Comprise: An acquisition module, an inference module and an early warning module; The acquisition module is configured to acquire target images under different virtual perspectives in front of the vehicle based on an image acquisition device of the vehicle; The inference module is configured to input the target images under different virtual perspectives into a deep neural network model for inference to obtain target prediction information of a target obstacle in an original image of the target images, which comprises: inputting the original image and a depth map of the target images into a first network model of the deep neural network model for inference to obtain a corresponding feature map; the feature map comprises first prediction information of an obstacle; inputting the feature map and a columnar expansion diagram of the target images into a second network model of the deep neural network model for inference to obtain second prediction information of the obstacle; based on a post-processing module of the deep neural network model, fusing and judging the first prediction information and the second prediction information to obtain the target prediction information of the target obstacle in the original image; The early warning module is configured to generate early warning information of the target obstacle at a target position according to the target prediction information and acquired vehicle state information, and perform an early warning operation based on the early warning information. 8.A computer device, comprising a memory and a processor, and characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to execute the steps of the vehicle chassis collision early warning method of any one of claims 1 to 6.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the vehicle chassis collision early warning method of any one of claims 1 to 6.

Images