A large vehicle walking anti-collision method based on video and double models

Abstract

The application belongs to the technical field of large vehicle safety operation, and specifically discloses a large vehicle walking anti-collision method based on video and double models, which comprises the following steps: acquiring image data of a target vehicle driving direction through an image acquisition device; based on the image data, obtaining an obstacle detection frame in the image through a detection model and obtaining ground segmentation data in the image through a segmentation model; in the case where the obstacle detection frame intersects with a preset protection area in the image, eliminating an obstacle ground part in the obstacle detection frame based on the ground segmentation data to obtain a remaining part; in the case where the remaining part intersects with the preset protection area in the image, calculating the distance between the obstacle and the image acquisition device; and based on the distance between the obstacle and the image acquisition device, controlling the target vehicle to avoid collision with the obstacle. The method can improve the anti-collision detection accuracy.

Description

Technical Field

[0001] This application belongs to the field of large vehicle safety operation technology, and more specifically, relates to a large vehicle collision avoidance method based on video and dual models. Background Technology

[0002] Currently, existing collision avoidance methods for rubber-tired container gantry cranes and rail-mounted container gantry cranes mainly rely on LiDAR to detect obstacles. However, LiDAR is complex to install, has high maintenance costs, and weak anti-interference capabilities, resulting in a high frequency of false alarms and missed alarms. Some methods use video to collect data and use detection models to detect obstacles in the video, improving anti-interference capabilities. However, because the detection model uses a rectangular frame, the detected obstacle frame includes not only the truck area of ​​the obstacle itself but also some ground data. This can lead to situations where the detected obstacle frame intersects with the protected area but does not actually collide, making this method prone to false alarms and resulting in low accuracy in collision avoidance detection. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this application aims to provide a large vehicle collision avoidance method based on video and dual models. This method addresses the problems of weak anti-interference capability of existing large vehicle collision avoidance methods using lidar and low collision avoidance detection accuracy due to false alarms caused by the use of video and detection models.

[0004] To achieve the above objectives, in a first aspect, this application provides a large vehicle collision avoidance method based on video and dual models, comprising: Image data of the target vehicle's direction of travel is acquired using an image acquisition device; Based on the image data, obstacle detection boxes in the image are obtained through a detection model, and ground segmentation data in the image is obtained through a segmentation model; When the obstacle detection box intersects with a preset protection area in the image, the obstacle ground portion in the obstacle detection box is removed based on the ground segmentation data to obtain the remaining portion; If the remaining portion intersects with a preset protection area in the image, calculate the distance between the obstacle and the image acquisition device; Based on the distance between the obstacle and the image acquisition device, the target vehicle is controlled to avoid collision with the obstacle.

[0005] This application acquires image data through an image acquisition device. While using a detection model to detect obstacles in the image, it also uses a segmentation model to acquire ground segmentation data in the image. When the detected obstacle detection box intersects with the protection area, in order to prevent false alarms, the obstacle target is separated from the ground area by combining the ground segmentation data. If they still intersect, it is determined that the obstacle target has invaded the protection area. At this time, the distance to the obstacle is calculated, and the vehicle collision avoidance is controlled according to the distance. This can fundamentally solve the defect of video collision avoidance schemes with single detection models that are prone to false alarms, improve the accuracy of collision avoidance detection, and at the same time, the installation and maintenance are relatively simple and the cost is low.

[0006] According to the large vehicle collision avoidance method based on video and dual models provided in this application, the calculation of the distance between the obstacle and the image acquisition device includes: Place markers on the lane lines; Based on the correspondence between the coordinates of the landmarks in the image and the actual physical coordinates of the landmarks, a distance estimation objective function is constructed. Based on the coordinates of the obstacle detection box in the image and the distance estimation objective function, the distance between the obstacle and the image acquisition device is calculated.

[0007] This application correlates image pixel coordinates with physical distances by placing markers on real-world lane lines and fitting them, eliminating the need for expensive depth cameras or complex multi-sensor calibration. It can achieve effective obstacle distance estimation using only ordinary cameras, reducing costs and complexity.

[0008] According to the large vehicle collision avoidance method based on video and dual models provided in this application, the method further includes: Lane line data in the image is obtained through the segmentation model; If the target vehicle deviates from its lane, the preset protection zone is adjusted based on the lane deviation angle.

[0009] This application obtains lane line data from images through a segmentation model, and ensures that the protection area is always aligned with the actual driving path of the vehicle by sensing the lane line angle in real time and adjusting the protection area synchronously. This avoids missed detections due to deviation and improves the accuracy of collision avoidance detection.

[0010] According to the large vehicle collision avoidance method based on video and dual models provided in this application, the image acquisition device is installed vertically.

[0011] This application avoids the problem of insufficient field of view and missing detection due to insufficient features caused by conventional horizontal installation when the obstacle target is large, as the part of the obstacle target exceeds the field of view. The vertical installation method can obtain more features of the obstacle target, thereby improving the accuracy of collision avoidance detection.

[0012] Secondly, this application provides a large vehicle collision avoidance method based on video and dual models, including: The acquisition module is used to acquire image data of the target vehicle's driving direction through an image acquisition device; The detection and segmentation module is used to obtain obstacle detection boxes in the image through a detection model and ground segmentation data in the image through a segmentation model based on the image data. The elimination module is used to eliminate the ground portion of the obstacle in the obstacle detection frame based on the ground segmentation data when the obstacle detection frame intersects with the preset protection area in the image, so as to obtain the remaining portion; The calculation module is used to calculate the distance between the obstacle and the image acquisition device when the remaining part intersects with the preset protection area in the image; The control module is used to control the target vehicle to avoid collision with the obstacle based on the distance between the obstacle and the image acquisition device.

[0013] Thirdly, this application provides an electronic device, comprising: at least one memory for storing a program; and at least one processor for executing the program stored in the memory. When the program stored in the memory is executed, the processor is used to execute the video and dual-model-based vehicle collision avoidance method described in the first aspect or any possible implementation of the first aspect.

[0014] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when run on a processor, causes the processor to execute the video and dual-model-based vehicle collision avoidance method described in the first aspect or any possible implementation of the first aspect.

[0015] Fifthly, this application provides a computer program product that, when run on a processor, causes the processor to execute the video and dual-model-based vehicle collision avoidance method described in the first aspect or any possible implementation of the first aspect.

[0016] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here.

[0017] Overall, the technical solutions conceived in this application have the following beneficial effects compared with the prior art: This application acquires image data through an image acquisition device. While using a detection model to detect obstacles in the image, it also uses a segmentation model to acquire ground segmentation data in the image. When the detected obstacle detection box intersects with the protection area, in order to prevent false alarms, the obstacle target is separated from the ground area by combining the ground segmentation data. If they still intersect, it is determined that the obstacle target has invaded the protection area. At this time, the distance to the obstacle is calculated, and the vehicle collision avoidance is controlled according to the distance. This can fundamentally solve the defect of video collision avoidance schemes with single detection models that are prone to false alarms, improve the accuracy of collision avoidance detection, and at the same time, the installation and maintenance are relatively simple and the cost is low. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating the large vehicle collision avoidance method based on video and dual models provided in the embodiments of this application; Figure 2 This is a schematic diagram of the protected area provided in an embodiment of this application; Figure 3 This is a schematic diagram of the placement of markers provided in an embodiment of this application; Figure 4 This is a schematic diagram of the protection area and lane line area after a large vehicle veers off course and after correction, provided in an embodiment of this application; wherein, Figure 4 (a) represents the normal situation where the vehicle does not veer off course. Figure 4 (b) is the case where the large vehicle veers off course. Figure 4 (c) shows the situation after the protection area has been corrected; Figure 5 This is a schematic diagram of image coordinate transformation provided in an embodiment of this application; Figure 5 (a) is the original horizontal image, and (b) in 5 is the vertical image after the field of view has been changed; Figure 6 This is a schematic diagram of the structure of the large vehicle anti-collision device based on video and dual models provided in the embodiments of this application; Figure 7 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0021] In this article, the term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The symbol " / " in this article indicates that the related objects are in an "or" relationship; for example, A / B means A or B.

[0022] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0023] In the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more, for example, multiple processing units means two or more processing units, multiple elements means two or more elements, etc.

[0024] Next, combined Figures 1-5 The collision avoidance method for large vehicles based on video and dual models provided in the embodiments of this application is described.

[0025] Figure 1 This is a flowchart illustrating the large vehicle collision avoidance method based on video and dual models provided in this application embodiment, as shown below. Figure 1 As shown, the method includes the following steps: Step S1: Acquire image data of the target vehicle's driving direction using an image acquisition device; Optionally, the image acquisition device may be a camera, webcam, etc., and this application does not limit it.

[0026] Optionally, a camera can be installed at each of the gate legs of the bridge. However, considering the convenience of camera maintenance and adjustment in the future, and the possibility that distant obstacles may appear too small if the camera is installed too high, the camera should not be installed too high.

[0027] Video or image data of the target vehicle's direction of travel is acquired using an image acquisition device.

[0028] Optionally, after acquiring the image data, the image's length and width are set to W and H, respectively. The image's length and width are adjusted to be consistent, resulting in an image with dimensions W1 and H1. Then, the image is compressed to the model's input dimensions W2 and H2, with a compression coefficient of α. The image is then padded and compressed, with a padded length of P, handled in two ways: When W is greater than or equal to H, P = WH; W1 = W, H1 = P + H; W2 = α * W1, H2 = α * H1.

[0029] When W is less than H, P = HW; W1 = W + P, H1 = H; W2 = α * W1, H2 = α * H1.

[0030] Step S2: Based on the image data, obtain obstacle detection boxes in the image through a detection model, and obtain ground segmentation data in the image through a segmentation model; Optionally, the detection model and segmentation model are first built and trained. Image data collected on-site from the container can be used as training data. The detection model is built and trained using the YOLOV8 framework until training is complete.

[0031] Optionally, image data collected on-site from the container can be used as training data to build and train a segmentation model using the YOLOV8 framework until training is complete.

[0032] Optionally, the dataset can be divided into a training set, a validation set, and a test set. The model can be trained on the training set to obtain a trained detection model and a segmentation model.

[0033] Image data is input into a trained detection model to detect obstacles within the field of view, resulting in a detection box for the suspected obstacle, which is the outer contour of the pixels it occupies on the image.

[0034] Image data is input into a trained segmentation model to obtain ground segmentation data in the image, including ground area data excluding objects such as buildings, containers, and mobile machinery.

[0035] Step S3: When the obstacle detection box intersects with the preset protection area in the image, remove the ground portion of the obstacle in the obstacle detection box based on the ground segmentation data to obtain the remaining portion; Optionally, a protection zone should first be defined. The scope of the protection zone should be referenced to the boundary line of the lane line. The protection zone should include the inner edge of the lane line to ensure that obstacle information can be obtained in a timely manner when an obstacle intrudes into the inner side of the lane line.

[0036] Figure 2 This is a schematic diagram of the protected area provided in an embodiment of this application. In one embodiment of this application, the defined protected area is as follows: Figure 2 As shown.

[0037] When the detection model detects an obstacle target, it first determines whether the obstacle detection box intersects with the protected area. If they intersect, in order to prevent false alarms, it combines ground segmentation data to peel off the ground area in the obstacle detection box and obtain the remaining part.

[0038] Step S4: If the remaining part intersects with the preset protection area in the image, calculate the distance between the obstacle and the image acquisition device; Determine whether the remaining part intersects with the protected area. If they no longer intersect, the suspicious obstacle is determined to be a non-dangerous object. If they still intersect, the obstacle target is determined to have invaded the protected area and is a dangerous object. Calculate the distance between the obstacle and the image acquisition device.

[0039] Step S5: Based on the distance between the obstacle and the image acquisition device, control the target vehicle to avoid collision with the obstacle.

[0040] Optionally, the distance values ​​between the obstacle and the image acquisition device can be sent to the target vehicle. The programmable logic controller (PLC) of the target vehicle can then perform operations such as slowing down or stopping the vehicle based on the distance values ​​to prevent the target vehicle from colliding with the obstacle.

[0041] The large vehicle collision avoidance method based on video and dual models provided in this application acquires image data through an image acquisition device. While using a detection model to detect obstacles in the image, a segmentation model is also used to acquire ground segmentation data in the image. When the detected obstacle detection box intersects with the protection area, in order to prevent false alarms, the obstacle target is separated from the ground area by combining the ground segmentation data. If they still intersect, it is determined that the obstacle target has invaded the protection area. At this time, the distance to the obstacle is calculated, and the vehicle collision avoidance is controlled according to the distance. This can fundamentally solve the defect of video collision avoidance schemes with single detection models that are prone to false alarms, improve the accuracy of collision avoidance detection, and at the same time, the installation and maintenance are relatively simple and the cost is low.

[0042] In some embodiments, step S4 specifically includes: Step S41: Place markers on the lane lines; Step S42: Based on the correspondence between the coordinates of the landmarks in the image and their actual physical coordinates, construct a distance estimation objective function; Step S43: Calculate the distance between the obstacle and the image acquisition device based on the coordinates of the obstacle detection box in the image and the distance estimation objective function.

[0043] Figure 3 This is a schematic diagram of the placement of markers provided in the embodiments of this application, such as... Figure 3As shown, two-dimensional image data cannot directly obtain the actual distance between the obstacle target and the camera. Some auxiliary parameters are needed to convert the pixel difference in the image into the actual distance. After the camera equipment is installed, several markers are placed in the lane line direction within the protected area. The actual distance between the markers and the target vehicle is measured. The actual distance of the markers is matched one-to-one with the pixels of the markers in the image. Then, these data are used to fit a distance estimation objective function for calculating the actual distance of the obstacle target.

[0044] Specifically, several markers are placed in the center of the lane lines. The pixel coordinates H of the markers in the image are mapped one-to-one with the actual physical coordinates Y of the markers to form new coordinate points (H, Y). Then, the least squares method is used to solve for the relevant parameters of the quadratic objective function based on these coordinate points. Least squares method for fitting a quadratic function: middle: H: Data matrix (with respect to X), where each row corresponds to an observed data point and each column corresponds to a basis function (or regressor).

[0045]

[0046] Y: Observation vector (output vector), which is a column vector consisting of all observed y values.

[0047]

[0048] θ: Parameter vector (unknowns), i.e., the a, b, c that we want to solve for.

[0049]

[0050] e: Error vector (residual), which is the difference between the model's predicted value and the actual value.

[0051]

[0052] The goal of least squares is to find a set of... This minimizes the sum of squares of the error vector e, where the sum of squares of the errors (i.e., the objective function) It can be written as:

[0053] After unfolding, we get:

[0054] To minimize ,right Find the derivative (matrix derivative) and set it equal to 0:

[0055] Simplifying, we get:

[0056] The final solution is (assuming) reversible):

[0057] After determining that the obstacle is a dangerous object, the pixel coordinates H of the obstacle are input into the distance estimation objective function to calculate the actual distance Y of the obstacle.

[0058] In some embodiments, the method further includes: Lane line data in the image is obtained through a segmentation model; If the target vehicle deviates from its lane, the preset protection zone is adjusted based on the lane deviation angle.

[0059] Figure 4 This is a schematic diagram of the protection area and lane line area after a large vehicle veers off course and after correction, provided in an embodiment of this application. Figure 4 Figure (a) shows the normal situation where the large vehicle does not veer off course; in this case, the protection zone normally covers the lane line area. Figure 4 Figure (b) shows that when a large vehicle veers off course, the pre-drawn protection area no longer overlaps with the area included by the lane line. The protection area may not be able to fully cover the lane line area, resulting in a protection blank area within the lane line range. When an obstacle appears in the protection blank area, the algorithm will determine that the obstacle target is a non-dangerous object, resulting in a missed alarm and a safety accident.

[0060] Therefore, this application inputs image data into a segmentation model to obtain lane line data, determines whether a large vehicle is veering off course, and when a large vehicle veers off course, calculates the lane line offset angle β. Based on the β angle, the protection area is corrected to obtain a new protection area, such as... Figure 4 As shown in Figure (c), the new protection zone will fully cover the offset lane line area, which can effectively reduce the false alarm rate and false alarm rate of obstacles.

[0061] In one embodiment of this application, for a tire-mounted container gantry crane, the protected area is corrected through the following steps: 1a. Acquire image data and input the image into the lane line segmentation model; 2a. The model infers and predicts the lane line area. Based on the pixel coordinates of the lane line area, the backbone coordinates of the lane line are extracted, and then RANSAC is used to fit the straight line. 3a. The angle between the fitted line and the initial line is β; 4a. For example Figure 4As shown in Figure (c), the initial protection area is finely adjusted according to this angle to ensure that the position of the protection area does not shift in world coordinates.

[0062] In some embodiments, the image acquisition device is installed vertically.

[0063] Traditional camera installation methods typically use horizontal mounting, which results in a smaller vertical field of view. Often, two cameras are needed to meet mission requirements. This application takes into account the narrow and long characteristics of lane lines and adopts vertical mounting, enabling the camera's field of view to cover remote obstacle targets.

[0064] Optionally, if a vertical installation method is used, the images acquired by the image acquisition device need to undergo coordinate transformation.

[0065] Figure 5 This is a schematic diagram of image coordinate transformation provided in an embodiment of this application, such as... Figure 5 As shown, in one embodiment of this application, the image is rotated 90 degrees. Figure 5 Figure (a) is the original horizontal image. Let the coordinates of a pixel in the original horizontal image be (x, y). The new coordinates (x′, y′) after rotating 90 degrees can be calculated using the following formula: x′ = -y + W - 1; y′=x; Where W is the width of the original image, the above operation is performed on each pixel in the image to obtain... Figure 5 Figure (b) shows the longitudinal image after the field of view has been changed.

[0066] The following describes the large vehicle collision avoidance device based on video and dual models provided in this application. The large vehicle collision avoidance device based on video and dual models described below can be referred to in correspondence with the large vehicle collision avoidance method based on video and dual models described above.

[0067] Figure 6 This is a structural schematic diagram of a large vehicle anti-collision device based on video and dual models provided in an embodiment of this application, as shown below. Figure 6 As shown, the device 600 includes: The acquisition module 610 is used to acquire image data of the target vehicle's driving direction through the image acquisition device; The detection and segmentation module 620 is used to obtain obstacle detection boxes in the image based on image data through a detection model and to obtain ground segmentation data in the image through a segmentation model. The elimination module 630 is used to eliminate the ground portion of the obstacle in the obstacle detection box based on ground segmentation data when the obstacle detection box intersects with the preset protection area in the image, so as to obtain the remaining part; The calculation module 640 is used to calculate the distance between the obstacle and the image acquisition device when the remaining part intersects with the preset protection area in the image; The control module 650 is used to control the target vehicle to avoid collisions with obstacles based on the distance between the obstacle and the image acquisition device.

[0068] It should be understood that the above-described device is used to execute the methods in the above embodiments. The implementation principle and technical effect of the corresponding program modules in the device are similar to those described in the above methods. The working process of the device can be referred to the corresponding process in the above methods, and will not be repeated here.

[0069] Based on the methods in the above embodiments, Figure 7 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 7 As shown in the illustration, this application provides an electronic device that may include a processor 710, a communications interface 720, a memory 730, and a communication bus 740. The processor 710, communications interface 720, and memory 730 communicate with each other via the communication bus 740. The processor 710 can call logical instructions stored in the memory 730 to execute the large vehicle collision avoidance method based on video and dual models described in the above embodiment.

[0070] Furthermore, the logical instructions in the aforementioned memory 730 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the large vehicle collision avoidance method based on video and dual models described in the various embodiments of this application.

[0071] Based on the methods in the above embodiments, this application provides a computer-readable storage medium storing a computer program. When the computer program runs on a processor, it causes the processor to execute the large vehicle collision avoidance method based on video and dual models in the above embodiments.

[0072] Based on the methods in the above embodiments, this application provides a computer program product that, when running on a processor, causes the processor to execute the large vehicle collision avoidance method based on video and dual models in the above embodiments.

[0073] It is understood that the processor in the embodiments of this application can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor can be a microprocessor or any conventional processor.

[0074] The method steps in this application embodiment can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an ASIC.

[0075] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0076] It is understood that the various numerical designations used in the embodiments of this application are merely for the convenience of description and are not intended to limit the scope of the embodiments of this application.

[0077] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for avoiding collisions on large vehicles based on video and dual models, characterized in that, include: Image data of the target vehicle's direction of travel is acquired using an image acquisition device; Based on the image data, obstacle detection boxes in the image are obtained through a detection model, and ground segmentation data in the image is obtained through a segmentation model. When the obstacle detection box intersects with a preset protection area in the image, the obstacle ground portion in the obstacle detection box is removed based on the ground segmentation data to obtain the remaining portion; If the remaining portion intersects with a preset protection area in the image, calculate the distance between the obstacle and the image acquisition device; Based on the distance between the obstacle and the image acquisition device, the target vehicle is controlled to avoid collision with the obstacle.

2. The large vehicle collision avoidance method based on video and dual models according to claim 1, characterized in that, The calculation of the distance between the obstacle and the image acquisition device includes: Place markers on the lane lines; Based on the correspondence between the coordinates of the landmarks in the image and the actual physical coordinates of the landmarks, a distance estimation objective function is constructed. Based on the coordinates of the obstacle detection box in the image and the distance estimation objective function, the distance between the obstacle and the image acquisition device is calculated.

3. The large vehicle collision avoidance method based on video and dual models according to claim 1, characterized in that, The method further includes: Lane line data in the image is obtained through the segmentation model; If the target vehicle deviates from its lane, the preset protection zone is adjusted based on the lane deviation angle.

4. The large vehicle collision avoidance method based on video and dual models according to claim 1, characterized in that, The image acquisition device is installed vertically.

5. A large vehicle collision avoidance device based on video and dual models, characterized in that, include: The acquisition module is used to acquire image data of the target vehicle's driving direction through an image acquisition device; The detection and segmentation module is used to obtain obstacle detection boxes in the image through a detection model and ground segmentation data in the image through a segmentation model based on the image data. The elimination module is used to eliminate the ground portion of the obstacle in the obstacle detection frame based on the ground segmentation data when the obstacle detection frame intersects with the preset protection area in the image, so as to obtain the remaining portion; The calculation module is used to calculate the distance between the obstacle and the image acquisition device when the remaining part intersects with the preset protection area in the image; The control module is used to control the target vehicle to avoid collision with the obstacle based on the distance between the obstacle and the image acquisition device.

6. An electronic device, characterized in that, include: At least one memory for storing computer programs; At least one processor is configured to execute a program stored in the memory, wherein when the program stored in the memory is executed, the processor is configured to execute the large vehicle walking collision avoidance method based on video and dual models as described in any one of claims 1-4.

7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is run on the processor, the processor performs the video and dual-model-based vehicle collision avoidance method as described in any one of claims 1-4.

8. A computer program product, characterized in that, When the computer program product is run on the processor, the processor performs the video and dual-model-based vehicle collision avoidance method as described in any one of claims 1-4.

Images