Environment sensing method and system based on monocular camera, vehicle and equipment
By extracting, aligning and fusing the images collected by the monocular camera, estimating the 9D information of the target, the problem that monocular cameras are difficult to achieve 3D perception in the autonomous driving system is solved, and low-cost and high-stability environmental perception is achieved Effect.
Patent Information
- Application Number
- CN202411347396.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-24
- Publication Date
- 2025-02-25
AI Technical Summary
In the existing autonomous driving system, the monocular camera mainly senses the 2D information of the target object, or needs to integrate inertial navigation data to achieve 3D perception, resulting in high cost, unstable perception, complex procedures, and inconvenient maintenance.
By acquiring two images collected by a monocular camera, feature extraction is performed separately, the feature maps of the two images are obtained, and the feature maps are aligned and fused to obtain the fusion feature maps, and then the 9D information of the target is estimated, realizing the 3D vehicle's traffic scene Perception.
The low-cost, high-stability environment perception based on monocular cameras is realized, which simplifies the design and maintenance of perception systems and avoids the high cost and complexity of depth cameras or binocular cameras.
Smart Images
Figure CN119516508A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of intelligent driving technology, and in particular to a method, system, vehicle and equipment for environmental perception based on a monocular camera. Background Art
[0002] Perceiving and understanding fast and dynamically changing traffic scenes is very important for modern autonomous driving systems. Autonomous driving systems not only need to detect target objects in the image domain, but also provide three-dimensional (3D) position information of target objects in the world coordinate system to prevent collisions with other traffic objects. At present, 3D perception of traffic scenes is mainly based on depth cameras or binocular cameras, while monocular cameras mainly perceive the two-dimensional (2D) position information of target objects in the image domain, or integrate inertial navigation data to achieve 3D perception of traffic scenes. However, depth cameras and binocular cameras are expensive and the perception results are unstable; monocular cameras mainly perceive the 2D information of target objects in traffic scenes, or integrate inertial navigation data to achieve 3D perception of traffic scenes, but this fusion method will make the fusion program code highly complex and difficult to maintain. Summary of the invention
[0003] Based on this, it is necessary to provide an environmental perception method, system, vehicle and equipment based on a monocular camera to address the above technical problems, which has the advantages of low cost and high perception stability. In addition, it is simple and easy to operate and has low maintenance cost.
[0004] In a first aspect, a method for environmental perception based on a monocular camera is provided, comprising:
[0005] Acquire a first image at a first moment and a second image at a second moment captured by a monocular camera;
[0006] Performing feature extraction on the first image and the second image respectively to obtain a first feature map of the first image and a second feature map of the second image;
[0007] Aligning the first feature map and the second feature map to obtain a projected feature map of the first feature map, and fusing the projected feature map and the second feature map to obtain a fused feature map;
[0008] According to the fused feature map, a rotation matrix, a translation vector, a current drivable area, a 2D detection result of the target, and 3D information of the target are obtained;
[0009] The 9D information of the target is estimated according to the rotation matrix, the translation vector, the current drivable area, the 2D detection result of the target, and the 3D information of the target, wherein the 9D information includes the length, width, height, three-dimensional coordinates, yaw angle, lateral speed, and longitudinal speed of the target.
[0010] In some examples, extracting features from the first image and the second image respectively to obtain a first feature map of the first image and a second feature map of the second image includes:
[0011] The first image and the second image are input into a convolutional neural network to extract a first feature map of the first image and a second feature map of the second image using the convolutional neural network.
[0012] In some examples, the aligning the first feature map and the second feature map to obtain a projected feature map of the first feature map, and fusing the projected feature map and the second feature map to obtain a fused feature map includes:
[0013] Projecting the first feature map onto the second feature map to align with the second feature map;
[0014] After alignment, obtaining a projected feature map obtained by projecting the first feature map onto the second feature map;
[0015] The projection feature map and the second feature map are fused to obtain the fused feature map.
[0016] In some examples, obtaining 9D information estimation of the target according to the rotation matrix, the translation vector, the current drivable area, the 2D detection result of the target, and the 3D information of the target includes:
[0017] obtaining an ego-motion estimate of the vehicle;
[0018] An environmental perception result is obtained based on the vehicle's ego motion estimation, the rotation matrix, the translation vector, the current drivable area, the 2D detection result of the target, and the 3D information of the target, wherein the environmental perception result includes a 9D information estimation of the target.
[0019] In some examples, the environment perception result is obtained according to the vehicle's ego motion estimation, the rotation matrix, the translation vector, the current drivable area, the 2D detection result of the target, and the 3D information of the target, wherein the environment perception result includes a 9D information estimation of the target, including:
[0020] Based on the vehicle's ego motion estimation, the rotation matrix, the translation vector, the current drivable area, the 2D detection result of the target, and the 3D information of the target, the target is tracked through the Kalman filter algorithm to obtain the 9D information of the target.
[0021] In some examples, the first moment and the second moment are acquisition moments of adjacent monocular cameras.
[0022] In a second aspect, an environment perception system based on a monocular camera is provided, comprising:
[0023] An acquisition module is used to acquire a first image captured by a monocular camera at a first moment and a second image captured at a second moment;
[0024] An extraction module, used to perform feature extraction on the first image and the second image respectively to obtain a first feature map of the first image and a second feature map of the second image;
[0025] A fusion module, configured to align the first feature map and the second feature map to obtain a projected feature map of the first feature map, and to fuse the projected feature map and the second feature map to obtain a fused feature map;
[0026] The perception module is used to obtain a rotation matrix, a translation vector, a current drivable area, a 2D detection result of the target, and 3D information of the target according to the fused feature map, and obtain a 9D information estimate of the target according to the rotation matrix, the translation vector, the current drivable area, the 2D detection result of the target, and the 3D information of the target, wherein the 9D information includes the length, width, height, three-dimensional coordinates, yaw angle, lateral speed, and longitudinal speed of the target.
[0027] In a third aspect, a vehicle is provided, comprising: an environment perception system based on a monocular camera according to the second aspect.
[0028] In a fourth aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of the environment perception method based on a monocular camera of the above-mentioned first aspect and any possible implementation method of the first aspect are implemented.
[0029] In a fifth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the steps of the environment perception method based on a monocular camera in the above-mentioned first aspect and any possible implementation method of the first aspect are implemented.
[0030] In a sixth aspect, a computer program product is provided, on which a computer program is stored. When the program is executed by a processor, the steps of the environment perception method based on a monocular camera in the above-mentioned first aspect and any possible implementation method of the first aspect are implemented.
[0031] According to the embodiment of the present application, the first image captured by the monocular camera at the first moment and the second image at the second moment are firstly acquired, and then the image features of the first image and the second image are extracted to obtain the first feature map of the first image and the second feature map of the second image. Then, the first feature map and the second feature map are aligned to obtain the projection feature map of the first feature map, and the projection feature map and the second feature map are fused to obtain the fusion feature map. Finally, according to the fusion feature map, the 9D information estimation of the target is obtained, so as to realize the environmental perception based on the monocular camera. In this way, the vehicle can realize the 3D perception of the traffic scene only by using the monocular camera. Compared with the perception method based on the depth camera or the binocular camera, it has the advantages of low cost and high perception stability. Compared with the method of realizing the 2D perception of the target in the traffic scene by using the monocular camera, or realizing the 3D perception of the traffic scene by fusing the inertial navigation data imu, which leads to the complex fusion program code and inconvenient maintenance, it has the advantages of simple and easy operation and low maintenance cost. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:
[0033] Figure 1 A flow chart of an environment perception method based on a monocular camera provided in an embodiment of the present application;
[0034] Figure 2 A structural block diagram of an environment perception system based on a monocular camera provided in an embodiment of the present application;
[0035] Figure 3 A structural block diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0036] The present application is further described in detail below in conjunction with the embodiments and drawings. It is to be understood that the specific embodiments described herein are only used to explain the relevant application, rather than to limit the application. It is also necessary to explain that, for ease of description, only the parts related to the application are shown in the drawings.
[0037] It should be noted that, in the absence of conflict, the embodiments of the present application, that is, the features of the embodiments, can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.
[0038] The following describes in detail the environment perception method, system, device and medium based on a monocular camera according to the embodiments of the present application in conjunction with the accompanying drawings.
[0039] Figure 1 FIG. 1 is a flow chart of a method for environmental perception based on a monocular camera according to an embodiment of the present application. Figure 1As shown, the environment perception method based on a monocular camera according to an embodiment of the present application includes the following steps:
[0040] S101: Acquire a first image at a first moment and a second image at a second moment captured by a monocular camera.
[0041] In a specific example, the first moment and the second moment may be adjacent monocular camera acquisition moments. With adjacent monocular camera acquisition moments, the monocular camera configured on the vehicle acquires the environment image to obtain two images I at adjacent moments. t ,I t-1 , where I t-1 refers to the first image at the first moment, I t Refers to the second image at the second moment.
[0042] S102: Perform feature extraction on the first image and the second image respectively to obtain a first feature map of the first image and a second feature map of the second image.
[0043] In one embodiment of the present application, feature extraction is performed on the first image and the second image to obtain a first feature map of the first image and a second feature map of the second image, including: inputting the first image and the second image into a convolutional neural network to extract the first feature map of the first image and the second feature map of the second image using the convolutional neural network. For example: using the backbone network (backbone) of the convolutional neural network + feature fusion (PAN) to extract images at two adjacent moments (e.g., two images I t ,I t-1 ) features, and obtain the corresponding feature map F t 、F t-1 , where F t For image I t The feature map, F t-1 For image I t-1 feature map.
[0044] S103: Align the first feature map and the second feature map to obtain a projected feature map of the first feature map, and fuse the projected feature map and the second feature map to obtain a fused feature map.
[0045] In one embodiment of the present application, the first feature map and the second feature map are aligned to obtain a projected feature map of the first feature map, and the projected feature map and the second feature map are fused to obtain a fused feature map, including: projecting the first feature map into the second feature map to align it with the second feature map; after alignment, obtaining a projected feature map of the first feature map projected into the second feature map; and fusing the projected feature map and the second feature map to obtain the fused feature map.
[0046] For example, the feature map F t-1 Projection to F t Next, and then F t Perform the alignment operation, thus obtaining the feature map F′ t-1 , where F′ t-1 Refers to the projection of the first feature map onto the second feature map and the subsequent feature map corresponding to the first feature map, then, F′ t-1 and F t The fusion is performed to obtain a feature map F, where the feature map F refers to a fused feature map.
[0047] S104: According to the fused feature map, a rotation matrix, a translation vector, a current drivable area, a 2D detection result of the target, and 3D information of the target are obtained.
[0048] As a specific example, different detection heads are connected to the fused feature map F. Different detection heads are used to perform detection, thereby outputting different information ((R, t), (free-space), (2Dbox), (3Dbox)). Among them, the moving speed of the target (i.e., the target object) can be estimated by the rotation matrix (R) and the translation vector (t); the current drivable area can be obtained by free-space; the 2D detection result of the target can be obtained by 2Dbox; and the 3D information of the target can be obtained by 3Dbox.
[0049] S105: Obtain 9D information estimation of the target according to the rotation matrix, the translation vector, the current drivable area, the 2D detection result of the target, and the 3D information of the target, wherein the 9D information includes the length, width, height, three-dimensional coordinates, yaw angle, lateral speed, and longitudinal speed of the target.
[0050] In one embodiment of the present application, the 9D information estimation of the target is obtained based on the rotation matrix, the translation vector, the current drivable area, the 2D detection result of the target, and the 3D information of the target, including: obtaining the ego-motion estimate of the vehicle; obtaining the environmental perception result based on the ego-motion estimate of the vehicle, the rotation matrix, the translation vector, the current drivable area, the 2D detection result of the target, and the 3D information of the target, wherein the environmental perception result includes the 9D information estimate of the target.
[0051] In this example, an environmental perception result is obtained based on the vehicle's ego motion estimation, the rotation matrix, the translation vector, the current drivable area, the 2D detection result of the target, and the 3D information of the target, wherein the environmental perception result includes a 9D information estimation of the target, including: based on the vehicle's ego motion estimation, the rotation matrix, the translation vector, the current drivable area, the 2D detection result of the target, and the 3D information of the target, tracking the target is achieved through a Kalman filtering algorithm to obtain the 9D information of the target.
[0052] Specifically, the target is tracked through the Kalman filter algorithm (Kalman) to obtain the target's 9D information (L, W, H, X, Y, Z, Yaw, Vx, Vy), where L, W, H are length, width and height, X, Y, Z refer to the three-dimensional coordinates in the three-dimensional coordinate system, Yaw refers to the yaw angle, Vx refers to the lateral velocity, and Vy refers to the longitudinal velocity.
[0053] According to the environment perception method based on a monocular camera of the embodiment of the present application, firstly, the first image captured by the monocular camera at the first moment and the second image at the second moment are obtained, and then the image features of the first image and the second image are extracted to obtain the first feature map of the first image and the second feature map of the second image, and then the first feature map and the second feature map are aligned to obtain the projection feature map of the first feature map, and the projection feature map and the second feature map are fused to obtain the fusion feature map, and finally, according to the fusion feature map, the 9D information estimation of the target is obtained, so as to realize the environment perception based on the monocular camera, so that the vehicle can realize the 3D perception of the traffic scene only by using the monocular camera. Compared with the perception method based on the depth camera or the binocular camera, it has the advantages of low cost and high perception stability. Compared with the method of realizing the 2D perception of the target in the traffic scene by using the monocular camera, or realizing the 3D perception of the traffic scene by fusing the inertial navigation data imu, which leads to the complex fusion program code and inconvenient maintenance, it has the advantages of simple and easy operation and low maintenance cost.
[0054] Figure 2 is a structural block diagram of an environment perception system based on a monocular camera according to an embodiment of the present application. Figure 2As shown, according to an embodiment of the present application, the environment perception system based on a monocular camera includes: an acquisition module 210, an extraction module 220, a fusion module 230 and a perception module 240, wherein:
[0055] An acquisition module 210 is used to acquire a first image captured by a monocular camera at a first moment and a second image captured at a second moment;
[0056] An extraction module 220, configured to perform feature extraction on the first image and the second image respectively to obtain a first feature map of the first image and a second feature map of the second image;
[0057] A fusion module 230 is used to align the first feature map and the second feature map to obtain a projected feature map of the first feature map, and to fuse the projected feature map and the second feature map to obtain a fused feature map;
[0058] The perception module 240 is used to obtain a rotation matrix, a translation vector, a current drivable area, a 2D detection result of the target, and 3D information of the target based on the fused feature map, and obtain a 9D information estimate of the target based on the rotation matrix, the translation vector, the current drivable area, the 2D detection result of the target, and the 3D information of the target, wherein the 9D information includes the length, width, height, three-dimensional coordinates, yaw angle, lateral speed, and longitudinal speed of the target.
[0059] According to the monocular camera-based environment perception system of the embodiment of the present application, firstly, the first image captured by the monocular camera at the first moment and the second image at the second moment are obtained, and then the image features of the first image and the second image are extracted to obtain the first feature map of the first image and the second feature map of the second image, and then the first feature map and the second feature map are aligned to obtain the projection feature map of the first feature map, and the projection feature map and the second feature map are fused to obtain the fusion feature map, and finally, according to the fusion feature map, the 9D information estimation of the target is obtained, so as to realize the environment perception based on the monocular camera, so that the vehicle can realize the 3D perception of the traffic scene only by using the monocular camera. Compared with the perception method based on the depth camera or the binocular camera, it has the advantages of low cost and high perception stability. Compared with the method of realizing the 2D perception of the target in the traffic scene by using the monocular camera, or realizing the 3D perception of the traffic scene by fusing the inertial navigation data imu, which leads to the complex fusion program code and inconvenient maintenance, it has the advantages of simple and easy operation and low maintenance cost.
[0060] For the specific definition of the monocular camera-based environmental perception system, please refer to the definition of the monocular camera-based environmental perception method mentioned above, which will not be repeated here. The various modules of the above-mentioned monocular camera-based environmental perception system can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.
[0061] In one embodiment, a vehicle is provided, comprising: an environment perception system based on a monocular camera according to any one of the above embodiments. The vehicle first acquires a first image captured by a monocular camera at a first moment and a second image at a second moment, then extracts image features of the first image and the second image, obtains a first feature map of the first image and a second feature map of the second image, then aligns the first feature map and the second feature map, obtains a projection feature map of the first feature map, and fuses the projection feature map and the second feature map to obtain a fusion feature map, and finally, according to the fusion feature map, obtains a 9D information estimate of the target, thereby realizing environment perception based on a monocular camera, so that the vehicle can realize 3D perception of traffic scenes using only a monocular camera. Compared with the perception method based on a depth camera or a binocular camera, it has the advantages of low cost and high perception stability. Compared with the method of realizing 2D perception of targets in traffic scenes by using a monocular camera, or realizing 3D perception of traffic scenes by fusing inertial navigation data imu, which leads to complex fusion program code and inconvenient maintenance, it has the advantages of being simple and easy to implement and having low maintenance cost.
[0062] In addition, other structures and functions of the vehicle according to the embodiment of the present application are known to ordinary technicians in the field and will not be elaborated here.
[0063] In one embodiment, a computer device is provided. Figure 3 This is a block diagram of the computer device provided in the embodiment of the present application, refer to Figure 3 The computer device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, the above-mentioned embodiment of the environment perception method based on a monocular camera is implemented. For example, the following steps are performed: obtaining a first image captured by a monocular camera at a first moment and a second image captured at a second moment;
[0064] Performing feature extraction on the first image and the second image respectively to obtain a first feature map of the first image and a second feature map of the second image;
[0065] Aligning the first feature map and the second feature map to obtain a projected feature map of the first feature map, and fusing the projected feature map and the second feature map to obtain a fused feature map;
[0066] According to the fused feature map, a rotation matrix, a translation vector, a current drivable area, a 2D detection result of the target, and 3D information of the target are obtained;
[0067] The 9D information of the target is estimated according to the rotation matrix, the translation vector, the current drivable area, the 2D detection result of the target, and the 3D information of the target, wherein the 9D information includes the length, width, height, three-dimensional coordinates, yaw angle, lateral speed, and longitudinal speed of the target.
[0068] The embodiment of the present application further provides a computer-readable storage medium, which stores a computer program, and the processor implements the above-mentioned embodiment of the environment perception method based on a monocular camera when executing the computer program. For example, the following steps are executed: obtaining a first image captured by the monocular camera at a first moment and a second image captured at a second moment;
[0069] Performing feature extraction on the first image and the second image respectively to obtain a first feature map of the first image and a second feature map of the second image;
[0070] Aligning the first feature map and the second feature map to obtain a projected feature map of the first feature map, and fusing the projected feature map and the second feature map to obtain a fused feature map;
[0071] According to the fused feature map, a rotation matrix, a translation vector, a current drivable area, a 2D detection result of the target, and 3D information of the target are obtained;
[0072] The 9D information of the target is estimated according to the rotation matrix, the translation vector, the current drivable area, the 2D detection result of the target, and the 3D information of the target, wherein the 9D information includes the length, width, height, three-dimensional coordinates, yaw angle, lateral speed, and longitudinal speed of the target.
[0073] The present application embodiment provides a computer program product, which includes instructions. When the instructions are executed, the method described in the embodiment of the present application is executed. For example, it can be executed Figure 1 The various steps of the monocular camera-based environment perception method shown, for example, are performed: acquiring a first image captured by the monocular camera at a first moment and a second image captured at a second moment;
[0074] Performing feature extraction on the first image and the second image respectively to obtain a first feature map of the first image and a second feature map of the second image;
[0075] Aligning the first feature map and the second feature map to obtain a projected feature map of the first feature map, and fusing the projected feature map and the second feature map to obtain a fused feature map;
[0076] According to the fused feature map, a rotation matrix, a translation vector, a current drivable area, a 2D detection result of the target, and 3D information of the target are obtained;
[0077] The 9D information of the target is estimated according to the rotation matrix, the translation vector, the current drivable area, the 2D detection result of the target, and the 3D information of the target, wherein the 9D information includes the length, width, height, three-dimensional coordinates, yaw angle, lateral speed, and longitudinal speed of the target.
[0078] Those of ordinary skill in the art can understand that all or part of the processes in the methods for implementing the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).
[0079] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0080] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the patent application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent application shall be subject to the attached claims.
Claims
1. A method for environmental perception based on a monocular camera, characterized in that: include: Acquire a first image at a first moment and a second image at a second moment captured by a monocular camera; Performing feature extraction on the first image and the second image respectively to obtain a first feature map of the first image and a second feature map of the second image; Aligning the first feature map and the second feature map to obtain a projected feature map of the first feature map, and fusing the projected feature map and the second feature map to obtain a fused feature map; According to the fused feature map, a rotation matrix, a translation vector, a current drivable area, a 2D detection result of the target, and 3D information of the target are obtained; The 9D information of the target is estimated according to the rotation matrix, the translation vector, the current drivable area, the 2D detection result of the target, and the 3D information of the target, wherein the 9D information includes the length, width, height, three-dimensional coordinates, yaw angle, lateral speed, and longitudinal speed of the target.
2. The method for environmental perception based on a monocular camera according to claim 1, characterized in that: The extracting features from the first image and the second image respectively to obtain a first feature map of the first image and a second feature map of the second image includes: The first image and the second image are input into a convolutional neural network to extract a first feature map of the first image and a second feature map of the second image using the convolutional neural network.
3. The environment perception method based on a monocular camera according to claim 1 or 2, characterized in that: The aligning the first feature map and the second feature map to obtain a projected feature map of the first feature map, and fusing the projected feature map and the second feature map to obtain a fused feature map, includes: Projecting the first feature map onto the second feature map to align with the second feature map; After alignment, obtaining a projected feature map obtained by projecting the first feature map onto the second feature map; The projection feature map and the second feature map are fused to obtain the fused feature map.
4. The method for environmental perception based on a monocular camera according to claim 1, characterized in that: The step of obtaining 9D information estimation of the target according to the rotation matrix, the translation vector, the current drivable area, the 2D detection result of the target, and the 3D information of the target includes: Obtaining an ego-motion estimate of the vehicle; An environmental perception result is obtained based on the vehicle's ego motion estimation, the rotation matrix, the translation vector, the current drivable area, the 2D detection result of the target, and the 3D information of the target, wherein the environmental perception result includes a 9D information estimation of the target.
5. The method for environmental perception based on a monocular camera according to claim 4, characterized in that: The step of obtaining an environmental perception result based on the vehicle's ego motion estimation, the rotation matrix, the translation vector, the current drivable area, the 2D detection result of the target, and the 3D information of the target, wherein the environmental perception result includes a 9D information estimation of the target, includes: Based on the vehicle's ego motion estimation, the rotation matrix, the translation vector, the current drivable area, the 2D detection result of the target, and the 3D information of the target, the target is tracked through the Kalman filter algorithm to obtain the 9D information of the target.
6. The method for environmental perception based on a monocular camera according to claim 1, characterized in that: The first moment and the second moment are acquisition moments of adjacent monocular cameras.
7. An environment perception system based on a monocular camera, characterized in that: include: An acquisition module is used to acquire a first image captured by a monocular camera at a first moment and a second image captured at a second moment; An extraction module, used to perform feature extraction on the first image and the second image respectively to obtain a first feature map of the first image and a second feature map of the second image; A fusion module, configured to align the first feature map and the second feature map to obtain a projected feature map of the first feature map, and to fuse the projected feature map and the second feature map to obtain a fused feature map; The perception module is used to obtain a rotation matrix, a translation vector, a current drivable area, a 2D detection result of the target, and 3D information of the target according to the fused feature map, and obtain a 9D information estimate of the target according to the rotation matrix, the translation vector, the current drivable area, the 2D detection result of the target, and the 3D information of the target, wherein the 9D information includes the length, width, height, three-dimensional coordinates, yaw angle, lateral speed, and longitudinal speed of the target.
8. A vehicle, characterized in that: include: An environment perception system based on a monocular camera according to claim 7.
9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the environment perception method based on a monocular camera according to any one of claims 1 to 6 is implemented.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the environment perception method based on a monocular camera according to any one of claims 1 to 6 is implemented.