A method and device for intelligent vehicle positioning in dynamic scenes based on large models

Abstract

The present application provides a method and device for positioning an intelligent vehicle in a dynamic scene based on a large model, wherein the method includes: obtaining an RGB image captured by a camera on the vehicle at the current moment; using the SLAM front-end visual odometry to estimate the position of the RGB image at the current moment to obtain the first position of the vehicle at the current moment; based on the first position of the vehicle at the current moment and the first position of the vehicle at the previous moment, as well as the RGB image at the current moment and the RGB image at the previous moment, determining the dynamic area in the RGB image at the current moment; removing the dynamic area in the RGB image at the current moment to obtain an RGB image containing only the static area of ​​the scene; using the SLAM front-end visual odometry to estimate the position of the RGB image containing only the static area of ​​the scene to obtain the second position of the vehicle at the current moment. The present application improves the positioning accuracy of intelligent vehicles in environments containing both highly dynamic and movable objects.

Description

Technical Field

[0001] The present application relates to the technical field of visual positioning and object tracking, and in particular to a method and device for positioning an intelligent vehicle in a dynamic scene based on a large model. Background Art

[0002] Current SLAM (Simultanesiological Localization and Mapping) methods typically rely on assuming a static environment or incorporate semantic information modules to achieve pose estimation in dynamic environments. While these SLAM systems can operate in specific static and dynamic environments, they fail in many real-world scenarios due to the large number of unknown, complex dynamic objects present on autonomous driving roads. Summary of the Invention

[0003] In view of this, the present application provides a method and device for positioning an intelligent vehicle in a dynamic scene based on a large model to solve the above technical problems.

[0004] In a first aspect, the present application provides a method for positioning an intelligent vehicle in a dynamic scene based on a large model, comprising:

[0005] Get the RGB image captured by the camera on the vehicle at the current moment;

[0006] Use the SLAM front-end visual odometry to estimate the position of the current RGB image and obtain the first position of the vehicle at the current moment;

[0007] Determining a dynamic area in the RGB image at the current moment based on the first position of the vehicle at the current moment and the first position of the vehicle at the previous moment, and the RGB image at the current moment and the RGB image at the previous moment;

[0008] Remove the dynamic area in the RGB image at the current moment to obtain an RGB image containing only the static area of ​​the scene at the current moment;

[0009] The SLAM front-end visual odometry is used to estimate the position of the RGB image containing only the static area of ​​the scene at the current moment, and the second position of the vehicle at the current moment is obtained.

[0010] In one possible implementation, determining a dynamic area in the RGB image at the current moment based on the first position of the vehicle at the current moment and the first position of the vehicle at the previous moment, as well as the RGB image at the current moment and the RGB image at the previous moment includes:

[0011] Determine a fully static hypothetical optical flow map between the RGB image at the current moment and the RGB image at the previous moment based on the first position of the ego vehicle at the current moment and the first position of the ego vehicle at the previous moment;

[0012] The optical flow neural network is used to process the RGB image at the current moment and the RGB image at the previous moment to determine the actual scene optical flow map between the RGB image at the current moment and the RGB image at the previous moment;

[0013] Comparing the fully static hypothetical optical flow map with the actual scene optical flow map to determine the dynamic optical flow points and the positions of the dynamic object areas;

[0014] Based on the dynamic optical flow points and the location of the dynamic object area, the segmentation model is used to segment the RGB image at the current moment and determine the dynamic object mask area;

[0015] The dynamic object region position and the dynamic object mask region are fused to obtain the dynamic region in the RGB image.

[0016] In one possible implementation, determining a fully static hypothetical optical flow map between the RGB image at the current moment and the RGB image at the previous moment based on the first position of the vehicle at the current moment and the first position of the vehicle at the previous moment includes:

[0017] determining an offset based on the first position of the vehicle at a current moment and the first position of the vehicle at a previous moment;

[0018] The offset is used as the pixel value of each pixel of the full static hypothesis optical flow map to form the full static hypothesis optical flow map.

[0019] In a possible implementation, comparing the fully static hypothetical optical flow map with the actual scene optical flow map to determine the location of the dynamic object region includes:

[0020] Calculate the dynamic optical flow difference ΔF(p) of pixel p:

[0021] ΔF(p)=F real (p)-F static (p)

[0022] Among them, F static (p) is the pixel value of pixel p in the fully static hypothetical optical flow map, F real (p) is the pixel value of pixel p in the actual scene optical flow map;

[0023] Determine whether the absolute value of the dynamic optical flow difference of pixel p is greater than a preset threshold. If so, pixel p is a dynamic optical flow point.

[0024] All dynamic optical flow points are clustered to form dynamic object areas and record their position information.

[0025] In one possible implementation, based on the dynamic optical flow points and the location of the dynamic object region, a large segmentation model is used to segment the RGB image at the current moment to determine the dynamic object mask region; this includes:

[0026] The dynamic optical flow points are used as the points prompt input of the segmentation model, and the dynamic area position is used as the box prompt input of the segmentation model; the segmentation model is run to output the dynamic object mask area.

[0027] In one possible implementation, the method further includes:

[0028] Get the pixel position of the dynamic object area at the current moment output by the segmentation model;

[0029] Determine the three-dimensional position of the dynamic object in the camera coordinate system using the camera's depth value and the pixel position of the dynamic object area at the current moment;

[0030] The three-dimensional position of the dynamic object in the geocentric rectangular coordinate system is determined using the second position of the vehicle at the current moment.

[0031] In a second aspect, an embodiment of the present application provides a smart car positioning device in a dynamic scene based on a large model, comprising:

[0032] An acquisition unit is used to acquire the RGB image captured by the camera on the vehicle at the current moment;

[0033] The first position estimation unit is used to estimate the position of the RGB image at the current moment using the SLAM front-end visual odometry to obtain the first position of the vehicle at the current moment;

[0034] a first processing unit, configured to determine a dynamic area in the RGB image at a current moment based on the first position of the vehicle at a current moment and the first position of the vehicle at a previous moment, the RGB image at a current moment and the RGB image at a previous moment;

[0035] The second processing unit is configured to remove the dynamic area in the RGB image at the current moment, thereby obtaining an RGB image containing only the static area of ​​the scene at the current moment;

[0036] The second position estimation unit is used to use the SLAM front-end visual odometry to estimate the position of the RGB image containing only the static area of ​​the scene at the current moment, and obtain the second position of the vehicle at the current moment.

[0037] In a third aspect, an embodiment of the present application provides an electronic device comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the method of the embodiment of the present application when executing the computer program.

[0038] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method of the embodiment of the present application is implemented.

[0039] This application improves the positioning accuracy of smart cars in environments containing both highly dynamic and movable objects. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the specific implementation methods or the description of the prior art. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0041] Figure 1 A technical roadmap for a large-model-based intelligent vehicle positioning method in dynamic scenarios provided in an embodiment of the present application;

[0042] Figure 2 A flowchart of a method for positioning an intelligent vehicle in a dynamic scene based on a large model provided in an embodiment of the present application;

[0043] Figure 3 A flowchart of the process of segmenting the large model SAM2 provided in an embodiment of the present application;

[0044] Figure 4 This is a functional structure diagram of a smart car positioning device in a dynamic scenario based on a large model provided by an embodiment of the present application;

[0045] Figure 5 This is a functional structure diagram of the electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0046] To make the objectives, technical solutions, and advantages of the embodiments of the present application more clear, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Generally, the components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations.

[0047] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for protection, but merely represents selected embodiments of the present application. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments in the present application without creative work are within the scope of protection of the present application.

[0048] First, a brief introduction to the design concept of the embodiments of the present application is given.

[0049] Visual simultaneous localization and mapping (V-SLAM) technology estimates the camera's pose and simultaneously constructs a map of the environment. Object tracking and pose estimation typically obtain the trajectory of objects in the environment, thereby enabling object tracking. Currently, visual SLAM and object tracking technologies have been widely used in robotics, autonomous driving, virtual / augmented reality, and other fields. However, some traditional SLAM algorithms rely on assuming a static environment or incorporate semantic information modules to achieve pose estimation in dynamic environments. These SLAM systems can operate in specific dynamic and static environments, but they fail in many real-world scenarios, such as the large number of unknown and complex dynamic objects on roads in autonomous driving. Therefore, achieving accurate pose estimation in diverse and unknown dynamic environments remains a major challenge. Current dynamic SLAM algorithms generally perform well only on datasets for which they have been trained. The gap between these datasets and real-world environments leads to a significant performance degradation in long-tail distributions or wild samples, limiting the performance of dynamic SLAM. Furthermore, the moment-to-moment changes of objects in dynamic environments affect the choice of navigation paths. Therefore, a dynamic SLAM system with object pose estimation capability and stronger generalization capability is needed to ensure better positioning accuracy in different scenarios and to estimate object pose in real time for obstacle avoidance navigation, especially in unknown scenarios outside the training domain.

[0050] To this end, the embodiment of the present application is a method for positioning an intelligent vehicle in a dynamic scene based on a large model, wherein the dynamic scene contains unknown and known objects, dynamic and movable static objects; the method uses dense optical flow to obtain basic position information of dynamic objects, inputs it into the large model as a prompt, completely segments the dynamic object area, and based on the memory tracking function of the optical flow and segmentation module, thoroughly distinguishes the dynamic and static areas in the scene; the static area estimates the vehicle's posture based on SLAM, and the dynamic area is used to track and estimate the posture of dynamic objects. The specific technical route is as follows Figure 1 shown.

[0051] Compared with the prior art, the advantages of this application are:

[0052] 1. The method of the present application improves the positioning accuracy of a smart car in an environment containing both highly dynamic and movable objects; it solves the problem of reduced positioning accuracy caused by the high dynamic characteristics of highly dynamic objects;

[0053] 2. This application also provides a method for tracking dynamic objects, which can track and obtain the position of dynamic objects in any environment, and ultimately generate a camera trajectory, a static area map, and a dynamic object trajectory in the camera coordinate system.

[0054] After introducing the application scenarios and design concepts of the embodiments of the present application, the technical solutions provided by the embodiments of the present application are described below.

[0055] like Figure 2 As shown, the present application provides a method for positioning an intelligent vehicle in a dynamic scene based on a large model, including:

[0056] Step 101: Obtain the RGB image captured by the camera on the vehicle at the current moment;

[0057] Step 102: Use the SLAM front-end visual odometry to estimate the position of the RGB image at the current moment to obtain the first position of the vehicle at the current moment;

[0058] Step 103: Determine a dynamic area in the RGB image at the current moment based on the first position of the vehicle at the current moment and the first position of the vehicle at the previous moment, and the RGB image at the current moment and the RGB image at the previous moment;

[0059] Step 104: removing the dynamic area in the RGB image at the current moment, and obtaining an RGB image containing only the static area of ​​the scene at the current moment;

[0060] Step 105: Use the SLAM front-end visual odometry to estimate the position of the RGB image that only contains the static area of ​​the scene at the current moment to obtain the second position of the vehicle at the current moment.

[0061] In some embodiments, the implementation process of step 103 includes:

[0062] Determine a fully static hypothetical optical flow map between the RGB image at the current moment and the RGB image at the previous moment based on the first position of the ego vehicle at the current moment and the first position of the ego vehicle at the previous moment;

[0063] The optical flow neural network (PWC-NET) is used to process the RGB image at the current moment and the RGB image at the previous moment to determine the actual scene optical flow map between the RGB image at the current moment and the RGB image at the previous moment;

[0064] Comparing the fully static hypothetical optical flow map with the actual scene optical flow map to determine the dynamic optical flow points and the positions of the dynamic object areas;

[0065] Based on the dynamic optical flow points and the location of the dynamic object area, the segmentation model (SAM2) is used to segment the RGB image at the current moment and determine the dynamic object mask area;

[0066] The dynamic object region position and the dynamic object mask region are fused to obtain the dynamic region in the RGB image.

[0067] In some embodiments, determining a fully static hypothetical optical flow map between the RGB image at the current moment and the RGB image at the previous moment based on the first position of the vehicle at the current moment and the first position of the vehicle at the previous moment includes:

[0068] determining an offset based on the first position of the vehicle at a current moment and the first position of the vehicle at a previous moment;

[0069] The offset is used as the pixel value of each pixel of the full static hypothesis optical flow map to form the full static hypothesis optical flow map.

[0070] In some embodiments, comparing the fully static hypothetical optical flow map with the actual scene optical flow map to determine the location of the dynamic object region includes:

[0071] Calculate the dynamic optical flow difference ΔF(p) of pixel p:

[0072] ΔF(p)=F real (p)-F static (p)

[0073] Among them, F static (p) is the pixel value of pixel p in the fully static hypothetical optical flow map, F real (p) is the pixel value of pixel p in the actual scene optical flow map;

[0074] Determine whether the absolute value of the dynamic optical flow difference of pixel p is greater than a preset threshold. If so, pixel p is a dynamic optical flow point.

[0075] All dynamic optical flow points are clustered to form dynamic object areas and record their position information.

[0076] In some embodiments, based on the dynamic optical flow points and the location of the dynamic object area, the current RGB image is segmented using a large segmentation model to determine the dynamic object mask area; including:

[0077] The dynamic optical flow points are used as the points prompt input of the segmentation model, and the dynamic area position is used as the box prompt input of the segmentation model; the segmentation model is run to output the dynamic object mask area.

[0078] like Figure 3 As shown in the figure, the memory storage module based on the large model records the position of dynamic objects at each moment and can also use information from previous frames to assist in segmenting the current frame, obtaining dynamic object mask information in spatiotemporal state. Even in the presence of occlusion and lighting changes, dynamic objects can be efficiently processed and segmented.

[0079] In some embodiments, the method further comprises:

[0080] Get the pixel position of the dynamic object area at the current moment output by the segmentation model;

[0081] Determine the three-dimensional position of the dynamic object in the camera coordinate system using the camera's depth value and the pixel position of the dynamic object area at the current moment;

[0082] The three-dimensional position of the dynamic object in the geocentric rectangular coordinate system is determined using the second position of the vehicle at the current moment.

[0083] For example, the center pixel of a dynamic object is (u, v), the corresponding depth information is di, and the camera intrinsic parameter matrix is ​​K. The corresponding three-dimensional space coordinates in the camera coordinate system are:

[0084] X cam =(x,y,z) T =K -1 ·di·(u,v,1) T

[0085] At this time, in the camera coordinate system, the object's position is X global =(X,Y,Z) T =X cam T. By recording the object's position at each moment and connecting them together to form the object's trajectory, it is expressed as the motion path of the dynamic object.

[0086] Based on the dynamic area information stored in the memory module of the large model and obtained from optical flow, the depth information obtained from the camera image, and the estimated high-precision pose, the pose of dynamic objects in the camera coordinate system is estimated and a dynamic object pose trajectory layer is formed. The trajectory layer is fused with the static map layer to form an overall map, ultimately achieving dynamic object pose tracking while maintaining the estimated camera pose. The resulting overall map can be used for real-time dynamic object navigation, without training, and can be used directly in unknown environments.

[0087] Based on the above embodiments, the present application provides a smart car positioning device in a dynamic scene based on a large model, see Figure 4 As shown, the embodiment of the present application provides a smart car positioning device 200 in a dynamic scene based on a large model, which at least includes:

[0088] An acquisition unit 201 is used to acquire an RGB image captured by a camera on the vehicle at the current moment;

[0089] The first position estimation unit 202 is used to estimate the position of the RGB image at the current moment using the SLAM front-end visual odometry to obtain the first position of the vehicle at the current moment;

[0090] The first processing unit 203 is configured to determine a dynamic area in the RGB image at a current moment based on the first position of the vehicle at a current moment and the first position of the vehicle at a previous moment, and the RGB image at a current moment and the RGB image at a previous moment;

[0091] The second processing unit 204 is configured to remove the dynamic area in the RGB image at the current moment, thereby obtaining an RGB image containing only the static area of ​​the scene at the current moment;

[0092] The second position estimation unit 205 is used to use the SLAM front-end visual odometry to perform position estimation on the RGB image containing only the static area of ​​the scene at the current moment to obtain the second position of the vehicle at the current moment.

[0093] It should be noted that the principle of solving the technical problem of the smart car positioning device 200 based on a large model in a dynamic scene provided in the embodiment of the present application is similar to the method provided in the embodiment of the present application. Therefore, the implementation of the smart car positioning device 200 based on a large model in a dynamic scene provided in the embodiment of the present application can refer to the implementation of the method provided in the embodiment of the present application, and the repeated parts will not be repeated.

[0094] Based on the above embodiments, the present application also provides an electronic device, referring to Figure 5 As shown, the electronic device 300 provided in the embodiment of the present application includes at least: a processor 301, a memory 302, and a computer program stored in the memory 302 and executable on the processor 301. When the processor 301 executes the computer program, the smart car positioning method in a dynamic scene based on a large model provided in the embodiment of the present application is implemented.

[0095] The electronic device 300 provided in the embodiment of the present application may further include a bus 303 connecting different components (including the processor 301 and the memory 302). The bus 303 represents one or more of several types of bus structures, including a memory bus, a peripheral bus, a local bus, and the like.

[0096] The memory 302 may include a readable medium in the form of a volatile memory, such as a random access memory (RAM) 3021 and / or a cache memory 3022 , and may further include a read-only memory (ROM) 3023 .

[0097] The memory 302 may also include a program tool 3025 having a set (at least one) of program modules 3024, including but not limited to: an operating subsystem, one or more application programs, other program modules and program data, each of which or some combination may include an implementation of a network environment.

[0098] The electronic device 300 may also communicate with one or more external devices 304 (e.g., keyboards, remote controls, etc.), and may also communicate with one or more devices that enable a user to interact with the electronic device 300 (e.g., mobile phones, computers, etc.), and / or any device that enables the electronic device 300 to communicate with one or more other electronic devices 300 (e.g., routers, modems, etc.). Such communication may be performed through an input / output (I / O) interface 305. Furthermore, the electronic device 300 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 306. Figure 5 As shown, the network adapter 306 communicates with other modules of the electronic device 300 via the bus 303. Figure 5 Not shown, other hardware and / or software modules may be used in conjunction with the electronic device 300, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, disk arrays (Redundant Arrays of Independent Disks, RAID) subsystems, tape drives, and data backup storage subsystems.

[0099] It should be noted that Figure 5 The electronic device 300 shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0100] The present application also provides a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the large-model-based intelligent vehicle positioning method for dynamic scenarios provided by the present application. Specifically, the executable program can be built into or installed in the electronic device 300. Thus, the electronic device 300 can implement the large-model-based intelligent vehicle positioning method for dynamic scenarios provided by the present application by executing the built-in or installed executable program.

[0101] The smart car positioning method in a dynamic scene based on a large model provided in an embodiment of the present application can also be implemented as a program product, which includes a program code. When the program product can be run on an electronic device 300, the program code is used to enable the electronic device 300 to execute the smart car positioning method in a dynamic scene based on a large model provided in an embodiment of the present application.

[0102] The program product provided in the embodiments of the present application may adopt any combination of one or more readable media, wherein the readable medium may be a readable signal medium or a readable storage medium, and the readable storage medium may be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any combination of the above. Specifically, more specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, RAM, ROM, Erasable Programmable Read-Only Memory (EPROM), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0103] The program product provided in the embodiments of the present application may be a CD-ROM and include program code, and may also be run on a computing device. However, the program product provided in the embodiments of the present application is not limited thereto. In the embodiments of the present application, the readable storage medium may be any tangible medium containing or storing a program, and the program may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0104] It should be noted that although several units or subunits of the device are mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, depending on the embodiment of the application, the features and functions of two or more units described above can be embodied in a single unit. Conversely, the features and functions of a single unit described above can be further divided and embodied by multiple units.

[0105] Furthermore, although the operations of the method of the present application are described in a particular order in the accompanying drawings, this does not require or imply that the operations must be performed in this particular order, or that all illustrated operations must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0106] Finally, it should be noted that the above embodiments are intended only to illustrate the technical solutions of this application and are not intended to limit the scope of the present invention. Although this application has been described in detail with reference to the embodiments, it should be understood by those skilled in the art that modifications or equivalent substitutions to the technical solutions of this application do not depart from the spirit and scope of the technical solutions of this application and should be encompassed by the claims of this application.

Claims

1. A method for positioning an intelligent vehicle in a dynamic scene based on a large model, characterized in that: include: Get the RGB image captured by the camera on the vehicle at the current moment; Use the SLAM front-end visual odometry to estimate the position of the current RGB image and obtain the first position of the vehicle at the current moment; Determining a dynamic area in the RGB image at the current moment based on the first position of the vehicle at the current moment and the first position of the vehicle at the previous moment, and the RGB image at the current moment and the RGB image at the previous moment; Remove the dynamic area in the RGB image at the current moment to obtain an RGB image containing only the static area of ​​the scene at the current moment; The SLAM front-end visual odometry is used to estimate the position of the RGB image containing only the static area of ​​the scene at the current moment, and the second position of the vehicle at the current moment is obtained; Determining a dynamic area in the RGB image at the current moment based on the first position of the vehicle at the current moment and the first position of the vehicle at the previous moment, as well as the RGB image at the current moment and the RGB image at the previous moment; including: Determine a fully static hypothetical optical flow map between the RGB image at the current moment and the RGB image at the previous moment based on the first position of the ego vehicle at the current moment and the first position of the ego vehicle at the previous moment; The optical flow neural network is used to process the RGB image at the current moment and the RGB image at the previous moment to determine the actual scene optical flow map between the RGB image at the current moment and the RGB image at the previous moment; Comparing the fully static hypothetical optical flow map with the actual scene optical flow map to determine the dynamic optical flow points and the positions of the dynamic object areas; Based on the dynamic optical flow points and the location of the dynamic object area, the segmentation model is used to segment the RGB image at the current moment and determine the dynamic object mask area; The dynamic object region position and the dynamic object mask region are fused to obtain the dynamic region in the RGB image.

2. The method for positioning an intelligent vehicle in a dynamic scene based on a large model according to claim 1, characterized in that: Based on the first position of the vehicle at the current moment and the first position of the vehicle at the previous moment, a full static hypothetical optical flow map between the RGB image at the current moment and the RGB image at the previous moment is determined; including: determining an offset based on the first position of the vehicle at a current moment and the first position of the vehicle at a previous moment; The offset is used as the pixel value of each pixel of the full static hypothesis optical flow map to form the full static hypothesis optical flow map.

3. The method for positioning an intelligent vehicle in a dynamic scene based on a large model according to claim 2, characterized in that: Comparing the fully static hypothetical optical flow map with the actual scene optical flow map to determine the location of the dynamic object area includes: Counting pixels Dynamic optical flow difference : in, is the pixel of the fully static hypothetical optical flow map The pixel value of is the pixel of the actual scene optical flow map Pixel value of Determine pixel Is the absolute value of the dynamic optical flow difference greater than the preset threshold? If yes, the pixel is a dynamic optical flow point; All dynamic optical flow points are clustered to form dynamic object areas and record their position information.

4. The method for positioning an intelligent vehicle in a dynamic scene based on a large model according to claim 3, characterized in that: Based on the dynamic optical flow points and the location of the dynamic object area, the current RGB image is segmented using the segmentation model to determine the dynamic object mask area; including: The dynamic optical flow points are used as the points prompt input of the segmentation model, and the dynamic area position is used as the box prompt input of the segmentation model; the segmentation model is run to output the dynamic object mask area.

5. The method for positioning an intelligent vehicle in a dynamic scene based on a large model according to claim 4, characterized in that: The method further comprises: Get the pixel position of the dynamic object area at the current moment output by the segmentation model; Determine the three-dimensional position of the dynamic object in the camera coordinate system using the camera's depth value and the pixel position of the dynamic object area at the current moment; The three-dimensional position of the dynamic object in the geocentric rectangular coordinate system is determined using the second position of the vehicle at the current moment.

6. A smart car positioning device in dynamic scenes based on a large model, characterized by: include: An acquisition unit is used to acquire the RGB image captured by the camera on the vehicle at the current moment; The first position estimation unit is used to estimate the position of the RGB image at the current moment using the SLAM front-end visual odometry to obtain the first position of the vehicle at the current moment; a first processing unit, configured to determine a dynamic area in the RGB image at a current moment based on the first position of the vehicle at a current moment and the first position of the vehicle at a previous moment, the RGB image at a current moment and the RGB image at a previous moment; The second processing unit is configured to remove the dynamic area in the RGB image at the current moment, thereby obtaining an RGB image containing only the static area of ​​the scene at the current moment; The second position estimation unit is used to use the SLAM front-end visual odometry to perform position estimation on the RGB image second position estimation unit containing only the static area of ​​the scene at the current moment to obtain the second position of the vehicle at the current moment; The first processing unit is specifically configured to: Determine a fully static hypothetical optical flow map between the RGB image at the current moment and the RGB image at the previous moment based on the first position of the ego vehicle at the current moment and the first position of the ego vehicle at the previous moment; The optical flow neural network is used to process the RGB image at the current moment and the RGB image at the previous moment to determine the actual scene optical flow map between the RGB image at the current moment and the RGB image at the previous moment; Comparing the fully static hypothetical optical flow map with the actual scene optical flow map to determine the dynamic optical flow points and the positions of the dynamic object areas; Based on the dynamic optical flow points and the location of the dynamic object area, the segmentation model is used to segment the RGB image at the current moment and determine the dynamic object mask area; The dynamic object region position and the dynamic object mask region are fused to obtain the dynamic region in the RGB image.

7. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 5 when executing the computer program.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images