Intelligent vehicle positioning method and device in dynamic scene based on large model

Abstract

The invention provides an intelligent vehicle positioning method and device in a dynamic scene based on a large model. The method comprises the steps that an RGB image collected by a camera on a vehicle at the current moment is acquired; performing position estimation on the RGB image at the current moment by using an SLAM front-end visual odometer to obtain a first position of the vehicle at the current moment; determining a dynamic area in the RGB image at the current moment based on the first position of the vehicle at the current moment, the first position of the vehicle at the previous moment, the RGB image at the current moment and the RGB image at the previous moment; removing a dynamic region in the RGB image at the current moment to obtain an RGB image only containing a scene static region; and performing position estimation on the RGB image only containing the scene static region by using an SLAM front-end visual odometer to obtain a second position of the vehicle at the current moment. According to the invention, the positioning precision of the intelligent vehicle in an environment containing high-dynamic and movable objects at the same time is improved.

Description

Technical Field

[0001] The present application relates to the technical field of visual positioning and object tracking, and in particular to a method and device for positioning an intelligent vehicle in a dynamic scene based on a large model. Background Art

[0002] Current SLAM (simultaneous localization and mapping) methods usually rely on assuming a static environment, or combine semantic information modules to achieve pose estimation in a dynamic environment. This SLAM system can operate in specific dynamic and static environments, but it cannot operate in many real-world scenarios because there are a large number of unknown complex dynamic objects in road scenes in autonomous driving. Summary of the invention

[0003] In view of this, the present application provides a method and device for positioning an intelligent vehicle in a dynamic scene based on a large model to solve the above technical problems.

[0004] In a first aspect, the present application provides a method for positioning an intelligent vehicle in a dynamic scene based on a large model, comprising:

[0005] Get the RGB image captured by the camera on the vehicle at the current moment;

[0006] Use the SLAM front-end visual odometer to estimate the position of the current RGB image and obtain the first position of the vehicle at the current moment;

[0007] Determine a dynamic area in the RGB image at the current moment based on the first position of the vehicle at the current moment and the first position of the vehicle at the previous moment, and the RGB image at the current moment and the RGB image at the previous moment;

[0008] Remove the dynamic area in the RGB image at the current moment to obtain an RGB image containing only the static area of ​​the scene at the current moment;

[0009] The SLAM front-end visual odometer is used to estimate the position of the RGB image that only contains the static area of ​​the scene at the current moment, and the second position of the vehicle at the current moment is obtained.

[0010] In a possible implementation, based on the first position of the vehicle at the current moment and the first position of the vehicle at the previous moment, and the RGB image at the current moment and the RGB image at the previous moment, determining the dynamic area in the RGB image at the current moment includes:

[0011] Based on the first position of the ego vehicle at the current moment and the first position of the ego vehicle at the previous moment, determining a full static hypothetical optical flow map between the RGB image at the current moment and the RGB image at the previous moment;

[0012] The optical flow neural network is used to process the RGB image at the current moment and the RGB image at the previous moment to determine the actual scene optical flow map between the RGB image at the current moment and the RGB image at the previous moment;

[0013] Compare the fully static hypothetical optical flow map with the actual scene optical flow map to determine the dynamic optical flow points and the positions of the dynamic object regions;

[0014] Based on the dynamic optical flow points and the location of the dynamic object area, the segmentation model is used to segment the RGB image at the current moment to determine the dynamic object mask area;

[0015] The dynamic object region position and the dynamic object mask region are fused to obtain the dynamic region in the RGB image.

[0016] In a possible implementation, based on the first position of the vehicle at the current moment and the first position of the vehicle at the previous moment, determining a fully static hypothetical optical flow map between the RGB image at the current moment and the RGB image at the previous moment includes:

[0017] Determine an offset based on the first position of the vehicle at the current moment and the first position of the vehicle at the previous moment;

[0018] The offset is used as the pixel value of each pixel of the full static hypothetical optical flow map to form the full static hypothetical optical flow map.

[0019] In a possible implementation, comparing the fully static hypothetical optical flow map with the actual scene optical flow map to determine the location of the dynamic object region includes:

[0020] Calculate the dynamic optical flow difference ΔF(p) of pixel p:

[0021] ΔF(p)=F real (p)-F static (p)

[0022] Among them, F static (p) is the pixel value of pixel p in the fully static hypothetical optical flow map, F real (p) is the pixel value of pixel p in the actual scene optical flow map;

[0023] Determine whether the absolute value of the dynamic optical flow difference of pixel p is greater than a preset threshold, if yes, pixel p is a dynamic optical flow point;

[0024] All dynamic optical flow points are clustered to form dynamic object areas and record their position information.

[0025] In a possible implementation, based on the dynamic optical flow points and the position of the dynamic object region, the RGB image at the current moment is segmented using a large segmentation model to determine the dynamic object mask region; including:

[0026] The dynamic optical flow points are used as the points prompt input of the segmentation model, and the dynamic area position is used as the box prompt input of the segmentation model; the segmentation model is run to output the dynamic object mask area.

[0027] In a possible implementation, the method further includes:

[0028] Obtain the pixel position of the dynamic object area at the current moment output by the segmentation model;

[0029] Using the depth value of the camera and the pixel position of the dynamic object area at the current moment, determine the three-dimensional position of the dynamic object in the camera coordinate system;

[0030] The three-dimensional position of the dynamic object in the geocentric rectangular coordinate system is determined using the second position of the vehicle at the current moment.

[0031] In a second aspect, an embodiment of the present application provides a smart car positioning device in a dynamic scene based on a large model, comprising:

[0032] An acquisition unit, used to acquire an RGB image captured by a camera on the vehicle at the current moment;

[0033] A first position estimation unit is used to estimate the position of the RGB image at the current moment by using the SLAM front-end visual odometer to obtain the first position of the vehicle at the current moment;

[0034] A first processing unit, configured to determine a dynamic area in the RGB image at a current moment based on a first position of the vehicle at a current moment and a first position of the vehicle at a previous moment, and the RGB image at a current moment and the RGB image at a previous moment;

[0035] The second processing unit is used to remove the dynamic area in the RGB image at the current moment, so as to obtain the RGB image containing only the static area of ​​the scene at the current moment;

[0036] The second position estimation unit is used to use the SLAM front-end visual odometer to estimate the position of the RGB image containing only the static area of ​​the scene at the current moment, so as to obtain the second position of the vehicle at the current moment.

[0037] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method of the embodiment of the present application when executing the computer program.

[0038] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method of the embodiment of the present application is implemented.

[0039] The present application improves the positioning accuracy of a smart car in an environment containing both highly dynamic and movable objects. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0041] Figure 1 A technical roadmap for a smart car positioning method in a dynamic scene based on a large model provided in an embodiment of the present application;

[0042] Figure 2 A flowchart of a method for positioning an intelligent vehicle in a dynamic scene based on a large model provided in an embodiment of the present application;

[0043] Figure 3 A processing flow chart of segmenting a large model SAM2 provided in an embodiment of the present application;

[0044] Figure 4 A functional structure diagram of a smart car positioning device in a dynamic scene based on a large model provided in an embodiment of the present application;

[0045] Figure 5 A functional structure diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0047] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0048] First, the design concept of the embodiment of the present application is briefly introduced.

[0049] Visual simultaneous localization and mapping (V-SLAM) technology can estimate the pose of the camera and build an environmental map at the same time. Object tracking and pose estimation can usually obtain the trajectory of objects in the environment, thereby realizing the trajectory tracking of objects. At present, visual SLAM and object tracking technologies have been widely used in robotics, autonomous driving, virtual / augmented reality and other fields. However, some traditional SLAM algorithms rely on assuming a static environment, or combine semantic information modules to achieve pose estimation in dynamic environments. These SLAM systems can operate in specific dynamic and static environments, but cannot operate in many real-world scenarios, such as the presence of a large number of unknown complex dynamic objects in road scenes in autonomous driving. Therefore, achieving accurate pose estimation in different unknown dynamic environments remains a major challenge, and current dynamic SLAM algorithms usually only perform well on trained datasets. The gap between these datasets and the real environment leads to a significant decrease in the performance of long-tail distributions or wild samples, which limits the performance of dynamic SLAM. At the same time, the momentary changes of objects in dynamic environments affect the choice of navigation paths. Therefore, a dynamic SLAM system with object pose estimation capability and stronger generalization capability is needed to ensure better positioning accuracy in different scenarios and to estimate object pose in real time for obstacle avoidance navigation, especially in unknown scenarios outside the training domain.

[0050] To this end, the present application embodiment provides a method for positioning an intelligent vehicle in a dynamic scene based on a large model, wherein the dynamic scene contains unknown and known objects, dynamic and movable static objects; the method uses dense optical flow to obtain basic position information of dynamic objects, and inputs it into the large model as a prompt, completely segments the dynamic object area, and based on the memory tracking function of the optical flow and segmentation module, thoroughly distinguishes the dynamic and static areas in the scene; the static area estimates the vehicle's posture based on SLAM, and the dynamic area is used to track and estimate the posture of dynamic objects. Specific technical routes are as follows: Figure 1 shown.

[0051] Compared with the prior art, the advantages of this application are:

[0052] 1. The method of the present application improves the positioning accuracy of the smart car in an environment containing both high-dynamic and movable objects; and solves the problem of reduced positioning accuracy due to the high-dynamic characteristics of high-dynamic objects;

[0053] 2. This application also provides a dynamic object tracking method, which can track and obtain the position and posture of dynamic objects in any environment, and finally generate a camera trajectory, a static area map and a dynamic object trajectory in the camera coordinate system.

[0054] After introducing the application scenarios and design concepts of the embodiments of the present application, the technical solutions provided by the embodiments of the present application are described below.

[0055] like Figure 2 As shown, the present application implements a method for positioning an intelligent vehicle in a dynamic scene based on a large model, including:

[0056] Step 101: Obtain an RGB image captured by a camera on the vehicle at the current moment;

[0057] Step 102: Use the SLAM front-end visual odometer to estimate the position of the RGB image at the current moment to obtain the first position of the vehicle at the current moment;

[0058] Step 103: determining a dynamic area in the RGB image at the current moment based on the first position of the vehicle at the current moment and the first position of the vehicle at the previous moment, and the RGB image at the current moment and the RGB image at the previous moment;

[0059] Step 104: removing the dynamic area in the RGB image at the current moment, and obtaining an RGB image containing only the static area of ​​the scene at the current moment;

[0060] Step 105: Use the SLAM front-end visual odometer to estimate the position of the RGB image that only contains the static area of ​​the scene at the current moment, and obtain the second position of the vehicle at the current moment.

[0061] In some embodiments, the implementation process of step 103 includes:

[0062] Based on the first position of the ego vehicle at the current moment and the first position of the ego vehicle at the previous moment, determining a full static hypothetical optical flow map between the RGB image at the current moment and the RGB image at the previous moment;

[0063] The optical flow neural network (PWC-NET) is used to process the RGB image at the current moment and the RGB image at the previous moment to determine the actual scene optical flow map between the RGB image at the current moment and the RGB image at the previous moment;

[0064] Compare the fully static hypothetical optical flow map with the actual scene optical flow map to determine the dynamic optical flow points and the positions of the dynamic object regions;

[0065] Based on the dynamic optical flow points and the location of the dynamic object area, the segmentation model (SAM2) is used to segment the RGB image at the current moment and determine the dynamic object mask area;

[0066] The dynamic object region position and the dynamic object mask region are fused to obtain the dynamic region in the RGB image.

[0067] In some embodiments, based on the first position of the vehicle at the current moment and the first position of the vehicle at the previous moment, determining a fully static hypothetical optical flow map between the RGB image at the current moment and the RGB image at the previous moment; comprising:

[0068] Determine an offset based on the first position of the vehicle at the current moment and the first position of the vehicle at the previous moment;

[0069] The offset is used as the pixel value of each pixel of the full static hypothetical optical flow map to form the full static hypothetical optical flow map.

[0070] In some embodiments, comparing the fully static hypothetical optical flow map with the actual scene optical flow map to determine the location of the dynamic object region includes:

[0071] Calculate the dynamic optical flow difference ΔF(p) of pixel p:

[0072] ΔF(p)=F real (p)-F static (p)

[0073] Among them, F static (p) is the pixel value of pixel p in the fully static hypothetical optical flow map, F real (p) is the pixel value of pixel p in the actual scene optical flow map;

[0074] Determine whether the absolute value of the dynamic optical flow difference of pixel p is greater than a preset threshold, if yes, pixel p is a dynamic optical flow point;

[0075] All dynamic optical flow points are clustered to form dynamic object areas and record their position information.

[0076] In some embodiments, based on the dynamic optical flow points and the dynamic object region positions, the RGB image at the current moment is segmented using a segmentation model to determine the dynamic object mask region; including:

[0077] The dynamic optical flow points are used as the points prompt input of the segmentation model, and the dynamic area position is used as the box prompt input of the segmentation model; the segmentation model is run to output the dynamic object mask area.

[0078] like Figure 3 As shown in the figure, the memory storage module based on the large model records the position of dynamic objects at each moment, and can also use the information of previous frames to assist in segmenting the current frame and obtain the mask information of dynamic objects in the spatiotemporal state. Even in the presence of occlusion and lighting changes, dynamic objects can be efficiently processed and segmented.

[0079] In some embodiments, the method further comprises:

[0080] Obtain the pixel position of the dynamic object area at the current moment output by the segmentation model;

[0081] Using the depth value of the camera and the pixel position of the dynamic object area at the current moment, determine the three-dimensional position of the dynamic object in the camera coordinate system;

[0082] The three-dimensional position of the dynamic object in the geocentric rectangular coordinate system is determined using the second position of the vehicle at the current moment.

[0083] For example, the center pixel of a dynamic object is (u, v), the corresponding depth information is di, and the camera intrinsic parameter matrix is ​​K. The corresponding three-dimensional space coordinates in the camera coordinate system are:

[0084] X cam =(x,y,z) T =K -1 ·di·(u,v,1) T

[0085] At this time, in the camera coordinate system, the object's position is X global =(X,Y,Z) T =X cam T. By recording the position of the object at each moment and connecting them in series to form the object trajectory, it is expressed as the motion path of the dynamic object.

[0086] Based on the dynamic area information in the memory storage module in the large model and the dynamic area information obtained by the optical flow, the depth information obtained from the camera image, and the estimated high-precision pose. Estimate the pose of the dynamic object in the camera coordinate system and form a dynamic object pose trajectory layer. The trajectory layer and the static map layer are fused as the overall map, and finally the pose of the dynamic object is tracked and obtained while estimating the camera pose. The final overall map can be used for real-time dynamic object navigation, and no training is required, and it can be used directly in unknown environments.

[0087] Based on the above embodiments, the present application embodiment provides a smart car positioning device in a dynamic scene based on a large model, see Figure 4 As shown, the embodiment of the present application provides a smart car positioning device 200 in a dynamic scene based on a large model, which at least includes:

[0088] An acquisition unit 201 is used to acquire an RGB image captured by a camera on the vehicle at the current moment;

[0089] The first position estimation unit 202 is used to estimate the position of the RGB image at the current moment by using the SLAM front-end visual odometer to obtain the first position of the vehicle at the current moment;

[0090] The first processing unit 203 is used to determine the dynamic area in the RGB image at the current moment based on the first position of the vehicle at the current moment and the first position of the vehicle at the previous moment, and the RGB image at the current moment and the RGB image at the previous moment;

[0091] The second processing unit 204 is used to remove the dynamic area in the RGB image at the current moment, so as to obtain an RGB image containing only the static area of ​​the scene at the current moment;

[0092] The second position estimation unit 205 is used to use the SLAM front-end visual odometer to perform position estimation on the RGB image that only contains the static area of ​​the scene at the current moment, so as to obtain the second position of the vehicle at the current moment.

[0093] It should be noted that the principle of solving the technical problem by the smart vehicle positioning device 200 based on a large model in a dynamic scene provided in the embodiment of the present application is similar to the method provided in the embodiment of the present application. Therefore, the implementation of the smart vehicle positioning device 200 based on a large model in a dynamic scene provided in the embodiment of the present application can refer to the implementation of the method provided in the embodiment of the present application, and the repeated parts will not be repeated.

[0094] Based on the above embodiments, the present application also provides an electronic device, referring to Figure 5 As shown, the electronic device 300 provided in the embodiment of the present application includes at least: a processor 301, a memory 302, and a computer program stored in the memory 302 and executable on the processor 301. When the processor 301 executes the computer program, the smart car positioning method in a dynamic scene based on a large model provided in the embodiment of the present application is implemented.

[0095] The electronic device 300 provided in the embodiment of the present application may further include a bus 303 connecting different components (including the processor 301 and the memory 302). The bus 303 represents one or more of several types of bus structures, including a memory bus, a peripheral bus, a local bus, and the like.

[0096] The memory 302 may include a readable medium in the form of a volatile memory, such as a random access memory (RAM) 3021 and / or a cache memory 3022 , and may further include a read-only memory (ROM) 3023 .

[0097] The memory 302 may also include a program tool 3025 having a set (at least one) of program modules 3024, the program modules 3024 including but not limited to: an operating subsystem, one or more applications, other program modules and program data, each of which or some combination may include the implementation of a network environment.

[0098] The electronic device 300 may also communicate with one or more external devices 304 (e.g., keyboards, remote controls, etc.), and may also communicate with one or more devices that enable a user to interact with the electronic device 300 (e.g., mobile phones, computers, etc.), and / or communicate with any device that enables the electronic device 300 to communicate with one or more other electronic devices 300 (e.g., routers, modems, etc.). Such communication may be performed through an input / output (I / O) interface 305. Furthermore, the electronic device 300 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 306. Figure 5 As shown, the network adapter 306 communicates with other modules of the electronic device 300 via the bus 303. It should be understood that although Figure 5 Not shown, other hardware and / or software modules may be used in conjunction with the electronic device 300, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, disk arrays (Redundant Arrays of Independent Disks, RAID) subsystems, tape drives, and data backup storage subsystems.

[0099] It should be noted that Figure 5 The electronic device 300 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0100] The embodiment of the present application also provides a computer-readable storage medium, which stores computer instructions, and when the computer instructions are executed by the processor, the smart car positioning method in a dynamic scene based on a large model provided in the embodiment of the present application is implemented. Specifically, the executable program can be built-in or installed in the electronic device 300, so that the electronic device 300 can implement the smart car positioning method in a dynamic scene based on a large model provided in the embodiment of the present application by executing the built-in or installed executable program.

[0101] The smart vehicle positioning method in a dynamic scene based on a large model provided in an embodiment of the present application can also be implemented as a program product, which includes a program code. When the program product can be run on an electronic device 300, the program code is used to enable the electronic device 300 to execute the smart vehicle positioning method in a dynamic scene based on a large model provided in an embodiment of the present application.

[0102] The program product provided in the embodiments of the present application may adopt any combination of one or more readable media, wherein the readable medium may be a readable signal medium or a readable storage medium, and the readable storage medium may be but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any combination of the above. Specifically, more specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, RAM, ROM, an erasable programmable read-only memory (EPROM), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0103] The program product provided in the embodiment of the present application may adopt a CD-ROM and include program code, and may also be run on a computing device. However, the program product provided in the embodiment of the present application is not limited thereto. In the embodiment of the present application, the readable storage medium may be any tangible medium containing or storing a program, and the program may be used by or in combination with an instruction execution system, apparatus, or device.

[0104] It should be noted that, although several units or subunits of the device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided into multiple units to be embodied.

[0105] In addition, although the operations of the method of the present application are described in a specific order in the drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0106] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present application and are not intended to limit it. Although the present application is described in detail with reference to the embodiments, a person skilled in the art should understand that any modification or equivalent replacement of the technical solution of the present application does not depart from the spirit and scope of the technical solution of the present application and should be included in the scope of the claims of the present application.

Claims

1. A method for positioning an intelligent vehicle in a dynamic scene based on a large model, characterized in that: include: Get the RGB image captured by the camera on the vehicle at the current moment; Use the SLAM front-end visual odometer to estimate the position of the current RGB image and obtain the first position of the vehicle at the current moment; Determine a dynamic area in the RGB image at the current moment based on the first position of the vehicle at the current moment and the first position of the vehicle at the previous moment, and the RGB image at the current moment and the RGB image at the previous moment; Remove the dynamic area in the RGB image at the current moment to obtain an RGB image containing only the static area of ​​the scene at the current moment; The SLAM front-end visual odometer is used to estimate the position of the RGB image that only contains the static area of ​​the scene at the current moment, and the second position of the vehicle at the current moment is obtained.

2. The method for positioning an intelligent vehicle in a dynamic scene based on a large model according to claim 1, characterized in that: Based on the first position of the vehicle at the current moment and the first position of the vehicle at the previous moment, and the RGB image at the current moment and the RGB image at the previous moment, a dynamic area in the RGB image at the current moment is determined; including: Based on the first position of the ego vehicle at the current moment and the first position of the ego vehicle at the previous moment, determining a full static hypothetical optical flow map between the RGB image at the current moment and the RGB image at the previous moment; The optical flow neural network is used to process the RGB image at the current moment and the RGB image at the previous moment to determine the actual scene optical flow map between the RGB image at the current moment and the RGB image at the previous moment; Compare the fully static hypothetical optical flow map with the actual scene optical flow map to determine the dynamic optical flow points and the positions of the dynamic object regions; Based on the dynamic optical flow points and the location of the dynamic object area, the segmentation model is used to segment the RGB image at the current moment to determine the dynamic object mask area; The dynamic object region position and the dynamic object mask region are fused to obtain the dynamic region in the RGB image.

3. The method for positioning an intelligent vehicle in a dynamic scene based on a large model according to claim 2 is characterized in that: Based on the first position of the vehicle at the current moment and the first position of the vehicle at the previous moment, a fully static hypothetical optical flow map between the RGB image at the current moment and the RGB image at the previous moment is determined; including: Determine an offset based on the first position of the vehicle at the current moment and the first position of the vehicle at the previous moment; The offset is used as the pixel value of each pixel of the full static hypothetical optical flow map to form the full static hypothetical optical flow map.

4. The method for positioning an intelligent vehicle in a dynamic scene based on a large model according to claim 3 is characterized in that: Comparing the fully static hypothetical optical flow map with the actual scene optical flow map to determine the location of the dynamic object area includes: Calculate the dynamic optical flow difference ΔF(p) of pixel p: ΔF(p)=F real (p)-F static (p) Among them, F static (p) is the pixel value of pixel p in the fully static hypothetical optical flow map, F real (p) is the pixel value of pixel p in the actual scene optical flow map; Determine whether the absolute value of the dynamic optical flow difference of pixel p is greater than a preset threshold, if yes, pixel p is a dynamic optical flow point; All dynamic optical flow points are clustered to form dynamic object areas and record their position information.

5. The method for positioning an intelligent vehicle in a dynamic scene based on a large model according to claim 4 is characterized in that: Based on the dynamic optical flow points and the location of the dynamic object area, the current RGB image is segmented using the segmentation model to determine the dynamic object mask area; including: The dynamic optical flow points are used as the points prompt input of the segmentation model, and the dynamic area position is used as the box prompt input of the segmentation model; the segmentation model is run to output the dynamic object mask area.

6. The method for positioning an intelligent vehicle in a dynamic scene based on a large model according to claim 5, characterized in that: The method further comprises: Obtain the pixel position of the dynamic object area at the current moment output by the segmentation model; Using the depth value of the camera and the pixel position of the dynamic object area at the current moment, determine the three-dimensional position of the dynamic object in the camera coordinate system; The three-dimensional position of the dynamic object in the geocentric rectangular coordinate system is determined using the second position of the vehicle at the current moment.

7. A smart car positioning device in a dynamic scene based on a large model, characterized in that: include: An acquisition unit, used to acquire an RGB image captured by a camera on the vehicle at the current moment; A first position estimation unit is used to estimate the position of the RGB image at the current moment by using the SLAM front-end visual odometer to obtain the first position of the vehicle at the current moment; A first processing unit, configured to determine a dynamic area in the RGB image at a current moment based on a first position of the vehicle at a current moment and a first position of the vehicle at a previous moment, and the RGB image at a current moment and the RGB image at a previous moment; The second processing unit is used to remove the dynamic area in the RGB image at the current moment, so as to obtain the RGB image containing only the static area of ​​the scene at the current moment; The second position estimation unit is used to use the SLAM front-end visual odometer to perform position estimation on the RGB image second position estimation unit that only contains the static area of ​​the scene at the current moment, and obtain the second position of the vehicle at the current moment.

8. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images