Vehicle control method, vehicle and electronic equipment

Abstract

The embodiment of the invention provides a vehicle control method, a vehicle and electronic device.The method comprises the steps that in response to a control instruction triggered by a terminal device for the vehicle, state information of a wireless signal and environment sensing information of the environment where the terminal device is located are obtained, the environment sensing information is used for representing the characteristics of target environment factors in the environment, and the target environment factors are sent to the terminal device; the target environment factor is used for influencing the accuracy degree of positioning the terminal equipment, and the fused map is obtained by fusing maps collected from a plurality of collection dimensions; determining a target position of the terminal device based on the state information and the environment perception information; determining a driving route between the position of the vehicle and the target position according to the fusion map of the vehicle; and according to the driving route, controlling the vehicle to drive from the position of the vehicle to the target position. The technical problem of low vehicle control flexibility is solved.

Description

Technical Field

[0001] This application relates to the field of vehicles, and more specifically, to a vehicle control method, a vehicle, and electronic equipment. Background Technology

[0002] Currently, remote intelligent summoning of vehicles by car owners typically involves summoning the vehicle through a fixed summoning point or by using the Global Positioning System (GPS).

[0003] The above method of summoning a vehicle through a fixed summoning point requires the car owner to stand at the fixed summoning point to summon the vehicle, and the vehicle will travel along a fixed route to the fixed summoning point.

[0004] The aforementioned method of summoning a vehicle using GPS location relies on a strong GPS signal and is difficult to use in scenarios without a GPS signal. Therefore, it still suffers from the technical problem of low flexibility in controlling the vehicle.

[0005] There is currently no good solution to the above problems. Summary of the Invention

[0006] This application provides a vehicle control method, a vehicle, and an electronic device to at least solve the technical problem of low flexibility in controlling a vehicle.

[0007] According to one aspect of the embodiments of this application, a vehicle control method is provided, wherein the vehicle and a terminal device communicate via a wireless signal. The method includes: responding to a control command triggered by the terminal device on the vehicle; acquiring state information of the wireless signal and environmental perception information of the environment in which the terminal device is located, wherein the environmental perception information is used to represent the characteristics of target environmental factors in the environment, the target environmental factors are used to affect the accuracy of positioning the terminal device, and the fused map is obtained by fusing maps collected from multiple acquisition dimensions; determining the target location of the terminal device based on the state information and the environmental perception information; determining the driving route between the vehicle's location and the target location according to the vehicle's fused map; and controlling the vehicle to travel from its location to the target location according to the driving route.

[0008] Further, based on state information and environmental perception information, the target location of the terminal device is determined, including: determining a first location of the terminal device based on state information and a fused map, wherein the first location is used to represent the location of the terminal device determined by the state information; determining a second location of the terminal device based on environmental perception information, wherein the second location is used to represent the location of the terminal device determined by the environmental perception information; and determining the target location based on the first location and the second location.

[0009] Furthermore, the method also includes: acquiring the posture information of the terminal device, wherein the posture information is used to represent the motion state of the terminal device; determining the first position of the terminal device based on the state information and the fused map, including: combining the state information and the posture information to locate the wireless signal from the fused map to obtain the first position; or, the environmental perception information includes environmental semantic information and an image, wherein the environmental semantic information is used to represent the attributes of environmental features, and the image is obtained by collecting environmental features, and the second position includes a first sub-position and a second sub-position; determining the second position of the terminal device based on the environmental perception information, including: determining the first sub-position based on the environmental semantic information and the posture information; extracting the attribute features of target feature points from the image, and determining the second sub-position based on the attribute features and the posture information, wherein the target feature points are feature points in the image with a recognition degree greater than a recognition degree threshold and used to improve the positioning accuracy of the terminal device.

[0010] Further, determining the target location based on the first location and the second location includes: determining the observation noise covariance of the first location, the first sub-location, and the second sub-location, respectively, wherein the observation noise covariance is used to represent the degree of difference between the first location, the first sub-location, or the second sub-location and the actual location of the terminal device; and determining the target location based on the observation noise covariance corresponding to the first location, the first sub-location, and the second sub-location, respectively.

[0011] Further, determining the target location based on the observation noise covariances corresponding to the first location, the first sub-location, and the second sub-location includes: in response to the existence of an observation noise covariance less than or equal to a covariance threshold among the observation noise covariances corresponding to the first location, the first sub-location, and the second sub-location, determining the target location based on the location where the observation noise covariance is less than or equal to the covariance threshold; in response to the existence of at least two observation noise covariances less than or equal to the covariance threshold among the observation noise covariances corresponding to the first location, the first sub-location, and the second sub-location, fusing the locations where the observation noise covariance is less than or equal to the covariance threshold to obtain the target location; in response to the existence of no observation noise covariance less than or equal to the covariance threshold among the observation noise covariances corresponding to the first location, the first sub-location, and the second sub-location, determining the target location based on the selection instruction for the candidate location on the terminal device, wherein the candidate location is a location on the fused map that can be controlled to drive the vehicle to.

[0012] Further, according to the driving route, controlling the vehicle to travel from its current location to the target location includes: projecting the target location onto the driving route; using the target location on the driving route as the endpoint and the vehicle's current location as the starting point; controlling the vehicle to travel from the starting point to the endpoint; or, if the vehicle is connected to the cloud, the method further includes: responding to the control command to obtain the fused map from the cloud; the map collected from multiple acquisition dimensions includes at least two of the following: a wireless signal map, a semantic map, and a visual feature map; the method further includes: responding to the cloud receiving the wireless signal map, the semantic map, and the visual feature map sent by the vehicle, using the cloud to process the wireless signal map, the semantic map, and the visual feature map. Figure 3 At least two of them are fused to obtain the fused map.

[0013] Furthermore, in response to receiving wireless signal maps, semantic maps, and visual feature maps sent by vehicles in the cloud, the cloud is used to analyze the wireless signal maps, semantic maps, and visual feature maps. Figure 3 At least two of these are fused to obtain a fused map, including: a map of wireless signals, a semantic map, and a visual feature map received from the cloud from the vehicle; and, in the time dimension, the cloud-based processing of the wireless signal map, semantic map, and visual feature map. Figure 3 At least two of them are aligned to obtain a time alignment result; in the spatial dimension, the coordinate system of the visual feature map and the coordinate system of the wireless signal map are converted into the coordinate system of the semantic map to obtain a spatial alignment result; based on the time alignment result and the spatial alignment result, a fused map is obtained.

[0014] Furthermore, the method also includes: in response to the vehicle being in manual driving mode and the vehicle passing through the driving route in the fused map, collecting state information and environmental perception data of at least one trajectory point on the driving route; fusing the collected state information and environmental perception data into the fused map to obtain a target fused map, wherein the target fused map is used to replace the fused map stored in the cloud connected to the vehicle; or, determining the driving route between the vehicle's location and the target location according to the vehicle's fused map, including: obtaining the target fused map from the cloud and determining the driving route according to the target fused map.

[0015] Furthermore, based on the vehicle's fused map, the driving route between the vehicle's current location and the target location is determined, including: obtaining the target fused map from the cloud, and determining the driving route based on the target fused map.

[0016] According to another aspect of the embodiments of this application, a vehicle control device is also provided. The device may include: an acquisition unit, which, in response to a control command triggered by a terminal device on a vehicle, acquires wireless signal status information and environmental perception information of the environment in which the terminal device is located; a first determination unit, which determines the target location of the terminal device based on the status information and the environmental perception information; a second determination unit, which determines the driving route between the vehicle's current location and the target location according to a fused map of the vehicle; and a control unit, which controls the vehicle to drive from its current location to the target location according to the driving route.

[0017] According to another aspect of the embodiments of this application, a vehicle is also provided, including: a memory storing an executable program; and a processor for running the program, wherein the program executes the methods in various embodiments of this application when it runs.

[0018] According to another aspect of the embodiments of this application, an electronic device is also provided, including: a memory storing an executable program; and a processor for running the program, wherein the program executes the methods in various embodiments of this application when it runs.

[0019] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored executable program, wherein, when the executable program is running, it controls the device where the computer-readable storage medium is located to perform the methods of various embodiments of this application.

[0020] According to another aspect of the embodiments of this application, a computer program product is also provided, including a computer program that, when executed by a processor, implements the methods of various embodiments of this application.

[0021] According to another aspect of the embodiments of this application, a computer program product is also provided, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the methods in various embodiments of this application.

[0022] According to another aspect of the embodiments of this application, a computer program is also provided, which, when executed by a processor, implements the methods of the various embodiments of this application.

[0023] In this embodiment, in response to a control command triggered by a terminal device on a vehicle, the system acquires wireless signal status information and environmental perception information of the terminal device's environment. The environmental perception information represents the characteristics of target environmental factors in the environment, which affect the accuracy of positioning the terminal device. The fused map is obtained by fusing maps collected from multiple dimensions. Based on the status information and environmental perception information, the target location of the terminal device is determined. According to the fused map, the driving route between the vehicle's current location and the target location is determined. Following the driving route, the vehicle is controlled to travel from its current location to the target location. In other words, in this embodiment, the system acquires wireless signal status information and environmental perception information of the terminal device's environment. By combining these two information, the target location of the terminal device is determined. A fused map is obtained by fusing maps collected from multiple dimensions, and then the driving route between the vehicle's current location and the target location of the terminal device is determined using the fused map. This method enables high-precision positioning of the terminal device even in environments with poor or absent GPS signals, making vehicle summoning unrestricted by signal coverage and enhancing the flexibility and range of vehicle control. Therefore, the above method overcomes the limitation of low positioning accuracy in environments with poor or absent GPS signals in related technologies, thereby solving the technical problem of low vehicle control flexibility and achieving the technical effect of improving vehicle control flexibility. Attached Figure Description

[0024] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0025] Figure 1 This is a flowchart of a vehicle control method according to an embodiment of this application;

[0026] Figure 2 This is a schematic diagram of a parking lot intelligent vehicle summoning architecture based on multi-mode fusion positioning according to an embodiment of this application;

[0027] Figure 3 This is a flowchart of a multi-mode localization fusion method according to an embodiment of this application;

[0028] Figure 4 This is a schematic diagram of a vehicle summoning process according to an embodiment of this application;

[0029] Figure 5 This is a schematic diagram of a vehicle control device according to an embodiment of this application. Detailed Implementation

[0030] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present application.

[0031] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0032] According to an embodiment of this application, an embodiment of a vehicle information processing method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0033] This embodiment provides a vehicle information processing method. Figure 1 This is a flowchart of a vehicle control method according to an embodiment of this application, such as... Figure 1 As shown, the vehicle and the terminal device communicate via wireless signals, and the method may include the following steps.

[0034] Step S102: Respond to the control command triggered by the terminal device on the vehicle, and obtain the status information of the wireless signal and the environmental perception information of the environment in which the terminal device is located.

[0035] In the technical solution provided by step S102 in the embodiments of this application, the environmental perception information is used to represent the characteristics of the target environmental factors in the environment, and the target environmental factors are used to affect the accuracy of the positioning terminal device.

[0036] Optionally, the aforementioned terminal device can be a smart device, such as a smartphone, smartwatch, or other device with wireless communication capabilities. The aforementioned control commands can be used to represent instructions issued by the terminal device to instruct the vehicle to perform specific actions, such as a command to remotely summon the vehicle.

[0037] In this embodiment, if the vehicle receives the control command triggered by the terminal device, it can obtain the status information of the wireless signal of the terminal device and the environmental perception information of the environment in which the terminal device is located. The status information can represent the quality and status of various wireless signals that the terminal device can currently detect, such as the quality and status of the wireless signals used for communication between the vehicle and the terminal device. These wireless signals can be wireless LAN signals or Bluetooth signals. The status of the wireless signals may include, but is not limited to, the name and strength of the wireless signals.

[0038] Optionally, the aforementioned environmental perception information can be used to represent the characteristics of target environmental factors in the environment. For example, the target environmental factors can be items such as parking spaces, pillars, decorations, and signs in the environment where the mobile device is located. Correspondingly, the characteristics of the target environmental factors can include the layout characteristics of parking spaces, the structural characteristics of pillars, and the appearance characteristics of specific decorations or signs. The aforementioned environmental perception information can include environmental semantic information and image information. The aforementioned environmental semantic information can be used to represent the type, shape, and encoding characteristics of target environmental factors in the environment. For example, the aforementioned environmental semantic information can be the shape, size, and encoding of parking spaces in the environment where the mobile device is located, the position and shape of pillars, and identifiers on walls. The aforementioned image information can be used to represent the view of the current environment captured by the terminal device's camera. For example, the image content of the aforementioned image information can include parking space frames, billboards, ground markings, etc., in the environment captured by the camera. Target environmental factors are used to influence the accuracy of the positioning terminal device.

[0039] Optionally, when the vehicle receives a control command triggered by the terminal device, the vehicle responds to the control command by continuously monitoring the signal strength and signal quality of the wireless signals (Wi-Fi signals or Bluetooth signals) in the surrounding environment of the terminal device through the wireless module on the terminal device.

[0040] Optionally, environmental perception can be achieved using a camera or other sensors (such as a gyroscope or accelerometer) on the terminal device to obtain environmental perception information. For example, the terminal device's camera can identify image data containing environmental features such as parking space frames, pillars, and parking space numbers in the surrounding environment. Specific parameters of target environmental factors can also be identified from the image data. For instance, specific parameters of the parking space frame, such as the location coordinates and size of each parking space, can be identified from the image data.

[0041] In this embodiment of the application, by responding to the instructions issued by the terminal device in step S102, the wireless signal status information and environmental awareness information of the terminal device can be collected and analyzed.

[0042] Step S104: Determine the target location of the terminal device based on the status information and environmental perception information.

[0043] In the technical solution provided by step S104 in the embodiments of this application, the target location can be used to indicate the location of the terminal device.

[0044] In this embodiment, if the status information of the wireless signal and the environmental perception information of the environment in which the terminal device is located are obtained, the target location of the terminal device can be determined based on the status information and the environmental perception information.

[0045] Optionally, based on state information and environmental perception information, the target location of the terminal device can be determined by comprehensively analyzing the signal strength and quality of wireless signals such as Wireless Fidelity (Wi-Fi) and Bluetooth, combined with environmental images captured by the camera of the terminal device (e.g., a smartphone). The target location may include, but is not limited to, the location of the terminal device, the location of the vehicle owner, or the projection point of the vehicle owner's location onto the vehicle's driving trajectory.

[0046] In this embodiment of the application, the above step S104 can combine the strength and quality of Wi-Fi and Bluetooth signals, as well as the image captured by the camera, to overcome the limitations of locating the location of the terminal device based on a single piece of information.

[0047] Step S106: Determine the driving route between the vehicle's current location and the target location based on the vehicle's fused map.

[0048] In the technical solution provided by step S106 of the embodiments of this application, the fused map is obtained by fusing maps collected from multiple acquisition dimensions. This fused map can be used to represent a map that incorporates various types of information. The aforementioned acquisition dimensions can be used to represent different aspects or types of collected information, such as the strength of wireless signals, visual features, and types of semantic elements.

[0049] In this embodiment, if the target location of the terminal device is determined, a driving route between the vehicle's current location and the target location can be determined based on the vehicle's fused map. This driving route can represent a feasible path planned based on the fused map between the vehicle's current location and the target location, guiding the vehicle on how to travel from its current location to the user's target location.

[0050] Optionally, a wireless signal fingerprint map can be constructed by collecting information such as the names and strengths of wireless signals like Wi-Fi and Bluetooth signals within the parking lot; images of the parking lot can be captured by cameras, and key visual features can be extracted from these images using image recognition technology to construct a visual feature layer map; important landmark information within the parking lot, such as parking spaces, pillars, and entrances / exits, can be recorded to form a semantic map of routes. The semantic map, the wireless signal map, and the visual feature map can then be fused to form a three-dimensional composite map, where each point in the three-dimensional composite map is associated with static semantic information, real-time signal strength, and visual feature data.

[0051] For example, when a vehicle is learning a route, it can build a semantic map using the semantic mapping module on the vehicle, collect wireless signals along the route using the wireless signal mapping module on the vehicle, and collect images along the route using the visual feature mapping module on the vehicle, thus building a visual feature map. The semantic map, wireless signal fingerprint map, and visual feature map can be uploaded to the map fusion module. The fusion module then combines these maps to obtain a semantic-feature-wireless signal fused map of the route.

[0052] Optionally, the precise location of the vehicle in the parking lot can be determined by using onboard sensors and wireless signals and visual feature information from the fused map; the target location of the terminal device can be determined based on the above method; and a path can be planned based on the target location of the terminal device to determine the driving route between the vehicle's current location and the target location.

[0053] In this embodiment of the application, the wireless signal fingerprint map, visual feature layer map and semantic map can be fused together through the above step S106 to form a three-dimensional composite map, namely a fused map, which provides the vehicle with rich environmental description data, enhances the vehicle's navigation and positioning capabilities in complex environments, and plans a feasible path from the current vehicle position to the target position by analyzing the information of the fused map.

[0054] Step S108: According to the driving route, control the vehicle to drive from its current location to the target location.

[0055] In the technical solution provided by step S108 in the embodiments of this application, if the driving route is determined, the vehicle can be controlled to travel from its current location to the target location.

[0056] Optionally, the planned driving route is broken down into a series of executable tasks, such as acceleration, deceleration, and steering commands, which are transmitted to the vehicle's autonomous driving control system. Using onboard LiDAR, cameras, ultrasonic sensors, and other sensors, the system continuously monitors changes in the vehicle's surrounding environment to ensure the real-time accuracy and safety of the driving route. Based on the needs of the driving route, the system dynamically adjusts the vehicle's speed, acceleration, and steering angle. After the vehicle begins to drive automatically according to the planned driving route and decision commands and reaches the target location, it can safely stop, completing the summoning task.

[0057] In this embodiment, steps S102 to S108 can respond to a control command triggered by the terminal device on the vehicle, acquire wireless signal status information and environmental perception information of the terminal device's environment. The environmental perception information represents the characteristics of target environmental factors in the environment, which affect the accuracy of the positioning of the terminal device. The fused map is obtained by fusing maps collected from multiple dimensions. Based on the status information and environmental perception information, the target location of the terminal device is determined. According to the fused map of the vehicle, the driving route between the vehicle's current location and the target location is determined. Following the driving route, the vehicle is controlled to travel from its current location to the target location. In other words, in this embodiment, the status information of the wireless signal and the environmental perception information of the terminal device's environment are acquired; by fusing the aforementioned status information and environmental perception information, the target location of the terminal device is determined; and the driving route between the vehicle's current location and the target location is determined using the fused map. The above method overcomes the limitation of low positioning accuracy in environments with poor or absent GPS signals in related technologies. It achieves the goal of determining the terminal position by integrating status information, environmental perception information, and fused maps, and then planning the driving route from the vehicle's current position to the target position, ultimately controlling the vehicle to automatically drive to the target position. This solves the technical problem of low vehicle control flexibility and achieves the technical effect of improving vehicle control flexibility.

[0058] The embodiments of this application will be described in detail below with reference to the steps described above.

[0059] As an optional implementation, determining the target location of the terminal device based on state information and environmental perception information includes: determining a first location of the terminal device based on state information and a fused map, wherein the first location is used to represent the location of the terminal device determined by the state information; determining a second location of the terminal device based on environmental perception information, wherein the second location is used to represent the location of the terminal device determined by the environmental perception information; and determining a target location based on the first location and the second location.

[0060] In this embodiment, the first location can be used to indicate the location of the terminal device determined by the status information; the second location information can be used to indicate the location of the terminal device determined by the environmental perception information.

[0061] Optionally, after determining the wireless signal status information, environmental perception information and obtaining the above-mentioned fused map, the first location and the second location of the terminal device can be determined, and the target location can be determined based on the first location and the second location.

[0062] Optionally, the names and strengths of wireless signals such as Wi-Fi hotspots and Bluetooth devices in the parking lot are received and analyzed by a terminal device (e.g., a smartphone). Combined with the wireless signal fingerprint map in the fused map, a positioning algorithm (e.g., trilateration) is used to determine the initial position. The attitude data in the inertial measurement unit (IMU) built into the terminal device is used to correct the initial position to obtain the first position.

[0063] Optionally, an environmental image of the current location is captured by the camera of the terminal device; key visual features in the image are extracted using image recognition technology, such as the SuperPoint algorithm; and the extracted key feature information is matched with the visual feature map in the fused map to determine the second location.

[0064] Optionally, the target location can be determined by fusing the first location (wireless signal positioning result) with the second location (positioning result based on environmental perception information) using weighted average, least squares method or other fusion algorithms.

[0065] In this embodiment of the application, the first and second locations of the wireless signal positioning are determined by the wireless signal reception and analysis of the terminal device and the image recognition technology of the terminal device through the above method. By combining the information of the first and second locations through the data fusion algorithm, a more accurate target location is determined.

[0066] As an optional implementation, the method further includes: acquiring attitude information of the terminal device, wherein the attitude information is used to represent the motion state of the terminal device; determining the first position of the terminal device based on the state information and the fused map, including: combining the state information and the attitude information to locate the wireless signal from the fused map to obtain the first position.

[0067] In this embodiment, the aforementioned attitude information can be used to represent the motion state of the terminal device, such as the dynamic changes of the terminal device captured by the gyroscope and accelerometer in the IMU built into the terminal device.

[0068] Optionally, after determining the status information of the wireless signal and obtaining the fused map, the first location of the terminal device can be determined.

[0069] Optionally, the terminal device's acceleration, angular velocity, and magnetic field direction are continuously measured by the IMU built into the terminal device. Based on the information obtained from the above measurements, the attitude change of the terminal device, i.e., attitude information, is further calculated.

[0070] Optionally, the state information of the wireless signal received from the terminal device is analyzed to preliminarily estimate the preliminary position information of the terminal device; the preliminary position information determined by the wireless signal state information is combined with the attitude information, and a fusion algorithm (such as Kalman filtering, particle filtering, etc.) is used to calculate and determine the first position.

[0071] In this embodiment, the above method can determine the device's attitude information based on continuous monitoring of acceleration, angular velocity, and magnetic field direction by the terminal device's built-in IMU. A precise first position is determined by combining the wireless signal's state information with the attitude information. This achieves the goal of ensuring high-precision positioning even in various complex scenarios, such as underground parking lots, where the terminal device moves rapidly or changes direction.

[0072] As an optional implementation, the method further includes: acquiring posture information of the terminal device, wherein the posture information is used to represent the motion state of the terminal device; environmental perception information includes environmental semantic information and an image, wherein the environmental semantic information is used to represent the attributes of environmental features, the image is obtained by collecting environmental features, the second position includes a first sub-position and a second sub-position, and determining the second position of the terminal device based on the environmental perception information, including: determining the first sub-position based on the environmental semantic information and posture information; extracting attribute features of target feature points from the image, and determining the second sub-position based on the attribute features and posture information, wherein the target feature points are feature points in the image with a recognition degree greater than a recognition degree threshold and used to improve the positioning accuracy of the terminal device.

[0073] In this embodiment, the image can be obtained by collecting environmental features. The first sub-location can be used to represent the result of semantic map-based localization. The second sub-location can be used to represent the result of feature map-based localization. The target feature point can be a feature point in the image with a recognition degree greater than a recognition degree threshold, used to improve the localization accuracy of the terminal device. In this embodiment, the target feature point can be a feature point of each image along the route collected by the visual feature mapping module on the vehicle side, i.e., a SuperPoint. The attribute feature can be used to represent the unique identifier of the target feature point. The recognition degree threshold can be used to represent the standard for selecting target feature points. In this embodiment, the attribute feature can be a SuperPoint feature.

[0074] Optionally, after determining the environmental perception information, the second location of the terminal device can be determined.

[0075] Optionally, environmental semantic information is extracted from the environmental perception information, such as the location of static environmental elements like the corner coordinates and number of the parking space; acceleration, angular velocity, and magnetic field direction information are continuously acquired through the IMU built into the terminal device to calculate the real-time attitude change of the terminal device; the extracted environmental semantic information is combined with the attitude information, and the semantic localization algorithm is used to match it with the semantic map in the fused map to determine the first sub-location.

[0076] Optionally, the camera of the terminal device is used to capture environmental images, and image feature detection, such as SuperPoint feature detector, is used to detect and extract the attribute features of the target feature points, i.e., SuperPoint features, from the environmental images. Based on the extracted SuperPoint features and pose information, the second sub-position is determined by a fusion algorithm, such as Kalman filtering algorithm.

[0077] In this embodiment, the above method enables high-precision, multi-dimensional positioning of the terminal device, effectively improving the accuracy and reliability of positioning. Based on the first sub-position determined by environmental semantic information and pose information, and combined with the second sub-position determined by SuperPoint features extracted from the image and pose information, cross-validation and fusion of positioning information are achieved. This ensures accurate estimation of the terminal device's position even in complex, dynamic, or poorly signaled environments.

[0078] As an optional implementation, determining the target location based on the first location and the second location includes: determining the observation noise covariance of the first location, the first sub-location, and the second sub-location, respectively, wherein the observation noise covariance is used to represent the degree of difference between the first location, the first sub-location, or the second sub-location and the actual location of the terminal device; and determining the target location based on the observation noise covariance corresponding to the first location, the first sub-location, and the second sub-location, respectively.

[0079] In this embodiment, the aforementioned observation noise covariance can be used to represent the degree of difference between the first position, the first sub-position, or the second sub-position and the actual position of the terminal device.

[0080] Optionally, after determining the first position and the second position, the target position can be determined based on the first position and the second position.

[0081] Optionally, based on the name and strength of the wireless signal received by the mobile application, combined with the mobile phone IMU information, the wireless signal is located using a semantic-feature-wireless signal fusion map. The obtained wireless signal location result and observation noise covariance can be calculated using the following formula.

[0082]

[0083]

[0084] in, It can be used to represent the location result obtained by locating a terminal device via wireless signals (S), that is, the first location. Used to represent x Observation noise variance along coordinate directions Used to represent the variance of observation noise in the y-coordinate direction. Used to represent the observation noise covariance at the first location. Used to represent the horizontal coordinate position of the terminal device calculated after positioning based on the received wireless signals. It is used to represent the horizontal coordinate position of the terminal device calculated after positioning based on the received wireless signal.

[0085] Optionally, based on the image recognition of the mobile phone camera, the environmental semantic information is obtained, and the semantic localization algorithm is used to tightly couple and fuse the environmental semantic information with the IMU to obtain the semantic map localization result. The observation noise covariance based on the semantic map localization result can be calculated by the following formula.

[0086]

[0087]

[0088] in, This is used to represent the semantic map localization result obtained by locating the terminal device using environmental semantic (Semantics, abbreviated as s) information, also known as the second sub-location. The observation noise covariance used to represent the first sub-position. Used to represent x Observation noise variance along coordinate directions Used to represent the variance of observation noise in the y-coordinate direction. Used to represent the horizontal coordinate position of the terminal device calculated after semantic positioning. Used to represent the vertical coordinate position of the terminal device calculated after semantic positioning.

[0089] Optionally, SuperPoint features are extracted from images captured by the mobile phone camera, and the feature localization algorithm is used to tightly couple and fuse the feature information with the IMU to obtain the feature map localization result. The observation noise covariance based on the feature map localization result can be calculated by the following formula.

[0090]

[0091]

[0092] in, This is used to represent the feature map localization result obtained by locating the terminal device using features (f) extracted from images, also known as the second sub-location. The observation noise covariance used to represent the second sub-position. Used to represent x Observation noise covariance along coordinate directions Used to represent the observation noise covariance in the y-coordinate direction. Used to represent the horizontal coordinate position of the terminal device calculated after positioning based on features. Used to represent the vertical coordinate position of the terminal device calculated after positioning based on features.

[0093] Optionally, based on the observation noise covariance of the first position, the first sub-position, and the second sub-position, the target position is determined by fusing the first position, the first sub-position, and the second sub-position with their corresponding observation noise covariance using a multi-sensor data fusion algorithm (e.g., Kalman filtering).

[0094] In the embodiments of this application, the above method can integrate three different sources of positioning information, namely, a first location based on wireless signals, a first sub-location based on environmental semantic information, and a second sub-location based on SuperPoint features, and determine the most accurate target location by evaluating the observation noise covariance of each.

[0095] As an optional implementation, the target location is determined based on the observation noise covariances corresponding to the first location, the first sub-location, and the second sub-location, respectively. This includes: in response to the existence of an observation noise covariance less than or equal to a covariance threshold among the observation noise covariances corresponding to the first location, the first sub-location, and the second sub-location, determining the target location based on the location where the observation noise covariance is less than or equal to the covariance threshold; in response to the existence of at least two observation noise covariances less than or equal to the covariance threshold among the observation noise covariances corresponding to the first location, the first sub-location, and the second sub-location, fusing the locations where the observation noise covariance is less than or equal to the covariance threshold to obtain the target location; and in response to the existence of no observation noise covariance less than or equal to the covariance threshold among the observation noise covariances corresponding to the first location, the first sub-location, and the second sub-location, determining the target location based on the selection instruction for candidate locations on the terminal device, wherein the candidate locations are locations on the fused map that can be controlled to be driven to.

[0096] In this embodiment, the aforementioned covariance threshold can be used as a standard to represent the reliability of the positioning result, and can be a pre-set distance value, such as 5 meters. The aforementioned candidate locations can be locations on the fused map that can control the vehicle's movement to.

[0097] Optionally, after determining the observation noise covariances corresponding to the first position, the first sub-position, and the second sub-position, the target position can be determined.

[0098] Optionally, when the observation noise covariance of only one of the first position, the first sub-position, or the second sub-position is less than or equal to a preset covariance threshold, the strategy for determining the target position based on the position information may include: comparing the observation noise covariance of the first position, the first sub-position, and the second sub-position with the covariance threshold, finding the positioning result that meets the condition that the observation noise covariance is less than or equal to the covariance threshold, and selecting the above positioning result as the target position.

[0099] Optionally, when the observation noise covariance of two of the first position, first sub-position, or second sub-position is less than or equal to a preset covariance threshold, the observation noise covariance of the first position, first sub-position, and second sub-position can be compared with the covariance threshold to find the positioning result that meets the condition that the observation noise covariance is less than or equal to the covariance threshold. The selected position information and the corresponding observation noise covariance can be fused using a Kalman filter algorithm. For each selected position information, the corresponding Kalman gain can be calculated using the following formula.

[0100]

[0101] Among them, Pk Used to represent the state covariance matrix This is used to represent the mapping matrix between states and measurements; for location information, it is usually an identity matrix. Used to represent the observation noise covariance matrix. The above It can be expressed by the following formula.

[0102]

[0103] Where I is used to represent the identity matrix.

[0104] Alternatively, state prediction can be expressed by the following formula.

[0105]

[0106] in, This represents the state vector estimated at time k based on information prior to time k (including time k-1).

[0107] Alternatively, covariance prediction can be expressed by the following formula.

[0108]

[0109] Among them, P k The state estimation error covariance matrix at time k is used. Q represents the covariance matrix of the system process noise. The process noise covariance can be expressed by the following formula.

[0110]

[0111] Optionally, Kalman gain is applied to update the state estimate and state covariance matrix to ensure that the fused result is as close as possible to the true position. The state estimate update for each position can be expressed by the following formula.

[0112]

[0113]

[0114] Among them, X K Used to represent the state vector. K k Let Z be the Kalman gain matrix of the Kalman filter at time k. k Used to represent the observation vector. P k Used to represent the state covariance matrix. I is used to represent the identity matrix.

[0115] Optionally, when the observation noise covariance of the first location, the first sub-location, or the second sub-location is less than or equal to a preset covariance threshold, the uncertainty of the location information is confirmed to exceed an acceptable range by comparing the observation noise covariance of each location information with the preset threshold. If it does, a prompt can be issued to the user on the terminal device, informing them that the current location information may be inaccurate, and a series of candidate locations can be generated or suggested for the user. The candidate locations can be based on previous reliable location information, historical user behavior, parking lot layout, and any available auxiliary information (such as map data, the last known location of the vehicle, etc.). The user can view and select (i.e. issue a selection command to the terminal device) a candidate location as the target location on the terminal device. That is, the target location is determined based on the selection command for the candidate location on the terminal device.

[0116] In the embodiments of this application, the above method can achieve flexible determination and high-precision positioning of the vehicle summoning target location. Even when the information quality of one or more modes in the multi-mode positioning system is poor, reasonable decisions can still be made based on existing information and user participation.

[0117] As an optional implementation, controlling the vehicle to travel from its current location to a target location according to the driving route includes: projecting the target location onto the driving route; using the target location on the driving route as the endpoint and the vehicle's current location as the starting point; and controlling the vehicle to travel from the starting point to the endpoint.

[0118] In this embodiment, the driving route can be a pre-learned and stored path, which can be automatically learned and constructed while the vehicle is driving.

[0119] Optionally, the coordinates of the target location in three-dimensional space and the coordinate system of the aforementioned driving route are determined, and the coordinates of the target location are transformed to the same coordinate system as the driving route to ensure the accuracy of the projection operation. The route point closest to the target location, i.e., the projection point, can be used as the destination location for vehicle summoning.

[0120] Based on the vehicle's current position and the projected destination, path planning is performed to generate a driving route from the starting point to the destination. The vehicle automatically controls steering, acceleration, and braking according to the planned route and obstacle avoidance strategy, thus achieving autonomous driving from the starting point to the destination.

[0121] For example, the car owner's current location can be forwarded to the vehicle's intelligent driving domain via a cloud location service module through a mobile application. The intelligent driving domain then projects the car owner's location onto a memory trajectory, using the projected point as the destination, and controls the vehicle to automatically drive to the destination, completing the vehicle summoning process.

[0122] In this embodiment of the application, the above method realizes intelligent vehicle summoning based on multi-mode positioning, which can not only effectively solve the positioning problem in complex environments such as underground parking lots, but also improve the flexibility and accuracy of vehicle summoning.

[0123] As an optional implementation, the vehicle is connected to the cloud, and the method further includes: responding to control commands to acquire a fused map from the cloud; the map acquired from multiple acquisition dimensions includes at least two of the following: a wireless signal map, a semantic map, and a visual feature map; the method further includes: responding to receiving the wireless signal map, semantic map, and visual feature map sent by the vehicle from the cloud, utilizing the cloud to process the wireless signal map, semantic map, and visual feature map... Figure 3 At least two of them must be merged to obtain a merged map.

[0124] In this embodiment, the cloud can be a centralized data processing and service center.

[0125] Optionally, in response to receiving the wireless signal map, semantic map, and visual feature map sent by the vehicle in the cloud, the cloud is used to process the wireless signal map, semantic map, and visual feature map. Figure 3 The fusion of at least two of these to obtain a fused map may include: time synchronization of wireless signals, semantic elements and visual feature data streams received from vehicles to ensure consistency of different types of positioning information on the time axis and avoid fusion errors caused by time errors; and conversion of the aforementioned wireless signals, semantic elements and visual feature data streams to a unified coordinate system.

[0126] Optionally, on the visual feature map, the cloud can use feature extraction algorithms such as SuperPoint to identify key points in the image and match them with semantic elements (such as parking spaces and pillars) in the semantic map and signal sources in the wireless signal map to establish connections between different maps.

[0127] Optionally, a fusion algorithm, such as Kalman filtering, can be used to fuse information from the wireless signal map, semantic map, and visual feature map. After the above information fusion is completed, the cloud can generate a fused map that includes vehicle location, environmental semantic elements, and the distribution of wireless signal sources.

[0128] In this embodiment of the application, the above method can effectively integrate multimodal information from the vehicle to generate a more accurate and comprehensive fusion map, thereby improving the accuracy and reliability of vehicle intelligent summoning.

[0129] As an optional implementation, in response to receiving wireless signal maps, semantic maps, and visual feature maps sent by the vehicle in the cloud, the cloud is used to process the wireless signal maps, semantic maps, and visual feature maps. Figure 3 At least two of these are fused to obtain a fused map, including: a map of wireless signals, a semantic map, and a visual feature map received from the cloud from the vehicle; and, in the time dimension, the cloud-based processing of the wireless signal map, semantic map, and visual feature map. Figure 3 At least two of them are aligned to obtain a time alignment result; in the spatial dimension, the coordinate system of the visual feature map and the coordinate system of the wireless signal map are converted into the coordinate system of the semantic map to obtain a spatial alignment result; based on the time alignment result and the spatial alignment result, a fused map is obtained.

[0130] In this embodiment, the aforementioned time alignment results can be used to represent the precise correspondence between various map data on the time axis. The aforementioned spatial alignment results can be used to represent the correct matching of different map information in geographic space.

[0131] Optionally, the timestamp of each map data point can be checked. If timestamps are inconsistent, interpolation methods can be used for correction to ensure that various types of map data are within the same time frame. The data acquisition frequency or processing frequency can be adjusted to match the update rates of wireless signal data, semantic information, and visual feature data, reducing information lag caused by differences in data rates. Minor errors that may occur during time alignment can be compensated for based on hardware characteristics or software calculations to further improve the accuracy of time alignment.

[0132] Optionally, the original coordinate system of each map is identified, including the coordinate reference points and scales of the visual feature map, the wireless signal map, and the semantic map. Using a coordinate transformation matrix, the coordinates of the visual feature map and the wireless signal map are transformed from their respective original coordinate systems to the coordinate system of the semantic map, ensuring that the map data are within the same spatial frame. Under the transformed coordinate system, the spatial correlation between visual features and wireless signal sources and semantic elements (such as parking spaces and pillars) is found. More refined spatial alignment can be achieved through feature point registration methods.

[0133] Optionally, Kalman filtering, particle filtering, or other fusion algorithms are applied to integrate the time-aligned and spatially aligned wireless signal information, semantic information, and visual feature information to form a fused map. In this embodiment, the fusion algorithm is Kalman filtering.

[0134] In the embodiments of this application, the above method can effectively process multimodal positioning data and generate a high-precision fused map, thereby providing a more reliable location service for vehicle summoning.

[0135] As an optional implementation, the method further includes: in response to the vehicle being in manual driving mode and the vehicle passing through the driving route in the fused map, collecting state information and environmental perception data of at least one trajectory point on the driving route; fusing the collected state information and environmental perception data into the fused map to obtain a target fused map, wherein the target fused map is used to replace the fused map stored in the cloud connected to the vehicle.

[0136] In this embodiment, the aforementioned manual driving mode can be a driving state where the vehicle is directly controlled by the driver, rather than by an autonomous driving system. The aforementioned target fusion map can be a multi-modal localization map that integrates the latest status information and environmental perception data.

[0137] Optionally, the vehicle's wireless signal acquisition module records the name, strength, and location information of signal sources such as Wi-Fi and Bluetooth when passing through trajectory points, and can update the wireless signal fingerprint map based on the information collected above.

[0138] Optionally, the vehicle can collect the latest information on semantic elements such as parking spaces and pillars along the driving route through onboard cameras and sensors, including changes in location, additions or removals of elements, in order to update the semantic map.

[0139] Optionally, the environmental images captured by the vehicle's camera are processed by a visual feature extraction algorithm to identify SuperPoint features or other visual key points in order to update the visual feature map.

[0140] Optionally, the collected status information and environmental perception data can be uploaded to the cloud to provide a data foundation for updating the fused map.

[0141] Optionally, in the cloud, the uploaded status information and environmental perception data are preprocessed, and new wireless signal maps, semantic maps, and visual feature maps are constructed based on the preprocessed status information and environmental perception data. These wireless signal maps, semantic maps, and visual feature maps are then aligned with the existing fused map in time and space to ensure information matching and consistency. Based on the aligned data, the wireless signal, semantic elements, and visual feature information in the fused map are updated to reflect the latest environmental status. A fusion algorithm is then used to integrate the new and old data to generate the target fused map.

[0142] In this embodiment of the application, the above method can take advantage of the fact that the vehicle can freely explore the environment in manual driving mode. By collecting and fusing new status information and environmental perception data in real time, it can reflect changes in the parking lot environment in a timely manner, update the fused map, and thus improve the positioning accuracy and navigation efficiency during the vehicle summoning process.

[0143] As an optional implementation, determining the driving route between the vehicle's current location and the target location based on the vehicle's fused map includes: obtaining the target fused map from the cloud, and determining the driving route based on the target fused map.

[0144] Optionally, when the car owner remotely initiates a vehicle summoning command via mobile phone, the phone sends a request to the cloud to obtain the latest target fusion map. In response to the car owner's request, the cloud packages the target fusion map data and transmits it to the car owner's mobile phone via the network. The phone's built-in sensors collect data of the current environment, including wireless signal information, visual features, and semantic elements. The car owner's mobile phone uploads the collected data of the current environment to the cloud for further precise positioning and route planning. Based on the target fusion map and the car owner's current environment data, the cloud plans a driving route from the vehicle's current location to the car owner's location. Once the driving route is determined, the cloud sends the route data to the vehicle's intelligent driving domain controller. Based on the received driving route data, the vehicle activates the autonomous driving system and controls the vehicle to depart from its current location and drive along the planned route to the target location where the car owner is located, completing the summoning process.

[0145] In this embodiment, the method described above responds to a control command triggered by a terminal device on a vehicle, acquiring wireless signal status information and environmental perception information of the terminal device's environment. The environmental perception information represents the characteristics of target environmental factors in the environment, which affect the accuracy of the positioning of the terminal device. The fused map is obtained by fusing maps collected from multiple dimensions. Based on the status information and environmental perception information, the target location of the terminal device is determined. According to the fused map of the vehicle, the driving route between the vehicle's current location and the target location is determined. Following the driving route, the vehicle is controlled to travel from its current location to the target location. In other words, in this embodiment, the status information of the wireless signal and the environmental perception information of the terminal device's environment are acquired, and by fusing these two information, the target location of the terminal device is determined. The driving route between the vehicle's current location and the target location is determined using the fused map. The above method overcomes the limitation of low positioning accuracy in environments with poor or absent GPS signals in related technologies. It achieves the goal of determining the terminal position by integrating status information, environmental perception information, and fused maps, and then planning the driving route from the vehicle's current position to the target position, ultimately controlling the vehicle to automatically drive to the target position. This solves the technical problem of low vehicle control flexibility and achieves the technical effect of improving vehicle control flexibility.

[0146] The technical solutions of the embodiments of this application will be illustrated below with reference to preferred embodiments.

[0147] With the rapid development of the automotive industry and the continuous improvement of people's living standards, people have increasingly higher demands for the parking experience. The memory parking function can autonomously learn and memorize frequently used parking routes. The car owner only needs to issue a start command, and the vehicle can automatically follow the learned route. For example, after learning the route from the parking space to the parking lot exit or elevator lobby, the car owner can activate the memory parking function while in the car, and the vehicle can automatically drive to the end of the route. The intelligent vehicle summoning technology in parking lots allows users to summon the vehicle via a mobile application from locations such as the parking lot elevator lobby. The vehicle, in an unmanned state, automatically drives to the user's designated location using the memory parking function.

[0148] Currently, the methods for summoning include: setting a fixed summoning point during route learning and using the user's GPS location as the summoning point. Setting a fixed summoning point during route learning is relatively complex and cannot be modified after route learning is complete, making it inflexible in practical use. Using GPS location as the summoning point relies on a strong GPS signal and is only applicable to outdoor parking lots; it cannot be used in underground parking lots where there is no GPS signal.

[0149] This application provides a parking lot intelligent vehicle summoning method based on multi-mode fusion positioning, including: when learning a route, collecting wireless signals (Wi-Fi, Bluetooth signals) at each trajectory point on the learning route, including information such as signal name and strength, to form a wireless signal fingerprint map of the route; collecting elements such as parking spaces (including corner coordinates and parking space numbers), pillars, etc. on the route to form a semantic map of the route; collecting images on the route and extracting visual features from the images to form a visual feature map of the route; and fusing the wireless signal fingerprint map, semantic map, and visual feature map in the cloud to generate a semantic-feature-wireless signal fusion map of the route.

[0150] Optionally, each time a vehicle is manually driven along the route, wireless signals (Wi-Fi, Bluetooth signals), semantic elements, and visual features along the route are re-collected and fused with the existing semantic-feature-wireless signal map in the cloud to enhance the real-time performance of the map.

[0151] Optionally, when the car owner remotely summons the vehicle using their mobile phone, the semantic-feature-wireless signal map of the parking lot, downloaded from the cloud, is used on the phone. Semantic matching results are obtained by tightly coupling the semantic elements of the route with the IMU (Integrated Device Unit). This semantic matching utilizes not only conventional location information such as parking spaces and pillars, but also strong semantic elements like parking space numbers for more accurate results. Wireless signal positioning results are obtained by tightly coupling the wireless signal received by the mobile app with the IMU, and visual feature positioning results are obtained by extracting visual features from images captured by the phone's camera and tightly coupling them with the IMU. These semantic matching, wireless signal positioning, and visual feature positioning results are further fused using a Kalman algorithm to obtain a relatively accurate location for the mobile phone. This location is then forwarded from the cloud to the vehicle via the network, and the vehicle automatically drives to the owner's location.

[0152] Figure 2 This is a schematic diagram of a parking lot intelligent vehicle summoning architecture based on multi-mode fusion positioning according to an embodiment of this application, such as... Figure 2 As shown, the above architecture may include a cloud platform 22, a vehicle-side platform 23, and a terminal device 24. The cloud platform 22 may include a crowdsourced map fusion module 221 and a location service 222. The vehicle-side platform 23 may include a semantic mapping module 231, a visual feature mapping module 232, and a wireless signal acquisition mapping module 233.

[0153] Optionally, during route learning, the semantic mapping module 231 of the vehicle terminal 23 constructs a semantic map, the wireless signal mapping module 233 of the vehicle terminal 23 collects wireless signals along the route to construct a wireless signal fingerprint map, and the visual feature mapping module 232 of the vehicle terminal 23 collects images along the route to construct a visual feature map (SuperPoint features). The semantic map, wireless signal fingerprint map and visual feature map are then uploaded to the crowdsourced map fusion module 221 of the cloud 22.

[0154] Optionally, the crowdsourced map fusion module 221 in the cloud 22 merges the three into a semantic-feature-wireless signal fusion map on the terminal device 24. The fusion process includes: time alignment, that is, aligning the data streams of the three maps through hardware timestamps or software interpolation; and spatial alignment, that is, converting the coordinate systems of visual feature mapping and wireless signal mapping into the semantic map coordinate system.

[0155] Optionally, the merged map may include elements such as: parking space coordinates, parking space number, pillar coordinates, visual features (SuperPoint features), and wireless signals (Wi-Fi signal, Bluetooth signal).

[0156] Optionally, when a car owner uses the summon function (when the user is not in the car), the user selects the current parking lot on the mobile application, and the application will download the semantic-feature-wireless signal fusion map of the parking lot from the cloud.

[0157] Optionally, based on the name and strength of the wireless signal (Wi-Fi signal, Bluetooth signal) received by the mobile application, combined with the mobile phone IMU information, the wireless signal is located using a semantic-feature-wireless signal fusion map. The wireless signal location result and the observation noise covariance can be expressed by the following formula.

[0158]

[0159] in, It can be used to represent the location result obtained by locating a terminal device via wireless signals (S), that is, the first location. Used to represent x Observation noise variance along coordinate directions Used to represent the variance of observation noise in the y-coordinate direction. Used to represent the observation noise covariance at the first location. Used to represent the horizontal coordinate position of the terminal device calculated after positioning based on the received wireless signals. It is used to represent the horizontal coordinate position of the terminal device calculated after positioning based on the received wireless signal.

[0160] Optionally, based on the image recognition of the mobile phone camera, the environmental semantic information (parking space frame, pillar, parking space number, etc.) is obtained. The semantic localization algorithm is used to tightly couple and fuse the environmental semantic information with the IMU to obtain the semantic map localization result and the observation noise covariance, which can be expressed by the following formula.

[0161]

[0162] in, This is used to represent the semantic map localization result obtained by locating the terminal device using environmental semantic (Semantics, abbreviated as s) information, also known as the second sub-location. The observation noise covariance used to represent the first sub-position. Used to represent x Observation noise variance along coordinate directions Used to represent the variance of observation noise in the y-coordinate direction. Used to represent the horizontal coordinate position of the terminal device calculated after semantic positioning. Used to represent the vertical coordinate position of the terminal device calculated after semantic positioning.

[0163] Optionally, SuperPoint features are extracted from images captured by the mobile phone camera, and the feature localization algorithm is used to tightly couple and fuse the feature information with the IMU. The feature map localization result and the observation noise covariance can be expressed by the following formula.

[0164]

[0165] in, This is used to represent the feature map localization result obtained by locating the terminal device using features (f) extracted from images, also known as the second sub-location. The observation noise covariance used to represent the second sub-position. Used to represent x Observation noise covariance along coordinate directions Used to represent the observation noise covariance in the y-coordinate direction. Used to represent the horizontal coordinate position of the terminal device calculated after positioning based on features. Used to represent the vertical coordinate position of the terminal device calculated after positioning based on features.

[0166] Figure 3 This is a flowchart of a multi-mode localization fusion method according to an embodiment of this application; as follows: Figure 3 As shown, this multi-mode localization fusion method includes the following steps.

[0167] Step S302: Initialize X0 and P0.

[0168] Optionally, the initialization of the state vector X0 includes a state estimate at the initialization time (usually the first run), which can be a known initial position and other state parameters, or a set of estimates based on prior knowledge. The initialization of the state covariance matrix P0 includes the uncertainty of the initial state vector, reflecting the accuracy of the initial state estimate. It is usually a diagonal matrix, with diagonal elements representing the variance of the corresponding state parameters.

[0169] Step S304, State prediction.

[0170] Optionally, based on the state estimate of the previous time step, the state at time k is predicted using the system dynamics model, and the state covariance matrix is ​​also predicted to reflect the uncertainty of the predicted state.

[0171] Step S306: Check the available positioning results.

[0172] Optionally, based on the location information received at the current time (including wireless signal location, semantic map location, and visual feature location results), it is checked whether a certain accuracy standard is met (e.g., the noise covariance is less than a certain threshold). If no location is available, step S310 can be executed. If a location is available, step S308 can be executed.

[0173] Step S308: Determine the number of available locations.

[0174] Optionally, if only a single type of location data is available, it can be used directly. If it is determined that there is only one available location, step S312 can be executed; if it is determined that there are multiple available locations, step S314 can be executed.

[0175] Step S310: Skip the update and use the prediction results directly.

[0176] Alternatively, if no positioning result is considered available at the current moment, the state prediction result can be used directly and the update can be skipped.

[0177] Step S312: Use the positioning result and its corresponding R matrix.

[0178] Optionally, if only one positioning result is available, the observation update step of Kalman filtering is performed using the above positioning result and the corresponding observation noise covariance matrix.

[0179] Step S314: Select the positioning result and its corresponding R matrix based on the noise covariance of each positioning result.

[0180] Optionally, if multiple positioning results exist, the positioning result with the smallest noise covariance is selected as a candidate for updating.

[0181] Step S316: Perform observation update, updating the state and covariance.

[0182] Optionally, using the selected positioning results and the corresponding observation noise covariance matrix, the Kalman gain is calculated, and the state vector and state covariance matrix are updated to reflect the impact of the new measurement data.

[0183] Step S318: Output the final fusion result X_k|k.

[0184] Optionally, the output is updated to include the state estimate at time k with available information.

[0185] Step S320, wait for the next moment.

[0186] Optionally, the algorithm enters a waiting state until new data arrives at the next moment, and then repeats the entire process.

[0187] In this embodiment, the usability of the three positioning results is determined based on the magnitude of their covariance; a covariance greater than 5 meters is considered unusable. If no positioning is available, the predicted result is used directly. If one positioning result is available, that result and its corresponding R matrix are used to perform observation updates. If multiple positioning results are available, the positioning result with the smallest covariance is selected for observation updates, rather than using the previously used result. The positioning results are two-dimensional, and the system state can be defined using the following formula.

[0188]

[0189] in, Used to represent the state vector at time k. Used to represent the x-coordinate at time k. Used to represent the y-coordinate at time k.

[0190] Alternatively, state prediction can be expressed by the following formula.

[0191]

[0192] in, This represents the state vector estimated at time k based on information prior to time k (including time k-1).

[0193] Alternatively, covariance prediction can be expressed by the following formula.

[0194]

[0195] Among them, P k The state estimation error covariance matrix at time k is used. Q represents the covariance matrix of the system process noise. The process noise covariance can be expressed by the following formula.

[0196]

[0197] Alternatively, the observation matrix can be updated using the following formula.

[0198]

[0199] in, I is used to represent the mapping matrix between states and measurements, and I is used to represent the identity matrix.

[0200] Alternatively, the Kalman gain can be expressed by the following formula.

[0201]

[0202] Among them, P k Used to represent the state covariance matrix This is used to represent the mapping matrix between states and measurements; for location information, it is usually an identity matrix. Used to represent the observation noise covariance matrix.

[0203] Alternatively, the state update can be represented by the following formula.

[0204]

[0205] Among them, X K Used to represent the state vector. K k Let and represent the Kalman gain matrix of the Kalman filter at time k, and Zk represent the observation vector.

[0206] Alternatively, the covariance update can be expressed by the following formula.

[0207]

[0208] Among them, P k Used to represent the state covariance matrix. I is used to represent the identity matrix.

[0209] The methods of the embodiments of this application will be further illustrated below.

[0210] Figure 4 This is a schematic diagram of a vehicle summoning process according to an embodiment of this application. Figure 4 As shown, the mobile application forwards the owner's current location to the vehicle's intelligent driving domain through the cloud location service module. The intelligent driving domain projects the owner's location onto the memory trajectory, uses the owner's projection point as the endpoint, and controls the vehicle to automatically drive to the endpoint according to the vehicle summoning trajectory, thus completing the vehicle summoning process.

[0211] Figure 5 This is a schematic diagram of a vehicle control device according to an embodiment of this application. Figure 5 As shown, the control device 500 of the vehicle includes: an acquisition unit 502, a first determination unit 504, a second determination unit 506, and a control unit 508.

[0212] The acquisition unit 502 is used to respond to the control command triggered by the terminal device on the vehicle and acquire the status information of the wireless signal and the environmental perception information of the environment in which the terminal device is located.

[0213] The first determining unit 504 is used to determine the target location of the terminal device based on state information and environmental perception information.

[0214] The second determining unit 506 is used to determine the driving route between the vehicle's current location and the target location based on the vehicle's fused map.

[0215] Control unit 508 is used to control the vehicle to travel from its current location to the target location according to the driving route.

[0216] In this embodiment, the vehicle control device can acquire wireless signal status information and environmental perception information of the terminal device's environment by responding to the control command triggered by the terminal device. A first determining unit determines the target location of the terminal device based on the status information and environmental perception information. A second determining unit determines the driving route between the vehicle's current location and the target location according to the vehicle's fused map. The control unit then controls the vehicle to travel from its current location to the target location according to the driving route. This solves the technical problem of low vehicle control flexibility and improves the technical effect of enhancing vehicle control flexibility.

[0217] Embodiments of this application also provide a vehicle, including: a memory storing an executable program; and a processor for running the program, wherein the program executes the methods described in various embodiments of this application when it runs.

[0218] According to another aspect of the embodiments of this application, an electronic device is also provided, including: a memory storing an executable program; and a processor for running the program, wherein the program executes the methods in various embodiments of this application during runtime. According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored executable program, wherein, when the executable program runs, it controls the device where the computer-readable storage medium is located to execute the methods in various embodiments of this application.

[0219] Embodiments of this application also provide a computer-readable storage medium including a stored executable program, wherein, when the executable program is running, it controls the device where the computer-readable storage medium is located to perform the methods of various embodiments of this application.

[0220] Embodiments of this application also provide a computer program product, including a computer program that, when executed by a processor, implements the methods of various embodiments of this application.

[0221] Embodiments of this application also provide a computer program product, including a non-volatile computer-readable storage medium for storing a computer program that, when executed by a processor, implements the methods in various embodiments of this application.

[0222] Embodiments of this application also provide a computer program that, when executed by a processor, implements the methods described in the various embodiments of this application.

[0223] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0224] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between units or modules may be electrical or other forms.

[0225] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0226] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0227] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0228] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for controlling a vehicle, characterized in that, The vehicle and the terminal device communicate via wireless signals, and the method includes: In response to the control command triggered by the terminal device on the vehicle, the system acquires the status information of the wireless signal and the environmental perception information of the environment in which the terminal device is located. The environmental perception information is used to represent the characteristics of target environmental factors in the environment, and the target environmental factors are used to affect the accuracy of locating the terminal device. Based on the status information and the environmental perception information, the target location of the terminal device is determined; Based on the fused map of the vehicle, the driving route between the vehicle's current location and the target location is determined, wherein the fused map is obtained by fusing maps collected from multiple acquisition dimensions. According to the driving route, control the vehicle to travel from its current location to the target location.

2. The method according to claim 1, characterized in that, Determining the target location of the terminal device based on the state information and the environmental perception information includes: Based on the status information and the fused map, a first location of the terminal device is determined, wherein the first location is used to represent the location of the terminal device determined by the status information; Based on the environmental perception information, a second location of the terminal device is determined, wherein the second location is used to represent the location of the terminal device determined by the environmental perception information; The target location is determined based on the first location and the second location.

3. The method according to claim 2, characterized in that, The method further includes: The attitude information of the terminal device is obtained, wherein the attitude information is used to represent the motion state of the terminal device; Based on the status information and the fused map, determining the first location of the terminal device includes: By combining the state information and the attitude information, the wireless signal is located from the fused map to obtain the first location; or, The environmental perception information includes environmental semantic information and images. The environmental semantic information represents the attributes of the environmental features, and the images are obtained by collecting data on the environmental features. The second location includes a first sub-location and a second sub-location. Based on the environmental perception information, determining the second location of the terminal device includes: Based on the environmental semantic information and the pose information, the first sub-position is determined; From the image, attribute features of target feature points are extracted. Based on the attribute features and the pose information, the second sub-position is determined. The target feature points are feature points in the image with a recognition degree greater than a recognition degree threshold, which are used to improve the positioning accuracy of the terminal device.

4. The method according to claim 3, characterized in that, Determining the target location based on the first location and the second location includes: The observation noise covariance of the first position, the first sub-position, and the second sub-position is determined respectively, wherein the observation noise covariance is used to represent the degree of difference between the first position, the first sub-position, or the second sub-position and the actual position of the terminal device; The target position is determined based on the observation noise covariance corresponding to the first position, the first sub-position, and the second sub-position, respectively.

5. The method according to claim 4, characterized in that, Determining the target location based on the observation noise covariance corresponding to the first location, the first sub-location, and the second sub-location, respectively, includes: In response to the existence of an observation noise covariance less than or equal to a covariance threshold among the observation noise covariances corresponding to the first position, the first sub-position, and the second sub-position, the target position is determined based on the position where the observation noise covariance is less than or equal to the covariance threshold. In response to the fact that at least two of the observation noise covariances corresponding to the first position, the first sub-position, and the second sub-position are less than or equal to the covariance threshold, the positions where the observation noise covariance is less than or equal to the covariance threshold are fused to obtain the target position; In response to the fact that there is no observation noise covariance less than or equal to the covariance threshold among the observation noise covariances corresponding to the first position, the first sub-position, and the second sub-position, the target position is determined based on the selection instruction for the candidate position on the terminal device, wherein the candidate position is a position on the fused map that can control the vehicle to travel to.

6. The method according to claim 1, characterized in that, Controlling the vehicle to travel from its current location to the target location according to the stated driving route includes: Project the target location onto the driving route; The destination is the target location on the driving route, and the starting point is the location of the vehicle. Control the vehicle to travel from the starting point to the destination; or, The vehicle is connected to the cloud, and the method further includes: In response to the control command, the fused map is retrieved from the cloud; The maps collected from multiple dimensions include at least two of the following: wireless signal maps, semantic maps, and visual feature maps. The method further includes: In response to receiving the wireless signal map, semantic map, and visual feature map sent by the vehicle from the cloud, the cloud is used to fuse at least two of the three maps to obtain the fused map.

7. The method according to claim 6, characterized in that, In response to receiving the wireless signal map, semantic map, and visual feature map sent by the vehicle from the cloud, the cloud is used to fuse at least two of the three maps to obtain the fused map, including: In response to receiving the wireless signal map, semantic map, and visual feature map sent by the vehicle from the cloud, at least two of the three maps—wireless signal map, semantic map, and visual feature map—are aligned using the cloud in the time dimension to obtain a time alignment result. In the spatial dimension, the coordinate system of the visual feature map and the coordinate system of the wireless signal map are converted into the coordinate system of the semantic map to obtain the spatial alignment result; The fused map is obtained based on the time alignment result and the spatial alignment result.

8. The method according to any one of claims 1 to 7, characterized in that, The method further includes: In response to the vehicle being in manual driving mode and the vehicle traversing the driving route in the fused map, the status information and environmental perception data of at least one trajectory point on the driving route are collected. The collected state information and environmental perception data are fused into the fused map to obtain a target fused map, wherein the target fused map is used to replace the fused map stored in the cloud connected to the vehicle; or, Based on the fused map of the vehicle, the driving route between the vehicle's current location and the target location is determined, including: The target fusion map is obtained from the cloud, and the driving route is determined according to the target fusion map.

9. A vehicle, characterized in that, include: Memory, which stores executable programs; A processor for running the program, wherein the program, when running, performs the method according to any one of claims 1 to 8.

10. An electronic device, characterized in that, include: Memory, which stores executable programs; A processor for running the program, wherein the program, when running, performs the method according to any one of claims 1 to 8.

Images