Method and apparatus for assisting vehicle control

Abstract

This application proposes a method for assisting vehicle control, comprising: acquiring perception data about a target object in the vehicle's surrounding environment using environmental sensors, wherein the target object is only partially visible to the environmental sensors; determining basic attribute information about the target object based on the perception data using a trained machine learning model; performing a retrieval in a reference database based on the basic attribute information to determine reference geometric information for characterizing the complete geometric shape of the target object; and providing the reference geometric information for vehicle control. This application also proposes a system for assisting vehicle control, a device for assisting vehicle control, and a computer program product. By combining machine learning techniques with database retrieval, the machine learning model only needs to perform relatively simple recognition tasks and more reliably obtains the complete geometric shape information of the target object from the reference database.

Description

Technical Field

[0001] This application relates to a method for assisting vehicle control, a system for assisting vehicle control, an apparatus for assisting vehicle control, and a computer program product. Background Technology

[0002] While modern vehicles are equipped with various sensors, their sensing range still cannot completely cover all areas around the vehicle. For example, in automatic parking scenarios, some objects may not be fully observed because they are obstructed or outside the sensor's field of view. If the system fails to issue a timely warning to the driver or fails to make obstacle avoidance decisions in autonomous driving mode, a collision is highly likely to occur.

[0003] Currently, some technologies utilize machine vision to recognize objects, but these technologies are typically limited to identifying the object's category or general shape. For occluded or partially visible objects, existing solutions exhibit poor accuracy and limited information output, failing to meet the demands of safe and reliable vehicle control. Furthermore, some existing solutions rely on large amounts of fully labeled training data to train machine learning models that directly output the object's complete geometry, resulting in high data acquisition and model training costs. These shortcomings severely restrict vehicle driving safety in complex environments. Summary of the Invention

[0004] The purpose of this application is to provide a method for assisting vehicle control, a system for assisting vehicle control, a device for assisting vehicle control, and a computer program product, so as to at least solve some of the problems in the prior art.

[0005] According to a first aspect of this application, a method for assisting vehicle control is provided, the method comprising: Step S1: Obtain perception data about target objects in the environment surrounding the vehicle through the vehicle's environmental sensors, wherein the target objects are only partially visible to the environmental sensors; Step S2: Using a trained machine learning model, determine basic attribute information about the target object based on the perceived data; Step S3: Based on the basic attribute information, perform a retrieval in the reference database to determine reference geometric information used to characterize the complete geometric shape of the target object; and Step S4: Provide the reference geometry information for the control of the vehicle.

[0006] This application specifically includes the following technical concept: combining machine learning technology with database retrieval, enabling the machine learning model to perform only a relatively simple recognition task, and using this as the retrieval basis to obtain the complete geometric morphology information of the target object from a standardized reference database. This not only significantly reduces the difficulty of model training and the cost of data annotation, but also avoids the shortcomings of pure end-to-end models, such as poor interpretability and output results that deviate significantly from the true geometric dimensions due to the lack of geometric constraints, since the reference geometric information comes from a reference database built based on real physical dimensions. This results in highly reliable and stable reference geometric information. Furthermore, the reference database can be easily updated and expanded; adding new object types does not require retraining the model, exhibiting good maintainability and scalability. In summary, this solution can provide geometric information of only partially visible objects in the vehicle's surrounding environment at a lower cost and with higher reliability, thereby effectively improving vehicle driving safety in complex environments.

[0007] In one exemplary embodiment, the basic attribute information includes a descriptive tag for the target object. The descriptive tag at least includes the category of the target object; specifically, it also includes the brand, model, color, component name, and / or typical motion characteristics of the target object. In step S3, using the descriptive tag as a query condition, matching structured data entries are retrieved from the reference database, and the reference geometric information is read from the corresponding fields of the structured data entries. Thus, by using the descriptive tag as a search condition, the standard geometric information of the target object can be quickly and accurately located.

[0008] In an exemplary embodiment, the basic attribute information includes local geometric parameters of the target object's visible portion relative to the environmental sensor. These local geometric parameters include: size parameters, contour curves, keypoint sequences, edge segments, surface curvature, normal vector distribution, and / or point cloud segments. In step S3, based on the local geometric parameters, a shape similarity retrieval is performed in the reference database to match and obtain the complete geometric parameters of the target object, which serve as the reference geometric information. Therefore, shape similarity retrieval using local geometric parameters supports search and recognition based on pure geometric features, exhibiting strong generalization ability. Furthermore, this method does not rely on image semantic information, which is beneficial for data privacy protection and anonymization.

[0009] In an exemplary embodiment, the basic attribute information includes local feature vectors of the target object, which characterize the local visual features of the target object as visible to the environmental sensor. In step S3, the local feature vectors are used as query conditions to perform vector similarity retrieval in the reference database to match and obtain reference feature vectors that characterize the complete visual features of the target object. The reference geometric information is then determined based on the reference feature vectors. Thus, by using feature vector retrieval and model regression output, the generalization ability of retrieval and the flexibility of regression are combined, improving the accuracy of recognizing the complete geometric shape of unknown or uncommon categories of objects.

[0010] In one exemplary embodiment, the method further includes: in step S2, using a trained machine learning model, additionally determining a confidence level corresponding to the basic attribute information of the target object based on the perceived data; and in step S4, using the confidence level, employing a worst-case strategy or a typical-case strategy to determine the final reference geometric information provided for vehicle control. Thus, a typical-case strategy or a worst-case strategy can be flexibly selected according to actual needs or different driving scenarios to meet different requirements regarding decision-making accuracy and safety.

[0011] In one exemplary embodiment, the method further includes: acquiring the position of the target object; comparing the position of the target object with an environmental map to determine whether the target object involves a fixed landmark or a movable object, wherein the environmental map stores the positions and types of fixed landmarks in the environment surrounding the vehicle; and in step S3, additionally determining the reference geometric information of the target object based on the result of the comparison. Thus, by comparing with the environmental map, fixed landmarks can be quickly identified and filtered out, thereby improving the accuracy of subsequent reference geometric information retrieval.

[0012] In one exemplary embodiment, the method further includes: acquiring at least one type of auxiliary information, namely: vehicle location information, current time information, weather information, and / or road type information; and in step S2, using the machine learning model, additionally determining basic attribute information of the target object based on the auxiliary information. Thus, by fusing multi-dimensional auxiliary information such as location, time, and weather, the model's recognition accuracy in complex environments can be improved.

[0013] In one exemplary embodiment, the method further includes: using the machine learning model to identify additional components associated with the target object based on the perceptual data; and correcting the reference geometry information determined for the target object based on the identification results of the additional components; specifically, the additional components include at least one of the following: a motorcycle storage box, a modified handlebar for a bicycle, a bicycle child seat, a trailer box, and / or roof luggage. Thus, by using the identification results of the additional components to correct the reference geometry information, the true outline of the object can be accurately reflected, avoiding the neglect of standard size deviations caused by modifications or additions during driving decisions.

[0014] In an exemplary embodiment, step S4 includes: determining the positional relationship of the target object as a whole and / or its invisible parts relative to the vehicle based on the reference geometric information and the perception data, and providing this positional relationship for the vehicle's intelligent driving functions. Specifically, the intelligent driving functions include: collision warning, parking assistance, adaptive cruise control, automatic parking, automatic emergency obstacle avoidance, automatic emergency braking, and / or lane keeping assist. This allows for a more accurate grasp of the relative positional relationship between the complete outline of the object and the vehicle, providing a more accurate decision-making basis for the intelligent driving system, thereby improving driving safety.

[0015] In an exemplary embodiment, step S4 includes: generating visual prompt information based on the reference geometric information, and outputting the visual prompt information to the vehicle user through an in-vehicle display unit; specifically, the visual prompt information includes: the remaining contour shape completion of the target object for the portion not visible to environmental sensors, a standard two-dimensional or three-dimensional geometric model of the target object, a schematic diagram of the relative positional relationship between the complete contour of the target object and the vehicle, the minimum distance between the vehicle and the target object, and / or the collision risk area between the vehicle and the target object. Thus, by visually displaying the complete shape and positional relationship of an object to the driver, the driver can be assisted in more accurately judging the surrounding environment and improving driving confidence.

[0016] In one exemplary embodiment, step S4 further includes: receiving feedback from the driver regarding the visual prompt information, the feedback operation including, for example, a confirmation operation or a rejection operation; controlling the operation of the vehicle based on the feedback operation; and / or fine-tuning or retraining the machine learning model. Thus, by considering the driver's feedback operation, the vehicle control strategy can be adjusted in real time, and the model's recognition capability can be continuously optimized.

[0017] In one exemplary embodiment, the method further includes: performing anonymization processing on the perceived data before providing it to the machine learning model to remove vehicle identification information and / or personnel identification information from the perceived data. Thus, data anonymization processing can be used to protect the privacy of vehicles and personnel in the environment, meeting data compliance requirements.

[0018] In one exemplary embodiment, the steps of determining at least the basic attribute information and the reference geometric information are performed using a remote server outside the vehicle. Thus, by deploying the object recognition and retrieval tasks on the server side, the computational burden on the vehicle can be reduced, and centralized updates and maintenance of the model and database can be facilitated.

[0019] According to a second aspect of this application, a system for assisting vehicle control is provided, the system being used to perform the method according to a first aspect of this application, the system comprising: a vehicle-mounted device configured to acquire perception data about a target object via environmental sensors, upload the perception data to a remote server outside the vehicle, receive reference geometric information of the target object sent by the remote server, and control the vehicle based on the reference geometric information; and a remote server configured to receive the perception data uploaded by the vehicle-mounted device, determine basic attribute information of the target object based on the perception data using a trained machine learning model, perform a retrieval in a reference database based on the basic attribute information to determine reference geometric information for characterizing the complete geometric shape of the target object, and send the reference geometric information to the vehicle-mounted device.

[0020] According to a third aspect of this application, an apparatus for assisting vehicle control is provided, the apparatus including a memory and a processor, the memory storing computer program instructions, which, when executed by the processor, enable the processor to perform the method according to the first aspect of this application.

[0021] According to a fourth aspect of this application, a computer program product is provided, comprising computer program instructions, wherein, when executed by a processor, the computer program instructions enable the processor to perform the method according to a first aspect of this application. Attached Figure Description

[0022] The principles, features, and advantages of this application will be better understood below with reference to the accompanying drawings. The drawings include: Figure 1 A block diagram of a system for assisting vehicle control according to an exemplary embodiment of this application is shown; Figure 2 A block diagram of an apparatus for assisting vehicle control according to an exemplary embodiment of this application is shown; Figure 3 A flowchart of a method for assisting vehicle control according to an exemplary embodiment of this application is shown; Figure 4 A flowchart of a method for assisting vehicle control according to another exemplary embodiment of this application is shown; Figure 5 A flowchart of a method for assisting vehicle control according to another exemplary embodiment of this application is shown; Figure 6 A flowchart of a method for assisting vehicle control according to another exemplary embodiment of this application is shown; Figure 7 A schematic diagram illustrating the use of the method according to this application in an exemplary application scenario is shown; Figure 8 A schematic diagram illustrating the use of the method according to this application in another exemplary application scenario is shown; and Figure 9 A schematic diagram illustrating the use of the method according to this application in another exemplary application scenario is shown. Detailed Implementation

[0023] To make the technical problems to be solved, the technical solutions, and the beneficial technical effects of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and several exemplary embodiments. It should be understood that the specific embodiments described herein are only for explaining this application and are not intended to limit the scope of protection of this application.

[0024] Figure 1 A block diagram of a system for assisting vehicle control according to an exemplary embodiment of this application is shown.

[0025] like Figure 1 As shown, the system 100 for assisting vehicle control includes a vehicle-side device 10 and a remote server 20. The vehicle-side device 10 and the remote server 20 establish a data connection through a communication network 6 (e.g., a 4G / 5G mobile communication network, a V2X vehicle network, or a Wi-Fi network).

[0026] The vehicle-mounted device 10 is installed in a vehicle 1 equipped with intelligent driving capabilities. The intelligent driving capabilities refer, for example, to the vehicle 1's ability to perform all or part of dynamic driving tasks within a defined Operating Design Domain (ODD), such as corresponding to driving automation levels L2+ to L4. The vehicle-mounted device 1 includes, for example, a processor and a memory (not shown for simplicity), the memory storing computer program instructions that, when executed by the processor, perform at least some steps of a method for assisting vehicle control.

[0027] like Figure 1As shown, the vehicle-mounted device 10 is connected to environmental sensors 11, 12, and 13 mounted on the vehicle 1. These environmental sensors 11, 12, and 13 include, but are not limited to, cameras, millimeter-wave radar, lidar, and / or ultrasonic sensors, used to collect perception data of the environment surrounding the vehicle. The perception data may include, for example, image data, point cloud data, and radar signals. In some scenarios, the target object may be partially visible due to occlusion or being outside the sensor's field of view (e.g., obscured by other vehicles or partially outside the sensor's field of view). Therefore, the collected perception data only contains local information about the target object, rather than its complete outline.

[0028] In one embodiment, the vehicle-mounted device 10 may perform preprocessing on the perceived data before uploading it to the remote server 20. This preprocessing includes, but is not limited to, image compression, format conversion, noise reduction filtering, and invalid data filtering. Furthermore, to protect the privacy of road users in the road environment, the vehicle-mounted device 10 may also perform anonymization processing on the perceived data, such as blurring, mosaicking, or color-blocking the license plate area or face area in the image to remove identifiable personnel or vehicle identity information.

[0029] Subsequently, the vehicle-mounted device 10 uploads the perceived data via the communication network 6 to a remote server 20 located outside the vehicle 1. This remote server 20 may be located on a cloud platform 2, such as a backend server of the original equipment manufacturer (OEM). The remote server 20 includes a processor and memory (not shown for simplicity) and has access to a reference database 30. This reference database 30 may be, for example, a networked third-party database or a database provided by other service providers.

[0030] The remote server 20 stores a trained machine learning model 21 (e.g., a deep neural network) in its memory. The processor loads and executes the machine learning model 21 to determine basic attribute information 25 of the target object from the perceptual data. Subsequently, the processor performs a retrieval of a reference database 30 based on the basic attribute information 25 and retrieves reference geometric information 35 from the reference database 30 that matches the basic attribute information 25. This reference geometric information 35 can characterize the complete geometric shape of the target object.

[0031] In other embodiments not shown, the reference database 30 may also be stored directly in the local storage of the remote server 20. In this case, the reference database 30 is no longer networked, but belongs to a private database pre-built by the original equipment manufacturer, or is deployed on a server on a different cloud platform 2 than the remote server 20.

[0032] After determining the reference geometric information 35 at the remote server 20, the remote server 20 sends the reference geometric information 35 to the vehicle-mounted device 10. Simultaneously, it may also send the recognition results obtained using the machine learning model 21, such as the basic attribute information 25 of the target object. Upon receiving this information, the vehicle-mounted device 10 can use it to assist in the control of the vehicle 1. For example, it can generate visual prompts based on the reference geometric information 35 and present them to the driver through the in-vehicle display unit, or directly provide them to the intelligent driving system for intelligent driving functions such as collision warning, automatic emergency braking, and parking assistance.

[0033] It should be noted that, although in Figure 1 The diagram illustrates a vehicle-cloud collaborative solution. However, in actual implementation, all or part of the steps of the method described in this application may be executed locally on vehicle 1 without relying on remote server 20. Furthermore, the method described in this application may also be implemented using other communication architectures, such as deploying some processing tasks on edge computing nodes, or employing vehicle-to-vehicle collaboration, vehicle-to-roadside unit collaboration, etc. This application does not impose any limitations on these methods.

[0034] Figure 2 A block diagram of an apparatus for assisting vehicle control according to an exemplary embodiment of this application is shown.

[0035] like Figure 2 As shown, the device 50 includes a processor 51 and a memory 52. ​​The memory 52 may include a computer-readable storage medium such as a hard disk, RAM, or flash memory, and stores computer program instructions. The processor 51 may be a central processing unit (CPU), a microcontroller unit (MCU), a graphics processing unit (GPU), a neural network processing unit (NPU), a digital signal processor (DSP), or other general-purpose or special-purpose processor. When the processor 51 executes the computer program instructions stored in the memory 52, it can perform the method for assisting vehicle control described in this application.

[0036] In one embodiment, device 50 may be deployed locally in the vehicle, with the method executed independently by the onboard computing platform.

[0037] In another embodiment, device 50 may be deployed on a remote server (e.g., the back-end server of the original equipment manufacturer), which receives sensing data from the vehicle, directly generates control commands, and sends them to the vehicle.

[0038] In another embodiment, such as Figure 1 As shown, the functionality of device 50 can be partially deployed locally in the vehicle and partially deployed on a remote server, with both working together to execute methods for assisting vehicle control.

[0039] Figure 3A flowchart illustrating a method for assisting vehicle control according to an exemplary embodiment of this application is shown. In this embodiment, Figure 3 The method shown includes steps S1 to S4, and can be achieved by means of... Figure 1 The system 100 shown and / or Figure 2 The device 50 shown is executed.

[0040] In step S1, perception data about target objects in the vehicle's surrounding environment is acquired through the vehicle's environmental sensors, wherein the target objects are only partially visible to the environmental sensors.

[0041] In one embodiment, the environmental sensor may include sensors installed outside and / or inside the vehicle for monitoring the road environment. Accordingly, the sensing data may include: images captured by an onboard camera, lidar point clouds, millimeter-wave radar signals, ultrasonic distance data, etc. The sensing data may originate from a single sensor or be the result of multi-sensor fusion detection (e.g., fusion of images and point clouds).

[0042] In one embodiment, the target object may include fixed landmarks and / or movable objects in the road environment, such as another vehicle, motorcycle, bicycle, pedestrian, animal, building (e.g., fence, wall, pillar, garage door, etc.), traffic facilities (e.g., traffic cones, crash barriers, traffic signs, etc.), natural objects (e.g., trees, rocks), or other obstacles (e.g., temporary construction barriers). For movable objects, the sensing data may also include continuous observation data over a period of time to reflect their motion characteristics (e.g., speed, acceleration, direction of motion, trajectory, etc.) and changes in position.

[0043] In this context, "the target object is only partially visible to the environmental sensor" means that part of the target object's outline or some parts are not within the sensor's field of view, preventing the sensor from acquiring its complete geometric information. This partially visible state can be caused by various factors, such as the sensor's limited field of view, mutual occlusion between objects, and the existence of perception blind spots even after multi-sensor fusion.

[0044] For example, the target object may only be partially within the field of view of the environmental sensor. Taking a camera as an example, if the target object is large or close, part of it may be outside the camera's field of view, resulting in only a localized area being captured.

[0045] For example, the target object may be occluded by other environmental objects. In a parking scenario, a motorcycle might be partially obscured by a parked vehicle in front, preventing its rear or handlebars from being observed by the sensors. These obstructions could be walls, other vehicles, roadblocks, pedestrians, etc.

[0046] For example, even if a vehicle is equipped with multiple sensors with different viewing angles (such as surround-view cameras, multiple LiDARs, etc.), after the data collected by each sensor is stitched or fused, some parts of the target object may still be located in the gaps, blind spots, or obscured by complex environments within the sensor coverage area, resulting in local areas not being covered by the perceived data.

[0047] In one embodiment, the perception data may include multiple partially visible objects in the vehicle's surrounding environment. The vehicle-mounted device can designate one or more of these objects as target objects, along with identification information (such as a temporary ID), and then report the perception data to a remote server for processing. Alternatively, all partially visible objects in the perception data can be treated as target objects; that is, as long as an object is partially invisible to the environmental sensors, the subsequent reference geometry information recognition process is triggered.

[0048] In one embodiment, after acquiring the perception data, it can first be preprocessed or lightweightly identified to determine whether there are partially visible objects. The subsequent reference geometry information recognition process is only triggered when partially visible objects are detected. If all objects in the perception data are fully visible, it can be directly used for vehicle control to save computing resources and network bandwidth.

[0049] In one embodiment, after acquiring the perceived data but before providing it to the machine learning model, the perceived data can be anonymized to remove vehicle identification information and / or personnel identification information. For example, this anonymization can be performed after data is acquired at the vehicle end but before it is uploaded to a remote server. The anonymization process includes at least one of the following: blurring or occluding the license plate area in the image, blurring or occluding the face area, and removing or encrypting the vehicle identification information. This achieves anonymization, allowing the remote server to subsequently identify and retrieve data based on data that does not contain privacy information, and send the results (such as reference geometric information) back to the vehicle, then immediately delete the received data without leaving a record.

[0050] In step S2, a trained machine learning model is used to determine basic attribute information about the target object based on the perceived data.

[0051] In one embodiment, perceived data is input into a trained machine learning model, which outputs basic attribute information of the target object. This basic attribute information is not the complete geometry or precise location of the target object, but rather a summary description or intermediate representation that characterizes a portion of the visible target object, such as object category, local geometric features, feature vectors, etc. By limiting the output to basic attribute information rather than complete geometric information, the learning difficulty and training cost of the model can be reduced, while enabling the model to have stronger generalization ability.

[0052] In one embodiment, in addition to providing the perceived data to the machine learning model, auxiliary information can also be acquired, and the basic attribute information of the target object can be determined by the machine learning model based on the perceived data and the auxiliary information.

[0053] The auxiliary information includes, for example: - Vehicle location information: For example, by obtaining the vehicle's current location through GPS, the geographical area where the vehicle is located can be determined, which helps machine learning models to identify the typical vehicle models or object features common in that area (such as small hatchbacks common in Europe, pickup trucks common in North America, etc.).

[0054] - Current time information: This includes the current time, season, and holidays. For example, during school hours near a school, the system can increase the recognition weight for pedestrians and two-wheeled vehicles. At night, the confidence requirement for certain objects can be reduced or the recognition strategy can be adjusted.

[0055] - Weather information: such as rain, snow, fog, and lighting conditions. In severe weather, the signal-to-noise ratio of the sensor may decrease, and the system can adjust the confidence threshold of the model output accordingly, or switch to a more robust recognition strategy.

[0056] - Road type information: such as highways, urban roads, parking lots, school zones, etc. The distribution of object categories and prior probabilities differ in different road types (e.g., a high proportion of stationary vehicles in parking lots, and high speeds of moving vehicles on highways). Utilizing this information can improve the model's recognition accuracy and decision-making rationality.

[0057] The aforementioned auxiliary information can be fused with perceptual data at the model input (e.g., as an additional feature channel) or used for post-processing adjustments after model output (e.g., adjusting confidence thresholds or prior probabilities).

[0058] In one embodiment, the machine learning model can employ deep neural networks, such as convolutional neural networks (CNN), recurrent neural networks (RNN), Transformers, attention-based neural networks, graph neural networks (GNN), etc. The machine learning model can also be a visual foundation model, for example, pre-trained on publicly available internet data (such as image-text pairs), and then fine-tuned using in-vehicle perception data to adapt to the road environment recognition task.

[0059] In one embodiment, the training data for the machine learning model can be derived from: vehicle-side perception data, roadside equipment perception data, publicly available internet datasets, synthetic data generated by simulation platforms, vehicle design specifications, and CAD models. Furthermore, transfer learning can be performed using general visual models. The aforementioned data is manually labeled before being used for model training.

[0060] In one embodiment, the training data can be filtered to prioritize or specifically construct perceptual data containing only partially visible objects, and the model can be specifically enhanced through training to improve its ability to recognize occluded scenes. For example, the training data may include images showing only parts of objects, point cloud fragments, etc., and cover different object types, materials, occlusion locations, and poses. Thus, the model can learn to infer the overall shape from local information, thereby becoming adept at recognizing target objects that are only partially visible.

[0061] In one embodiment, historical perception data of specific geographical areas, statistical information of typical vehicle models, and road environment feature data can also be used to perform regional pre-training or fine-tuning of the machine learning model, so that the model can recognize objects unique to each geographical area (such as small hatchbacks common in Europe, motorcycles common in Southeast Asia, and pickup trucks common in North America), thereby improving the model's scene adaptability.

[0062] In step S3, a retrieval is performed in the reference database based on the basic attribute information to determine reference geometric information for characterizing the complete geometric shape of the target object.

[0063] In one embodiment, the reference database can be deployed in various ways: for example, an internet-based online database, a proprietary database built by the original equipment manufacturer (OEM), or an offline database pre-deployed locally. Data sources may include standard dimensional parameters from different road traffic design specifications, publicly available vehicle geometry model libraries, perception data shared by other vehicles, and publicly available image information from the internet.

[0064] In this context, "reference geometric information" refers to data used to characterize the complete two-dimensional and / or three-dimensional geometry of a target object. Reference geometric information includes, but is not limited to, at least one of the following: the complete dimensions of the object (such as length, width, height, diameter, etc.), shape description (such as contour curves, geometric structure, surface features, etc.), volume, key point coordinates (such as corner points, center points, edge points, etc.), two-dimensional or three-dimensional models (such as CAD models, point cloud models, mesh models, etc.), and other geometric parameters that can reflect the complete spatial occupancy of the object.

[0065] In one embodiment, the reference database stores structured data, such as data organized using relational database tables or JSON format. Each data entry corresponds to one or a class of objects and includes identification fields (such as category, brand, model) and their corresponding values ​​(such as category name, brand name, model code, specific size parameters, model file path, etc.). For example, in a structured data entry, the values ​​of the identification fields category, brand, and model are motorcycle, Mercedes, and R1250GS, respectively, and the values ​​of the reference geometry information fields length, width, and height are 1.8m, 0.9m, and 1.2m, respectively.

[0066] In this configuration, the basic attribute information determined by the machine learning model can include a descriptive label for the target object, which at least includes a category and optionally brand, model, etc. Using this descriptive label as a query condition (such as an SQL equivalence query or key-value matching), matching structured data entries can be retrieved from the reference database. For example, comparing field values ​​in the descriptive label (such as category "motorcycle", brand "Mercedes", model "R1250GS") with identifier field values ​​in the database entry locates a structured data entry whose identifier field value matches exactly. Subsequently, the values ​​of reference geometry information fields (such as length, width, and height) are read from this matching entry to obtain the complete dimensional parameters of the target object.

[0067] In one embodiment, the reference database stores complete geometric models (e.g., 2D contour templates, 3D CAD models, point cloud models, etc.) and / or complete geometric parameters (e.g., length, width, height, volume, etc.) of various objects.

[0068] In this configuration, the basic attribute information determined by the machine learning model includes the local geometric parameters of the target object's visible portion to environmental sensors, such as local dimensions, contour curves, keypoint sequences, edge segments, surface curvature, normal vector distribution, and point cloud segments. Subsequently, based on these local geometric parameters, a shape similarity retrieval is performed in a reference database. Shape similarity retrieval refers to calculating the degree of shape matching between the target object's local geometric parameters and pre-stored geometric models in the database (e.g., through contour comparison, point cloud registration, curvature distribution comparison, etc.), and selecting one or more models with the highest matching degree as the retrieval results. Thus, the complete geometric parameters of the target object can be obtained as the reference geometric information.

[0069] In this scenario, the reference database does not need to store image data, and the model does not need to identify object categories or perform complex semantic reasoning. Instead, it can obtain complete geometric information through shape matching based solely on local geometric morphology, which is beneficial for anonymization and data desensitization.

[0070] In one embodiment, the reference database stores feature vectors (also known as embeddings). A feature vector is a fixed-dimensional numerical vector extracted and output by a machine learning model from raw perceptual data, such as a 512-dimensional floating-point vector [0.23, -0.45, ..., 0.67]. This vector is not a human-readable category or size label, but rather a high-dimensional abstract encoding of the visual features of the target object (such as shape, texture, edges, etc.).

[0071] In this configuration, the basic attribute information determined by the machine learning model includes local feature vectors of the target object, used to characterize the local visual features of its visible portion. Using these local feature vectors as query conditions, a vector similarity retrieval (e.g., using cosine similarity or Euclidean distance) is performed in a reference database to obtain a reference feature vector characterizing the complete visual features of the target object. Subsequently, this reference feature vector can be input into a trained regression model (e.g., a lightweight neural network), which outputs reference geometric information of the target object.

[0072] It should be noted that the data format in the reference database is not limited to a single type, but can be a mixture of multiple data formats to support different types of retrieval methods.

[0073] In one embodiment, at least steps S2 and S3 described above are performed using a remote server located outside the vehicle (e.g., Figure 1 The remote server (as shown) performs the operation to reduce the computational overhead on the vehicle side. Specifically, after acquiring the perception data, the vehicle-side device immediately reports it to the remote server, which then determines the basic attribute information and retrieves the reference geometric information. The remote server then sends the obtained reference geometric information back to the vehicle-side device for subsequent vehicle control.

[0074] In step S4, reference geometry information is provided for vehicle control.

[0075] In one embodiment, the positional relationship of the target object as a whole and / or its invisible parts relative to the vehicle can be determined based on reference geometric information of the target object and perception data acquired in advance in step S1. This positional relationship is then provided for intelligent driving functions of the vehicle. The intelligent driving functions include at least one of the following: collision warning, parking assistance, automatic parking, automatic emergency obstacle avoidance, automatic emergency braking, intersection assistance, and / or lane keeping assistance.

[0076] Specifically, the local dimensions and positions of the visible parts of the target object can be determined based on the perceived data. Then, combined with the complete dimensions provided in the reference geometry, the remaining dimensions of the invisible parts of the target object can be calculated. Subsequently, based on the position of the visible parts and the remaining dimensions, the spatial position of the invisible parts relative to the vehicle can be deduced.

[0077] For example, the vehicle's environmental sensors detect that the bicycle wheel is located 1.5 meters to the right front of the vehicle. Reference geometry information shows that the standard height of this model is 1.1 meters and the wheel height is 0.3 meters. Based on this, the system calculates that the handlebars are located approximately 0.8 meters above the wheel and obtains their shape and span, thereby determining the positional relationship of the invisible parts. This allows the system to assess the collision risk between the handlebars and the rearview mirror or provide parking assistance prompts.

[0078] For example, in a parking scenario, the camera only captures the lower half of the fire hydrant (visible height 0.2 meters), while reference geometry indicates a standard height of 0.5 meters. It is inferred that the remaining 0.3 meters of the upper half lies above the visible portion, and the entire hydrant is located 1.2 meters behind the vehicle. The system completes the outline and issues a warning or automatically brakes if the distance is too close.

[0079] In one embodiment, a visual prompt is generated based on reference geometric information and output to the vehicle user via an in-vehicle display unit. The in-vehicle display unit includes, for example, a central control display, an instrument panel display, and / or a head-up display (HUD).

[0080] For example, the visual prompt information includes at least one of the following: - Remaining contour completion shape: Completes the contour shape of the target object in the image for the parts that are not visible to the environmental sensor (e.g., by dashed lines, highlights, semi-transparent or colored fills) to represent the complete geometric shape; - Standard 2D or 3D geometric model: Displays the standard geometric model of the target object (e.g., a CAD model retrieved from a reference database), supports interactive operations such as rotation and scaling, and helps drivers identify the shape, structure, and even texture features of objects that are only partially visible; - Schematic diagram of the relationship between the complete outline of the target object and the vehicle position: In the form of top view, side view or panoramic view, schematically show the relative positional relationship between the complete outline of the target object and the vehicle, for example, indicating that the fire hydrant is located 1.2 meters behind the vehicle; - Minimum distance or collision risk zone between vehicle and target object: Calculates and displays the minimum distance between the target object and the vehicle, circles the collision risk zone of the vehicle, and indicates the collision risk level with color (green / yellow / red) or sound according to the distance threshold.

[0081] In one embodiment, in addition to using reference geometry information for vehicle control, basic attribute information output by the machine learning model (such as object category, model, typical motion characteristics, etc.) can also be considered to assist the vehicle's intelligent driving system in making more accurate decisions. Similarly, when providing visual prompts to the driver, basic attribute information of the target object, such as the object's category, model, expected motion trend, and typical motion characteristics, can also be displayed to help the driver better anticipate collision risks.

[0082] In one embodiment, after outputting visual prompts to the driver, feedback from the driver on the visual prompts can also be received. Based on the feedback, at least one of the following operations can be performed: controlling the operation of the vehicle, fine-tuning or retraining the machine learning model.

[0083] The feedback operations include, for example, confirmation or rejection. For instance, the driver can confirm or reject the system's displayed object category or completed outline using a touchscreen, voice command, or physical button.

[0084] For example, if the system incorrectly identifies a motorcycle with an added storage box as a standard motorcycle, the driver can click "Deny" and correct the identification result.

[0085] Upon receiving a rejection request, the vehicle can be controlled to perform actions such as deceleration and emergency braking to avoid collision risks that may arise from inaccurate identification. Furthermore, the rejection request can be used as negative feedback to retrain the machine learning model. For example, if a user repeatedly rejects the identification result of a certain type of target object (such as the in-vehicle display showing "motorcycle" while the user denies it), the default confidence level of that type of result can be reduced in subsequent identifications.

[0086] Upon receiving a confirmation command, the recognition result can be used as a positive feedback sample for model retraining to improve recognition accuracy. Furthermore, the vehicle can be controlled to make driving decisions or obstacle avoidance maneuvers based on this reference geometric information.

[0087] The feedback operation may also include manual fine-tuning of the visual prompts. For example, the driver can manually adjust the completed outline displayed on the in-vehicle display unit (such as dragging the outline edges to correct the length or width), or input information such as the brand, model, and size of an object that they believe to be correct. After verification, the information adjusted manually can be used to correct erroneous reference geometry information for corresponding entries in the reference database, or to add reference geometry information for new objects.

[0088] Figure 4 A flowchart of a method for assisting vehicle control according to another exemplary embodiment of this application is shown. In this embodiment, Figure 3The method shown includes sub-steps S11 and S12 in step S1, and sub-step S3 includes sub-step S30 in step S3.

[0089] In step S11, the position of the target object is additionally obtained. For example, the approximate position of the target object in space can be determined by analyzing the pixel position of the target object in the image captured by the vehicle-mounted camera, and combining the installation parameters of the vehicle-mounted camera relative to the vehicle body and the vehicle's own positioning information (such as GPS coordinates).

[0090] In step S12, the position of the target object is compared with the environmental map to determine whether the target object is a fixed landmark or a movable object. The environmental map may be, for example, a high-precision map that stores the location and type information of fixed landmarks in the environment surrounding the vehicle. Fixed landmarks include, for example, lampposts, traffic signs, fire hydrants, crash barriers, parking space barriers, building pillars, etc.

[0091] In one embodiment, if the location of the target object overlaps with the location of a fixed landmark recorded in the environment map, or the deviation between the two is within a preset threshold range, then the target object is determined to be a fixed landmark. Otherwise, it is determined to be a movable object.

[0092] In step S30, in addition to performing database retrieval based on the basic attribute information determined by the machine learning model in step S2, the comparison results in step S12 (i.e., whether the target object is a fixed landmark or a movable object) can also be considered.

[0093] In one embodiment, if the target object is a fixed landmark and its size parameters (such as length, width, and height) are already recorded in the environmental map, the size parameters can be directly read from the environmental map as its reference geometric information, or the read size parameters can be used to correct the database retrieval results. Alternatively, a database retrieval can be triggered only when the target object is a movable object; if it is a fixed landmark, the reference geometric information can be directly read from the environmental map to save computational resources.

[0094] In one embodiment, if the target object is determined to be a fixed landmark, the basic attribute information determined by the machine learning model can be verified using basic information such as the type and location of the fixed landmark read from the environmental map before performing a database search. For example, if the category output by the model is inconsistent with the environmental map record, the map record is used for correction first. As another example, if the model outputs multiple candidate results (such as streetlights, fire hydrants, traffic signs, etc.), map information can be used to assist in selecting the most matching candidate (e.g., selecting streetlights from the candidates based on the actual types of fixed landmarks existing in the current location map), and then a database search is performed based on the filtered results.

[0095] Figure 5A flowchart of a method for assisting vehicle control according to another exemplary embodiment of this application is shown. In this embodiment, Figure 3 Step S2 is shown as including sub-steps S21 and S22, and step S3 is shown as including sub-steps S31 and S32.

[0096] The execution process of sub-step S21 and Figure 3 The described step S2 is similar, namely, using a machine learning model to determine the basic attribute information (such as category, model, local geometric features, etc.) of the target object based on perceptual data. Subsequently, in sub-step S31, reference geometric information is retrieved from the reference database and determined based on this basic attribute information.

[0097] For example, if the basic attribute information indicates that the target object is a certain model of motorcycle, the standard size parameters of that model can be found in the reference database.

[0098] In sub-step S22, with the help of a machine learning model, additional components associated with the target object are identified based on perceptual data.

[0099] The additional components refer to, for example, non-original parts of the target object, which are added to the target object later through installation or modification and become integrated with it. These additional components will change the actual shape and overall size of the target object, directly causing a deviation between the standard size parameters read from the reference database and the actual shape of the target object.

[0100] The additional components include, for example, motorcycle storage boxes, modified handlebars for bicycles, bicycle child seats, trailer boxes, and / or roof luggage.

[0101] In sub-step S32, the reference geometric information that was determined for the target object in step S31 is corrected based on the identification results of the additional components.

[0102] In one embodiment, if an additional component is identified, redundancy or a buffer value can be added to the original reference geometry information sent out based on the reference database, or multiplied by a preset coefficient to reserve a safety margin. The size of the buffer value and / or preset coefficient can be determined based on the type and size of the additional component, or predicted by a machine learning model, or determined by looking up a table or based on experience.

[0103] For example, suppose the standard width of a bicycle found in the reference database is 0.6 meters. If it is detected that the bicycle has been fitted with an irregularly shaped handlebar (the width of which is significantly greater than that of a standard handlebar), the width can be multiplied by a factor of 1.2 to obtain 0.72 meters, which is used as the corrected width to allow for a safety margin.

[0104] In another embodiment, the additional component can be treated as an independent target object, and its corresponding geometric dimensions or typical protruding dimensions (such as the protruding length of the storage box, the extension dimension of the child seat, the width increment of the side box, etc.) can be retrieved from a reference database. Then, this dimension can be superimposed on the reference geometric information of the original target object to obtain the corrected complete dimension.

[0105] For example, if the standard length of a motorcycle found in the reference database is 1.8 meters, but a storage box for storing food delivery videos is found to be installed at the rear of the motorcycle, and the storage box is found to have a protruding length of 0.3 meters as an independent object in the reference database, then the corrected total length of the motorcycle is 1.8 meters + 0.3 meters = 2.1 meters.

[0106] It should be noted that the above values ​​are merely illustrative. In practical applications, the reference geometry may be more complex (e.g., 3D models, surface functions, multidimensional contour curves, etc.), and the corresponding corrections may involve comprehensive adjustments to multiple parameters (e.g., simultaneously correcting length, width, height, and overall contour shape). All of the above complex situations fall within the scope of protection of this application.

[0107] Figure 6 A flowchart of a method for assisting vehicle control according to another exemplary embodiment of this application is shown. In this embodiment, Figure 3 Step S2 is shown as including sub-steps S210 and S220, and steps S3 and S4 are shown as including sub-steps S300 and S400, respectively.

[0108] The execution process of sub-step S210 and Figure 3 The described step S2 is similar, namely, using a machine learning model to determine the basic attribute information of the target object based on perceptual data.

[0109] In sub-step S220, with the help of a trained machine learning model, the confidence level corresponding to the basic attribute information of the target object is additionally determined based on the perceptual data.

[0110] In one embodiment, the machine learning model does not output a single recognition result, but rather outputs recognition results for multiple candidate objects and assigns a corresponding confidence level to each candidate object.

[0111] The confidence level can be expressed in various forms, such as a percentage (e.g., 60%), a probability value (e.g., 0.6), or a score. For example, the confidence level can represent: the uncertainty of the model's overall category judgment (e.g., "the confidence level of the target object is a bicycle is 70%)", the uncertainty of identifying a specific part (e.g., "the confidence level of the visible part is a bicycle wheel is 85%)", or the uncertainty of the existence of the object as a whole.

[0112] For example, a trained machine learning model can output that the target object has a 60% probability of being a "motorcycle", a 30% probability of being a "bicycle", and a 10% probability of being a "pedestrian".

[0113] Subsequently, in sub-step S300, reference geometric information is retrieved and determined from the reference database based on the basic attribute information determined for the target object.

[0114] In one embodiment, if the machine learning model identifies multiple candidate target objects, the corresponding reference geometric information can be retrieved and determined for each candidate target object from the reference database.

[0115] For example, suppose the machine learning model outputs two candidate recognition results: motorcycle model A has a confidence level of 80% and its standard size parameters (e.g., total length 2m) are retrieved from the reference database; motorcycle model B has a confidence level of 85% and its standard size parameters (e.g., total length 1.8m) are retrieved.

[0116] In sub-step S400, based on the confidence level, a worst-case strategy or a typical-case strategy is adopted to determine the final reference geometry information provided for vehicle control.

[0117] In one embodiment, when employing a worst-case strategy, candidates with a confidence level higher than a preset threshold (e.g., 70%) are first selected from all candidate identification results. Then, the candidate with the largest reference geometric information (such as size parameters) is chosen from the selected results. Taking the motorcycle mentioned above as an example, both model A (80% confidence level, 2m total length) and model B (85% confidence level, 1.8m total length) have confidence levels higher than the 70% threshold. Therefore, the worst-case strategy selects the larger model A (2.0m), even though its confidence level is not the highest.

[0118] In one embodiment, when using a typical case strategy, the reference geometric information corresponding to the candidate recognition result with the highest confidence is directly selected. Taking the motorcycle mentioned above as an example, the confidence of model B (85%) is higher than that of model A (80%), so the reference geometric information (1.8m) of model B is selected.

[0119] In one embodiment, a typical scenario strategy or a worst-case scenario strategy can be dynamically selected based on the driving scenario in which the vehicle is operating. For example, in safety-critical driving scenarios such as high-speed driving, automatic emergency braking, and automatic parking, a worst-case scenario strategy (i.e., selecting the candidate result with the largest geometric size) is adopted to prioritize driving safety. When the vehicle is stationary and is only used to provide prompts or warnings to the user (such as navigation destination reminders), a typical scenario strategy (i.e., selecting the candidate result with the highest confidence) can be adopted to pursue accuracy in judgment.

[0120] In one embodiment, multiple candidate target objects can be visualized simultaneously on the in-vehicle display unit, allowing the user to select the correct result. This selection serves as feedback information for retraining the machine learning model. For example, if the machine learning model has a high confidence level in vehicle model B, but the user actually selects vehicle model A, this can be used as negative feedback for model correction or fine-tuning.

[0121] In one embodiment, historical observation data for a specific region can also be used to implement a worst-case strategy. For example, if historical data shows that the probability of motorcycle model A appearing in a certain region is significantly higher than that of model B, and the overall size of model A is larger than that of model B, then even if the confidence level of model B is slightly higher, the system can still prioritize model A as the output of the worst-case strategy to balance safety and scenario adaptability.

[0122] It should be noted that the method of determining the final reference geometric information using confidence scores in this embodiment is merely exemplary. In practical applications, candidate target objects can also be screened directly using confidence scores before searching the reference database, or the confidence score results can be used to influence the database retrieval process (e.g., adjusting retrieval thresholds or weights). All of the above implementation methods fall within the protection scope of this application.

[0123] Figure 7 A schematic diagram illustrating the use of the method according to this application in an exemplary application scenario is shown.

[0124] like Figure 7 As shown in the upper part, in the road environment image 7 taken with the help of a vehicle-mounted camera, a food delivery rider is driving a motorcycle 71 on the road in front of the vehicle. Due to the limited field of view of the camera and the close distance between the motorcycle and the rider, only the lower half of the motorcycle 71 and a portion of the storage box 710 at the rear for storing food can be seen in the image 7; the upper half of the motorcycle (such as the handlebars and rearview mirrors) and another part of the storage box 710 are not visible in the image 7 because they are outside the field of view of the vehicle-mounted camera.

[0125] like Figure 7 As shown in the lower part, in the vehicle display unit 15 (e.g., the central control screen), a visual prompt message about the motorcycle is generated, that is, the invisible parts of the motorcycle are fully outlined 72. This completion not only covers the complete body of the motorcycle (including the upper handlebars, rearview mirrors, etc.), but also completes the appearance of the takeout storage box added to the rear of the motorcycle.

[0126] Below the interface of the vehicle display unit 15, there is also an interactive option 74, which allows the driver to judge whether the outline completed by the system is correct. The driver can select "confirm" or "no" according to the actual situation.

[0127] If the driver confirms that the completed outline is correct, the intelligent driving system can calculate the relative position between the vehicle and the motorcycle, calculate the minimum distance between the outer outlines of the two vehicles, and perform obstacle avoidance operations accordingly.

[0128] Figure 8 A schematic diagram illustrating the use of the method according to this application in another exemplary application scenario is shown.

[0129] like Figure 8 As shown in the upper part, in the road environment image 8 captured by the vehicle-mounted camera, a cyclist 81 is crossing the road against a red light in front of the vehicle. Due to the limited field of view of the vehicle-mounted camera, only the lower half of the bicycle 81 (such as the wheels and pedals) can be observed in the captured image 8, while the upper half (such as the seat and handlebars) is not visible because it is beyond the field of view.

[0130] If relying solely on existing obstacle detection methods (i.e., based only on the visible portion), the intelligent driving system may underestimate the actual width of the bicycle, especially since the handlebars are significantly wider than the frame. When the vehicle is in autonomous driving mode, ignoring the lateral extension of the handlebars could potentially lead to a collision between the vehicle and the bicycle.

[0131] To address the aforementioned issues, a machine learning model is used to identify the basic attribute information of the target object (e.g., the category is "bicycle") based on perceptual data. This information is then combined with a reference database to retrieve the bicycle's reference geometry (including the standard width of the handlebars and the bicycle's height). Based on this, and considering the wheel steering angle and the relative positional relationship between the vehicle and the currently visible parts of the bicycle, the actual position of the handlebars in space and their lateral width are further calculated.

[0132] like Figure 8 As shown in the lower part, a visual prompt about the bicycle is generated in the vehicle display unit 15 (such as the central control screen). This information shows the relative positional relationship between the complete outline 82 of the bicycle and the vehicle 80 in a top-down view. The top-down view clearly shows that the complete outline 82 of the bicycle (including the handlebars) has been completed. The lateral width of the handlebars significantly exceeds the width of the wheels, making the overall outline 82 of the bicycle closer to the vehicle 80 than the wheels, thus more realistically reflecting the bicycle's footprint in actual space. This visual prompt can assist the driver and / or the intelligent driving system in fully considering the space occupied by the bicycle handlebars, thereby making safer obstacle avoidance decisions.

[0133] Figure 9 A schematic diagram illustrating the use of the method according to this application in another exemplary application scenario is shown.

[0134] like Figure 9As shown above, vehicle 1 is performing parallel parking in a road environment. The figure schematically illustrates the field of view 110 of vehicle 1's environmental sensors. It can be seen that a fire hydrant 91 on the roadside only partially falls within this field of view 110, so only a local edge is perceived by vehicle 1.

[0135] Next, based on the perceived data of the visible portion of the fire hydrant 91, its basic attribute information (such as the category "fire hydrant") is identified through a machine learning model, and a matching standard three-dimensional geometric model 93 is retrieved from the reference database 30. From this three-dimensional geometric model 93, the overall outer contour, side structures, and dimensions of the protruding portion towards the road of the fire hydrant 91 can be derived.

[0136] like Figure 9 As shown in the lower part, in the vehicle display unit 15, a visual prompt is generated based on the retrieved three-dimensional geometric model 93, completing the full outline 92 of the fire hydrant and overlaying it onto the simulated surrounding environment of the vehicle 90. Simultaneously, potential collision risk locations between the vehicle 90 and the full outline 92 of the fire hydrant are marked, and distance warning information 96 is displayed. Through this visual prompt, the driver can intuitively understand the actual location and appearance of the fire hydrant, thus effectively avoiding obstacles when parking. Furthermore, this information can also be provided to assist the automatic parking process of the vehicle 1, so as to plan or adjust the parking trajectory based on the relative distance between the vehicle 1 and the full outline of the fire hydrant.

[0137] It should be noted that Figures 7 to 9 The examples illustrate the generation of visual prompts, but this is not mandatory. In other embodiments, the determined reference geometry information can also be used directly for the generation of intelligent driving decisions or vehicle control commands.

[0138] Although specific embodiments of this application are described in detail herein, they are given for illustrative purposes only and should not be construed as limiting the scope of this application. Various substitutions, modifications, and alterations can be conceived without departing from the spirit and scope of this application.

Claims

1. A method for assisting vehicle control, the method comprising: Step S1: Obtain perception data about target objects in the environment surrounding the vehicle through the vehicle's environmental sensors, wherein the target objects are only partially visible to the environmental sensors; Step S2: Using a trained machine learning model, determine basic attribute information about the target object based on the perceived data; Step S3: Perform a retrieval in the reference database based on the basic attribute information to determine reference geometric information used to characterize the complete geometric shape of the target object; as well as Step S4: Provide the reference geometry information for the control of the vehicle.

2. The method according to claim 1, wherein, The basic attribute information includes a description tag for the target object, which includes at least the category of the target object. In particular, the description tag also includes the brand, model, color, component name and / or typical motion characteristics of the target object. In step S3, the description tag is used as the query condition to retrieve matching structured data entries from the reference database, and the reference geometric information is read from the corresponding field of the structured data entry.

3. The method according to claim 1 or 2, wherein, The basic attribute information includes the local geometric parameters of the target object visible to the environmental sensor. The local geometric parameters include: size parameters, contour curves, key point sequences, edge segments, surface curvature, normal vector distribution, and / or point cloud segments. In step S3, based on the local geometric parameters, a shape similarity search is performed in the reference database to match and obtain the complete geometric parameters of the target object, which are then used as the reference geometric information.

4. The method according to any one of claims 1 to 3, wherein, The basic attribute information includes the local feature vector of the target object, which is used to characterize the local visual features of the target object in relation to the visible part of the environmental sensor. In step S3, the local feature vector is used as a query condition to perform vector similarity retrieval in the reference database to match and obtain a reference feature vector that represents the complete visual features of the target object, and the reference geometric information is determined based on the reference feature vector.

5. The method according to any one of claims 1 to 4, wherein, The method further includes: In step S2, a confidence level corresponding to the basic attribute information of the target object is additionally determined based on the perceived data using a trained machine learning model. In step S4, based on the confidence level, a worst-case strategy or a typical-case strategy is used to determine the final reference geometry information provided for vehicle control.

6. The method according to any one of claims 1 to 5, wherein, The method further includes: Obtain the position of the target object; The location of the target object is compared with the environment map to determine whether the target object involves a fixed landmark or a movable object, wherein the environment map stores the location and type of fixed landmarks in the environment surrounding the vehicle. In step S3, reference geometric information of the target object is additionally determined based on the results of the comparison.

7. The method according to any one of claims 1 to 6, wherein, The method further includes: Obtain at least one of the following auxiliary information: vehicle location information, current time information, weather information and / or road type information; In step S2, the basic attribute information of the target object is additionally determined based on the auxiliary information using the machine learning model.

8. The method according to any one of claims 1 to 7, wherein, The method further includes: Using the machine learning model, additional components associated with the target object are identified based on the perceived data; Based on the identification results of the additional components, the reference geometric information for determining the target object is corrected. Specifically, the additional components include at least one of the following: motorcycle storage box, bicycle modified handlebars, bicycle child seat, trailer box and / or roof luggage.

9. The method according to any one of claims 1 to 8, wherein, Step S4 includes: Based on the reference geometric information and the perception data, the positional relationship of the target object as a whole and / or its invisible parts relative to the vehicle is determined, and this positional relationship is provided for the vehicle's intelligent driving functions. In particular, the intelligent driving functions include: collision warning, parking assistance, adaptive cruise control, automatic parking, automatic emergency obstacle avoidance, automatic emergency braking, and / or lane keeping assist.

10. The method according to any one of claims 1 to 9, wherein, Step S4 includes: Visual prompts are generated based on the reference geometric information and output to vehicle users through the in-vehicle display unit. Specifically, the visual prompts include: the remaining outline shape of the target object for the portion not visible to the environmental sensors, the standard two-dimensional or three-dimensional geometric model of the target object, a schematic diagram of the relative position of the complete outline of the target object and the vehicle, the minimum distance between the vehicle and the target object, and / or the collision risk area between the vehicle and the target object.

11. The method according to claim 10, wherein step S4 further comprises: Receive feedback from the driver regarding the visual prompt information, including, for example, a confirmation or rejection operation; The operation of the vehicle is controlled based on the feedback operation, and / or the machine learning model is fine-tuned or retrained.

12. The method according to any one of claims 1 to 11, wherein, The method further includes: Before providing the perceived data to the machine learning model, the perceived data is anonymized to remove vehicle identification information and / or personnel identification information.

13. The method according to any one of claims 1 to 12, wherein, The steps of determining the basic attribute information and the steps of determining the reference geometry information are performed using a remote server outside the vehicle.

14. A system for assisting vehicle control, the system being used to perform the method according to any one of claims 1 to 13, the system comprising: The vehicle-mounted device is configured to acquire perception data about a target object through environmental sensors, upload the perception data to a remote server outside the vehicle, receive reference geometric information of the target object sent by the remote server, and control the vehicle based on the reference geometric information. as well as A remote server is configured to receive perception data uploaded by the vehicle-mounted device, determine the basic attribute information of the target object based on the perception data using a trained machine learning model, perform a retrieval in a reference database based on the basic attribute information to determine reference geometric information used to characterize the complete geometric shape of the target object, and send the reference geometric information to the vehicle-mounted device.

15. An apparatus for assisting vehicle control, the apparatus comprising a memory and a processor, the memory storing computer program instructions that, when executed by the processor, enable the processor to perform the method according to any one of claims 1 to 13.

16. A computer program product comprising computer program instructions, wherein, When executed by a processor, the computer program instructions enable the processor to perform the method according to any one of claims 1 to 13.

Images