Vehicle control method, device, vehicle, storage medium and program product

Abstract

The present disclosure relates to the field of vehicle control, and in particular to a vehicle control method, device, vehicle, storage medium and program product. A first position of a vehicle and a first environment image of the vehicle in the driving direction at the current moment can be obtained; a target relocation model corresponding to the first position can be determined; a second position of the vehicle can be determined according to the first environment image through the target relocation model, wherein the position accuracy of the second position is higher than that of the first position; and the vehicle can be controlled to perform a preset driving operation according to the second position.

Description

Technical Field

[0001] The present disclosure relates to the field of vehicle control, and in particular to a vehicle control method, device, vehicle, storage medium and program product. Background Art

[0002] In recent years, autonomous driving technology has developed rapidly. Automated Valet Parking (AVP) is an important application of autonomous driving in parking scenarios. It generates a customized parking map by learning the parking routes and parking spaces in the parking lot. When parking is needed later, the vehicle can autonomously complete the parking in and out of the parking lot according to the customized map. In this process, how to match the correct parking lot customized map and determine the precise location of the vehicle itself in the map is the key link for the vehicle to perform automatic valet parking. Summary of the invention

[0003] To overcome the problems existing in the related art, the present disclosure provides a vehicle control method, device, vehicle, storage medium and program product.

[0004] According to a first aspect of an embodiment of the present disclosure, there is provided a vehicle control method, comprising:

[0005] Acquire a first current position of the vehicle and a first environment image of the vehicle in a current driving direction;

[0006] Determining a target relocation model corresponding to the first position;

[0007] Determining, according to the first environment image, a second position where the vehicle is currently located by using the target relocation model, wherein the position accuracy of the second position is higher than that of the first position;

[0008] The vehicle is controlled to perform a preset driving operation according to the second position.

[0009] Optionally, determining the second position where the vehicle is currently located by using the target relocation model according to the first environment image includes:

[0010] After inputting the first environment image into the target relocation model, obtaining the first position coordinates of each pixel point in the first environment image in the world coordinate system;

[0011] The second position where the vehicle is currently located is determined according to the first position coordinates of each pixel point in the first environment image.

[0012] Optionally, determining the second position where the vehicle is currently located according to the first position coordinates of each pixel point in the first environment image includes:

[0013] Determine a camera pose corresponding to the first environment image according to a first position coordinate of each pixel point in the first environment image and a camera intrinsic parameter corresponding to the first environment image;

[0014] The second position where the vehicle is currently located is determined according to the camera posture and a preset external parameter relationship, where the preset external parameter relationship is an external parameter relationship between the vehicle and an image acquisition device that acquires the first environment image.

[0015] Optionally, the target relocation model includes at least one, and determining the second position where the vehicle is currently located by using the target relocation model according to the first environment image includes:

[0016] In the case where the target relocation model includes multiple ones, after the first environment image is input into each of the target relocation models, the first position coordinates of each pixel point in the first environment image respectively output by each of the target relocation models in the world coordinate system are obtained;

[0017] The second position where the vehicle is currently located is determined according to the first position coordinates of each pixel point in the first environment image respectively output by each of the target relocation models.

[0018] Optionally, the second position comprises a first global pose of the vehicle in a world coordinate system;

[0019] Determining the second position of the vehicle according to the first position coordinates of each pixel point in the first environment image respectively output by each target relocation model comprises:

[0020] For each of the target relocation models, determining a plurality of undetermined global poses of the vehicle according to the first position coordinates of each pixel point in the first environment image output by the target relocation model;

[0021] Determining the pose consistency of the plurality of pending global poses, wherein the pose consistency is used to characterize the degree of consistency of the plurality of pending global poses;

[0022] The first global pose is determined according to the pose consistency of the multiple pending global poses corresponding to each of the target relocation models, wherein the first global pose is a pose determined according to the pending global pose with the highest pose consistency.

[0023] Optionally, determining the target relocation model corresponding to the first position includes:

[0024] The relocation model corresponding to the preset area where the first position is located is used as the target relocation model, and different preset areas correspond to different relocation models.

[0025] Optionally, the target relocation model is pre-trained in the following manner:

[0026] Acquire a plurality of second environment images of the vehicle at a preset position, and a second global position of the vehicle in a world coordinate system when the vehicle acquires the second environment image, wherein different second environment images correspond to different image acquisition angles, and the preset position includes the first position;

[0027] After model training is performed on a preset relocation model according to the plurality of second environment images and the second global pose, the target relocation model corresponding to the preset position is obtained.

[0028] Optionally, the obtaining the target relocation model corresponding to the preset position after model training is performed on the preset relocation model according to the plurality of second environment images and the second global pose comprises:

[0029] For each second environment image, after inputting the second environment image into the preset relocation model, obtain the second position coordinates of each pixel point in the second environment image output by the preset relocation model in the world coordinate system;

[0030] Reprojecting the second position coordinates of each pixel point in the second environment image into an image coordinate system according to the second global pose and the camera internal and external parameters corresponding to the plurality of second environment images, to obtain a reprojection point corresponding to each pixel point;

[0031] For each of the pixel points, a reprojection error is constructed according to the reprojection point corresponding to the pixel point and the pixel point, and the preset relocation model is trained according to the reprojection error to obtain the target relocation model.

[0032] Optionally, the preset relocation model includes a feature extraction network and a location prediction network connected to the feature extraction network;

[0033] After the preset relocation model is trained according to the reprojection error, obtaining the target relocation model comprises:

[0034] After model training is performed on the position prediction network according to the reprojection error, the target relocation model is obtained.

[0035] According to a second aspect of an embodiment of the present disclosure, there is provided a vehicle control device, comprising:

[0036] An acquisition module is configured to acquire a first position where the vehicle is currently located and a first environment image of the vehicle in a driving direction at a current moment;

[0037] a determination module configured to determine a target relocation model corresponding to the first position; and determine a second position where the vehicle is currently located by using the target relocation model according to the first environment image, wherein the position accuracy of the second position is higher than that of the first position;

[0038] The control module is configured to control the vehicle to perform a preset driving operation according to the second position.

[0039] According to a third aspect of an embodiment of the present disclosure, a vehicle is provided, comprising:

[0040] processor;

[0041] a memory for storing processor-executable instructions;

[0042] Wherein, the processor is configured to execute the steps of the method described in the first aspect of the present disclosure.

[0043] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method described in the first aspect of the present disclosure are implemented.

[0044] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, including a computer program, which implements the steps of the method described in the first aspect of the present disclosure when executed by a processor.

[0045] The technical solution provided by the embodiments of the present disclosure may include the following beneficial effects: obtaining the first position where the vehicle is currently located and the first environmental image of the vehicle in the driving direction at the current moment; determining the target relocation model corresponding to the first position; determining the second position where the vehicle is currently located through the target relocation model according to the first environmental image, wherein the position accuracy of the second position is higher than that of the first position; and controlling the vehicle to perform a preset driving operation according to the second position. In this way, based on the rough first position and environmental image of the vehicle, the vehicle can be accurately positioned (i.e., the second position is determined) through the pre-trained target relocation model, and the tolerance for the external environment is high. Even in harsh scenes such as cities with severe obstructions, canyons, old garages with few traffic signs, or parking lots without underground tunnels, the vehicle positioning initialization for valet parking can still be stably completed, thereby improving the user's parking experience.

[0046] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0048] Figure 1 The present invention is a flow chart of a vehicle control method according to an exemplary embodiment.

[0049] Figure 2 is based on Figure 1 The illustrated embodiment shows a flow chart of a vehicle control method.

[0050] Figure 3 is based on Figure 1 The illustrated embodiment shows a flow chart of a vehicle control method.

[0051] Figure 4 is based on Figure 1 The illustrated embodiment shows a flow chart of a model training method.

[0052] Figure 5 is a block diagram showing a vehicle control device according to an exemplary embodiment.

[0053] Figure 6 is based on Figure 5 The illustrated embodiment shows a block diagram of a vehicle control device.

[0054] Figure 7 is a block diagram of a vehicle according to an exemplary embodiment. DETAILED DESCRIPTION

[0055] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0056] It should be noted that all actions of acquiring signals, information or data in the present disclosure are carried out in compliance with the relevant data protection laws and policies of the country where the device is located and with the authorization given by the owner of the corresponding device.

[0057] The present disclosure is mainly applied to vehicle positioning in the scenario of autonomous driving. Taking autonomous valet parking as an example, during the process of autonomous valet parking, the vehicle position needs to be accurately positioned so as to match the correct parking lot customized map based on the location of the vehicle, and autonomously complete the parking in and out of the parking lot based on the customized map.

[0058] In related technologies, RTK (Real-Time Kinematic) can be used to provide the initial global position of the vehicle, and based on the initial global position, a customized map of the parking lot can be loaded for positioning. Since the positioning effect of RTK is severely restricted by the environment, the stability and accuracy of this solution are not ideal in scenes with severe obstruction such as cities with high-rise buildings and canyons. In another related technology, the underground tunnel is used for positioning triggering to complete the initial positioning of the vehicle. However, this method can only complete initialization after entering or walking through the tunnel, and the parking experience is average. In addition, not every parking lot has an underground tunnel, so it is not suitable for parking lots without tunnels, and the underground tunnel scene is relatively simple. In addition, multiple underground tunnels in the city may be very close to each other, so the mapping density will also be limited.

[0059] In order to solve the above problems, the present disclosure provides a vehicle control method, device, vehicle, storage medium and program product. The specific implementation of the present disclosure is described in detail below with reference to the accompanying drawings.

[0060] Figure 1 FIG. 1 is a flow chart of a vehicle control method according to an exemplary embodiment. The method can be applied to a vehicle. Figure 1 As shown, the method comprises the following steps:

[0061] In step S101, a first position of a vehicle and a first environment image of the vehicle in a current driving direction are acquired.

[0062] The first position can be understood as the rough position of the vehicle, and the first position can be regarded as the rough initial global position of the vehicle. In actual application scenarios, the first position of the vehicle can be obtained based on satellite positioning or other methods. In addition, the first environmental image can be an environmental image in front of the vehicle captured by the front camera of the vehicle.

[0063] Taking the automatic parking of a vehicle in a parking lot as an example, in order to realize automatic parking after the vehicle enters the parking lot, in one possible implementation method, the position of the vehicle can be accurately located when the vehicle drives to the entrance of the parking lot, so as to accurately load the customized map of the corresponding parking lot for automatic parking based on the precise vehicle position. Therefore, in an actual application scenario, when the vehicle drives to the entrance of the parking lot, by executing this step, the first position of the vehicle can be obtained and the first environment image can be captured, so as to subsequently accurately locate the current position of the vehicle based on the first position and the first environment image.

[0064] In step S102, a target relocation model corresponding to the first position is determined.

[0065] Among them, the target relocation model is a pre-trained deep learning model. In the present disclosure, the target relocation model is used to accurately locate the position of the vehicle based on the first environment image.

[0066] In order to accurately locate the vehicle through the relocalization model, different relocalization models can be trained at different locations (either a location point or an area). The relocalization models corresponding to the different locations after training can be implicit representations of the real scene where the corresponding location is located. In this way, the relocalization models trained at different locations can be used to accurately locate the different locations where the vehicle is located.

[0067] In a possible implementation of this step, the relocation model corresponding to the preset area where the first position is located can be used as the target relocation model, and different preset areas correspond to different relocation models. In addition, different position points in the same preset area can also correspond to different relocation models. Therefore, the target relocation model corresponding to the first position may include one or more.

[0068] The preset area may be an area within a preset distance range (eg, within a range of 100 meters) around the first position as the center point.

[0069] In step S103, a second position where the vehicle is currently located is determined according to the first environment image by using the target relocation model, wherein the position accuracy of the second position is higher than that of the first position.

[0070] In this step, after the first environment image is input into the target relocation model, the first position coordinates of each pixel point in the first environment image in the world coordinate system are output through the model, and then the second position is determined according to the first position coordinates of each pixel point in the first environment image.

[0071] The first position coordinate of each pixel point in the world coordinate system is the global position coordinate of each pixel point.

[0072] It should be noted that the target relocation model is pre-trained based on the multi-angle environment image (i.e., the second environment image in the following text) collected by the vehicle at the first position, and the global posture of the vehicle at the first position, so that the real scene where the first position is located can be implicitly represented by the trained target relocation model. Therefore, by inputting the first environment image in the scene where the first position is located into the target relocation model, the global position coordinates of each pixel in the image can be restored.

[0073] In step S104, the vehicle is controlled to perform a preset driving operation according to the second position.

[0074] The preset driving operation may be, for example, automatic parking.

[0075] For example, when controlling the vehicle to perform automatic parking according to the second position, the third position corresponding to the second position can be determined based on a preset position mapping relationship. The third position is the precise position of the vehicle in the customized map of the parking lot. The vehicle can then be controlled to perform automatic parking in the parking lot based on the third position. This is only an example and the present disclosure is not limited to this.

[0076] By adopting the above method, based on the rough first position of the vehicle and the environment image, the pre-trained target relocalization model can be used to accurately locate the vehicle (i.e., determine the second position). It has a high tolerance for the external environment and can be used to stably initialize the vehicle positioning for valet parking even in harsh scenarios such as cities and canyons with severe occlusion, old garages with few traffic signs, or parking lots without underground tunnels, thereby improving the user's parking experience.

[0077] Figure 2 is based on Figure 1 The embodiment shown is a flow chart of a vehicle control method, such as Figure 2 As shown, step S103 includes the following sub-steps:

[0078] In step S1031, after the first environment image is input into the target relocation model, the first position coordinates of each pixel point in the first environment image in the world coordinate system are obtained.

[0079] The first position coordinate is the global three-dimensional position of the corresponding pixel point in the world coordinate system.

[0080] In one possible implementation, the model structure of the target relocalization model may include two parts. The first part is a convolutional network, which is used to extract high-dimensional features from the input first environment image. The second part is a network regression head part composed of a multi-layer perceptron, which is used to predict the global three-dimensional position of the pixel coordinates of each pixel point in the first environment image.

[0081] By executing this step, after the first environment image is input into the target relocation model, feature extraction can be first performed based on the convolutional network of the model, and then the extracted features are input into the network regression head part, so as to obtain the first position coordinates of each pixel point in the first environment image output by the model in the world coordinate system.

[0082] In step S1032, the second position where the vehicle is currently located is determined according to the first position coordinates of each pixel point in the first environment image.

[0083] In this step, the camera pose corresponding to the first environmental image can be determined according to the first position coordinates of each pixel point in the first environmental image and the camera intrinsic parameters corresponding to the first environmental image; the second position where the vehicle is currently located can be determined according to the camera pose and a preset external parameter relationship, and the preset external parameter relationship is the external parameter relationship between the vehicle and the image acquisition device that acquires the first environmental image.

[0084] The camera intrinsic parameters corresponding to the first environment image are the camera intrinsic parameters of the image acquisition device that acquires the first environment image. The camera pose corresponding to the first environment image refers to the camera global pose of the image acquisition device that acquires the first environment image, and the camera global pose generally includes the position (translation vector) and direction (rotation matrix or quaternion) of the camera. In addition, the second position can be the global pose of the vehicle in the world coordinate system.

[0085] For example, for each preset number of pixels (for example, at least 4 pixels) in the pixel points of the first environment image, the three-dimensional space point (i.e., the first position coordinate) of each pixel in the preset number of pixels can be obtained, and the two-dimensional pixel point corresponding to each pixel in the image can be determined based on the constraint relationship (2D-3D point pair) between the two-dimensional pixel point and the three-dimensional space point of each pixel in the first environment image, and then the PNP (Perspective-n-Point) algorithm can be applied to calculate the global pose of the camera based on the known camera intrinsic parameters. Then, according to the global pose of the camera and the preset external parameter relationship between the vehicle body and the camera, the global pose of the vehicle is obtained. The above examples are only illustrative, and the present disclosure is not limited to this.

[0086] As described above, the target relocalization model may include at least one. When the target relocalization model includes multiple models, each target relocalization model can determine the global pose of a vehicle in the world coordinate system. In this case, the global poses of the vehicle determined by multiple target relocalization models need to be screened in order to determine the second position with the highest accuracy.

[0087] Figure 3 is based on Figure 1 The embodiment shown is a flow chart of a vehicle control method, such as Figure 3 As shown, step S103 includes the following sub-steps:

[0088] In step S1033, when the target relocation model includes multiple ones, after the first environment image is input into each of the target relocation models, the first position coordinates of each pixel point in the first environment image output by each of the target relocation models in the world coordinate system are obtained.

[0089] In step S1034, the second position where the vehicle is currently located is determined according to the first position coordinates of each pixel point in the first environment image respectively output by each of the target relocation models.

[0090] The second position includes a first global pose of the vehicle in a world coordinate system.

[0091] In this step, for each of the target relocation models, a plurality of pending global poses of the vehicle can be determined according to the first position coordinates of each pixel point in the first environment image output by the target relocation model; the pose consistency of the plurality of pending global poses can be determined, and the pose consistency is used to characterize the degree of consistency of the plurality of pending global poses; the first global pose can be determined according to the pose consistency of the plurality of pending global poses corresponding to each of the target relocation models, wherein the first global pose is a pose determined according to the pending global pose with the highest pose consistency.

[0092] Among them, whether multiple pending global postures are consistent can be determined by judging the position distance between the multiple pending global postures. It can be understood that the smaller the position distance between the multiple pending global postures, the higher the posture consistency of the corresponding multiple pending global postures.

[0093] In the present disclosure, the target relocation model is an implicit reconstruction of the real scene where the first position is located. Therefore, if the scene currently located by the vehicle is not the implicitly reconstructed scene corresponding to the target relocation model, then after the first environment image is input into the target relocation model, the inferred global three-dimensional position of the pixel point is messy. Therefore, when it is determined that the target relocation model corresponding to the acquired first position includes multiple ones, the global posture of the vehicle can be screened and filtered based on the posture consistency of multiple undetermined global postures after reprojection of the first position coordinates of each pixel point, so as to avoid obtaining an erroneous relocation result.

[0094] For each target relocation model, in the process of determining multiple pending global poses of the vehicle based on the first position coordinates of each pixel point in the first environment image output by the target relocation model, for each preset number of pixels in the first environment image, the first position coordinates of the preset number of pixels can be reprojected based on the camera intrinsic parameters and the preset external parameter relationship (i.e., the external parameter relationship between the vehicle and the camera) to obtain a pending global pose of the vehicle. The specific implementation steps of reprojecting the first position coordinates of the preset number of pixels to obtain the pending global pose of the vehicle can refer to the relevant description in step S1032 above, which will not be repeated here.

[0095] It can be understood that if the scene in which the vehicle is currently located is the implicitly reconstructed scene corresponding to the target relocation model, then the global three-dimensional position of the pixel point inferred by the target relocation model and the global pose of the vehicle obtained by reprojection are consistent, that is, the multiple pending global poses of the vehicle are the same or close in position. Therefore, in the present disclosure, when the target relocation model includes multiple, the pose consistency of the multiple pending global poses corresponding to each target relocation model can be determined, so that the pending global pose with the highest pose consistency can be used as the first global pose of the vehicle.

[0096] Figure 4 is based on Figure 1 The embodiment shown is a flow chart of a model training method, such as Figure 4 As shown, the target relocalization model can be pre-trained in the following way:

[0097] In step S401, multiple second environment images of the vehicle at a preset position and a second global posture of the vehicle in a world coordinate system when the vehicle captures the second environment image are obtained, different second environment images correspond to different image capture angles, and the preset position includes the first position.

[0098] The plurality of second environment images may include environment images from multiple angles collected by the vehicle at a preset position. The second global position may be a global three-dimensional position of the vehicle. In actual application scenarios, the second global position may be obtained based on existing positioning technology (such as satellite positioning technology).

[0099] For example, in order to realize automatic valet parking of a vehicle in a parking lot, the position of the vehicle can be accurately located when the vehicle drives to the entrance of the parking lot. In the present disclosure, the parking lot entrance can be implicitly reconstructed with global position information so as to relocate the position of the vehicle according to the implicitly reconstructed entrance. Among them, the process of implicitly reconstructing the parking lot entrance with global position information is the process of training a relocation model corresponding to the parking lot entrance based on the environmental images of multiple angles collected by the vehicle at the parking lot entrance, and the second global pose of the vehicle in the world coordinate system when the environmental image is collected. It can be understood that the relocation model obtained by training using environmental images collected at different locations and the global pose of the vehicle is used to characterize the implicit representation of the environment in which the vehicle is located at different locations.

[0100] Therefore, the preset position may include the entrance to the parking lot, and the multiple second environment images of the vehicle at the preset position are multi-angle environment images collected when the vehicle drives to the entrance of the parking lot. In addition, the preset position may include the first position mentioned above. The relocation model trained based on the multiple second environment images collected at the first position is the target relocation model. The preset position may also include other positions (for example, the entrance positions of other parking lots, or the positions of other entrances corresponding to the same parking lot). Of course, the relocation model trained using the multiple second environment images collected at other positions is an implicit representation of the real scenes corresponding to other positions.

[0101] In step S402, the preset relocation model is trained according to the plurality of second environment images and the second global pose to obtain the target relocation model corresponding to the preset position.

[0102] Among them, the second global pose can be understood as the true value in the model training process.

[0103] In this step, for each of the second environment images, the second environment image can be input into the preset relocation model to obtain the second position coordinates of each pixel in the second environment image output by the preset relocation model in the world coordinate system; according to the second global pose and the camera internal and external parameters corresponding to the plurality of second environment images, the second position coordinates of each pixel in the second environment image are reprojected into the image coordinate system to obtain the reprojection points corresponding to each pixel; for each of the pixel points, a reprojection error is constructed according to the reprojection point corresponding to the pixel point and the pixel point, and the preset relocation model is trained according to the reprojection error to obtain the target relocation model. For example, based on the back-propagation algorithm, the gradient back-propagation training of the model can be performed according to the reprojection error to obtain the target relocation model.

[0104] The preset relocation model includes a feature extraction network and a location prediction network connected to the feature extraction network. The feature extraction network may be, for example, a convolutional neural network, and the location prediction network may include a multi-layer perceptron.

[0105] When the present disclosure performs model training, in order to improve the efficiency of model training, the feature extraction network can be a network with fixed parameters obtained by pre-training. In this way, in the process of performing model training on the preset relocation model according to the reprojection error to obtain the target relocation model, the position prediction network can be model trained according to the reprojection error to obtain the target relocation model. In other words, the model parameters of the position prediction network in the preset relocation model can be adjusted based on the multi-angle environmental images collected at different positions, thereby improving the efficiency of model training.

[0106] It should be noted that the training of the target relocation model can be real-time training or offline training. Among them, real-time training refers to the real-time collection of multi-angle environmental images when the vehicle drives to the entrance of the parking lot, and the global posture of the vehicle when collecting environmental images is combined to perform model training to obtain the target relocation model, and then the target relocation model can be directly used to relocate the vehicle position. Offline training means that the model training has been completed before the model reasoning stage. For example, at 10 o'clock in the morning of a certain day, the vehicle drives to position A, and collects multi-angle environmental images of position A and the global posture of the vehicle. Then, based on the multi-angle environmental images of position A collected at 10 o'clock in the morning and the global posture of the vehicle, the relocation model corresponding to position A is pre-trained. When the vehicle drives to position A again at 2 o'clock in the afternoon, the relocation model corresponding to position A obtained by training in the morning can be directly obtained based on the corresponding relationship between position A and the relocation model, and the vehicle is relocated based on the relocation model.

[0107] The above method is used to implicitly reconstruct the preset position (such as the parking lot entrance) with global position information, and relocate the vehicle based on the relocalization model obtained by implicit reconstruction. It has high stability in complex scenes and supports accurate initialization of the vehicle position before entering the target area (such as the parking lot), which provides a better user experience. The target area refers to the area where the vehicle performs preset driving operations.

[0108] Figure 5 is a block diagram showing a vehicle control device according to an exemplary embodiment. Figure 5 As shown, the device comprises:

[0109] An acquisition module 501 is configured to acquire a first position where a vehicle is currently located and a first environment image of the vehicle in a driving direction at a current moment;

[0110] The determination module 502 is configured to determine a target relocation model corresponding to the first position; determine a second position where the vehicle is currently located by using the target relocation model according to the first environment image, wherein the position accuracy of the second position is higher than that of the first position;

[0111] The control module 503 is configured to control the vehicle to perform a preset driving operation according to the second position.

[0112] Optionally, the determination module 502 is configured to obtain the first position coordinates of each pixel point in the first environment image in the world coordinate system after inputting the first environment image into the target relocation model; and determine the second position where the vehicle is currently located based on the first position coordinates of each pixel point in the first environment image.

[0113] Optionally, the determination module 502 is configured to determine the camera pose corresponding to the first environmental image based on the first position coordinates of each pixel point in the first environmental image and the camera intrinsic parameters corresponding to the first environmental image; determine the current second position of the vehicle based on the camera pose and a preset external parameter relationship, and the preset external parameter relationship is the external parameter relationship between the vehicle and the image acquisition device that acquires the first environmental image.

[0114] Optionally, the target relocation model includes at least one, and the determination module 502 is configured to, when the target relocation model includes multiple, input the first environment image into each of the target relocation models, obtain the first position coordinates of each pixel point in the first environment image output by each of the target relocation models in the world coordinate system; and determine the current second position of the vehicle based on the first position coordinates of each pixel point in the first environment image output by each of the target relocation models.

[0115] Optionally, the second position includes a first global pose of the vehicle in a world coordinate system; the determination module 502 is configured to determine, for each target relocation model, a plurality of pending global poses of the vehicle according to the first position coordinates of each pixel point in the first environment image output by the target relocation model; determine the pose consistency of the plurality of pending global poses, wherein the pose consistency is used to characterize the degree of consistency of the plurality of pending global poses; determine the first global pose according to the pose consistency of the plurality of pending global poses corresponding to each target relocation model, wherein the first global pose is a pose determined according to the pending global pose with the highest pose consistency.

[0116] Optionally, the determining module 502 is configured to use the relocation model corresponding to the preset area where the first position is located as the target relocation model, and different preset areas correspond to different relocation models.

[0117] Optionally, Figure 6 is based on Figure 5The block diagram of a vehicle control device shown in the embodiment shown in FIG. Figure 6 As shown, the device also includes:

[0118] The model training module 504 is configured to obtain multiple second environment images of the vehicle at a preset position, and a second global posture of the vehicle in a world coordinate system when the vehicle captures the second environment image, different second environment images correspond to different image acquisition angles, and the preset position includes the first position; after model training is performed on a preset relocation model based on the multiple second environment images and the second global posture, the target relocation model corresponding to the preset position is obtained.

[0119] Optionally, the model training module 504 is configured to, for each second environment image, input the second environment image into the preset relocation model, and obtain the second position coordinates of each pixel point in the second environment image output by the preset relocation model in the world coordinate system; reproject the second position coordinates of each pixel point in the second environment image to the image coordinate system according to the second global pose and the camera internal and external parameters corresponding to the multiple second environment images, and obtain the reprojection points corresponding to each pixel point; for each pixel point, construct a reprojection error according to the reprojection point corresponding to the pixel point and the pixel point, and obtain the target relocation model after model training the preset relocation model according to the reprojection error.

[0120] Optionally, the preset relocation model includes a feature extraction network and a location prediction network connected to the feature extraction network;

[0121] The model training module 504 is configured to obtain the target relocation model after performing model training on the position prediction network according to the reprojection error.

[0122] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0123] The present disclosure also provides a computer-readable storage medium having computer program instructions stored thereon, and the program instructions, when executed by a processor, implement the steps of the vehicle control method provided by the present disclosure.

[0124] Figure 7 600 is a block diagram of a vehicle according to an exemplary embodiment. For example, vehicle 600 may be a hybrid vehicle, a non-hybrid vehicle, an electric vehicle, a fuel cell vehicle, or other types of vehicles. Vehicle 600 may be an autonomous vehicle, a semi-autonomous vehicle, or a non-autonomous vehicle.

[0125] Reference Figure 7 , the vehicle 600 may include various subsystems, for example, an infotainment system 610, a perception system 620, a decision control system 630, a drive system 640, and a computing platform 650. The vehicle 600 may also include more or fewer subsystems, and each subsystem may include multiple components. In addition, each subsystem and each component of the vehicle 600 may be interconnected by wire or wireless means.

[0126] In some embodiments, the infotainment system 610 may include a communication system, an entertainment system, and a navigation system, among others.

[0127] The perception system 620 may include several sensors for sensing information about the environment around the vehicle 600. For example, the perception system 620 may include a global positioning system (the global positioning system may be a GPS system, or a Beidou system or other positioning systems), an inertial measurement unit (IMU), a laser radar, a millimeter wave radar, an ultrasonic radar, and a camera.

[0128] The decision control system 630 may include a computing system, a vehicle controller, a steering system, a throttle, and a braking system.

[0129] The drive system 640 may include components that provide powered motion for the vehicle 600. In one embodiment, the drive system 640 may include an engine, an energy source, a transmission system, and wheels. The engine may be one or a combination of an internal combustion engine, an electric motor, and an air compression engine. The engine is capable of converting energy provided by the energy source into mechanical energy.

[0130] Some or all functions of the vehicle 600 are controlled by a computing platform 650. The computing platform 650 may include at least one processor 651 and a memory 652, and the processor 651 may execute instructions 653 stored in the memory 652.

[0131] The processor 651 may be any conventional processor, such as a commercially available CPU. The processor may also include a graphics processor (Graphic Process Unit, GPU), a field programmable gate array (Field Programmable Gate Array, FPGA), a system on chip (System on Chip, SOC), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC) or a combination thereof.

[0132] The memory 652 may be implemented by any type of volatile or nonvolatile memory device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0133] In addition to the instructions 653 , the memory 652 may also store data, such as road maps, route information, and data such as the location, direction, and speed of the vehicle. The data stored in the memory 652 may be used by the computing platform 650 .

[0134] In the embodiment of the present disclosure, the processor 651 can execute instruction 653 to complete all or part of the steps of the above-mentioned vehicle control method.

[0135] In another exemplary embodiment, a computer program product is also provided. The computer program product includes a computer program executable by a programmable device. The computer program has a code portion for executing the above-mentioned vehicle control method when executed by the programmable device.

[0136] Those skilled in the art may also understand that the various illustrative logical blocks and steps listed in the embodiments of the present application may be implemented by electronic hardware, computer software, or a combination of the two. Whether such functions are implemented by hardware or software depends on the specific application and the design requirements of the entire system. Those skilled in the art may use various methods to implement the functions described for each specific application, but such implementation should not be understood as exceeding the scope of protection of the embodiments of the present application.

[0137] In addition, the word "exemplary" is used herein to indicate serving as an example, instance, or diagram. Any aspect or design described as "exemplary" in this article is not necessarily understood to be advantageous compared to other aspects or designs. On the contrary, the use of the word exemplary is intended to present concepts in a specific way. As used herein, the term "or" is intended to represent an inclusive "or" rather than an exclusive "or". That is, unless otherwise specified or clear from the context, "X applies A or B" is intended to represent any one of the natural inclusive arrangements. That is, if X applies A; X applies B; or X applies both A and B, "X applies A or B" is satisfied under any of the aforementioned examples. In addition, unless otherwise specified or clearly pointed to a singular form from the context, the articles "one" and "an" as used in this application and the appended claims are generally understood to mean "one or more".

[0138] Likewise, although the present disclosure has been shown and described with respect to one or more implementations, equivalent variations and modifications will occur to those skilled in the art after reading and understanding the specification and drawings. The present disclosure includes all such modifications and variations and is limited only by the scope of the claims. In particular, with respect to the various functions performed by the components (e.g., elements, resources, etc.) described above, unless otherwise indicated, the terms used to describe such components are intended to correspond to any component (functionally equivalent) that performs the specific functions of the described components, even if the structure is not equivalent to the disclosed structure. In addition, although specific features of the present disclosure may have been disclosed with respect to only one of several implementations, such features may be combined with one or more other features of other implementations as may be desired and beneficial to any given or specific application. In addition, with respect to "including", "having", "having", "having", or variations thereof used in a specific embodiment or claim, such terms are intended to be inclusive in a manner similar to the term "comprising".

[0139] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any modification, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the appended claims.

[0140] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

[0141] In the above detailed description, reference is made to the accompanying drawings, which illustrate specific aspects of the present disclosure that can be practiced by way of illustration. In this regard, terms such as "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc., indicating directions or representing positional relationships, can be used with reference to the orientation of the described figures. Since the components of the described device can be positioned in a plurality of different orientations, directional terms can be used for illustrative purposes rather than restrictive. It should be understood that other aspects can be utilized and structural or logical changes can be made without departing from the concepts of the present disclosure. Therefore, the following detailed description should not be considered in a limiting sense.

[0142] It should be understood that, unless otherwise specifically noted, the features of some embodiments of the various present disclosures described herein may be combined with each other. As used herein, the term "and / or" includes any one of the related listed items and any combination of any two or more; similarly, "at least one of . . . " includes any one of the related listed items and any combination of any two or more.

[0143] It should be understood that, unless otherwise clearly specified and limited, the terms "joining", "attaching", "installing", "connecting", "connecting", "fixing" and the like used in the embodiments of the present disclosure should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral one; it can be a mechanical connection, an electrical connection, or communication with each other; it can be a direct connection, or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly limited. For ordinary technicians in this field, the specific meanings of the above terms in this article can be understood according to specific circumstances.

[0144] In addition, the term "above" used in relation to a component, element or material layer formed "above" or located "above" a surface may be used herein to indicate that the component, element or material layer is "indirectly" positioned (e.g., placed, formed, deposited, etc.) on the surface such that one or more additional components, elements or layers are arranged between the surface and the component, element or material layer. However, the term "above" used in relation to a component, element or material layer formed "above" or located "above" a surface may also optionally have a specific meaning: the component, element or material layer is "directly" positioned (e.g., placed, formed, deposited, etc.) on the surface, such as in direct contact with the surface.

[0145] Although terms such as "first", "second" and "third" can be used herein to describe various components, parts, regions, layers or sections, these components, parts, regions, layers or sections are not limited to these terms. On the contrary, these terms are only used to distinguish one component, component, region, layer or section from another component, component, region, layer or section. Therefore, without departing from the teachings of each example, the first component, component, region, layer or section mentioned in the examples described herein may also be referred to as the second component, component, region, layer or section. In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, the features defined as "first" and "second" may expressly or implicitly include at least one of the features. In the description herein, the meaning of "multiple" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0146] It should be understood that spatially relative terms such as "above", "upper", "below", and "lower" are used herein to describe the relationship of one element to another element shown in the figures. In addition to the orientation depicted in the drawings, such spatially relative terms are also intended to include different orientations of the device in use or operation. For example, if the device in the drawings is turned over, the element described as being "above" or "upper" relative to another element will be "below" or "lower" relative to the other element. Therefore, depending on the spatial orientation of the device, the term "above" includes both upper and lower orientations. The device may have other orientations (e.g., rotated 90 degrees or in other orientations), and the spatially relative terms used herein should be interpreted accordingly.

Claims

1. A vehicle control method, characterized in that: include: Acquire a first current position of the vehicle and a first environment image of the vehicle in a current driving direction; Determining a target relocation model corresponding to the first position; The target relocation model is an implicit reconstruction of the real scene where the first position is located, and different positions correspond to different relocation models; the target relocation model is a deep learning model obtained by pre-training; Determining, according to the first environment image, a second position where the vehicle is currently located by using the target relocation model, wherein the position accuracy of the second position is higher than that of the first position; controlling the vehicle to perform a preset driving operation according to the second position; The target relocation model is pre-trained in the following way: Acquire a plurality of second environment images of the vehicle at a preset position, and a second global position of the vehicle in a world coordinate system when the vehicle acquires the second environment image, wherein different second environment images correspond to different image acquisition angles, and the preset position includes the first position; After model training is performed on a preset relocation model according to the plurality of second environment images and the second global pose, the target relocation model corresponding to the preset position is obtained.

2. The method according to claim 1, characterized in that Determining the second position where the vehicle is currently located by using the target relocation model according to the first environment image includes: After inputting the first environment image into the target relocation model, obtaining the first position coordinates of each pixel point in the first environment image in the world coordinate system; The second position where the vehicle is currently located is determined according to the first position coordinates of each pixel point in the first environment image.

3. The method according to claim 2, characterized in that The determining, according to the first position coordinates of each pixel point in the first environment image, the second position where the vehicle is currently located comprises: Determine a camera pose corresponding to the first environment image according to a first position coordinate of each pixel point in the first environment image and a camera intrinsic parameter corresponding to the first environment image; The second position where the vehicle is currently located is determined according to the camera posture and a preset external parameter relationship, where the preset external parameter relationship is an external parameter relationship between the vehicle and an image acquisition device that acquires the first environment image.

4. The method according to claim 1, characterized in that: The target relocation model includes at least one, and determining the second position where the vehicle is currently located by using the target relocation model according to the first environment image includes: In the case where the target relocation model includes multiple ones, after the first environment image is input into each of the target relocation models, the first position coordinates of each pixel point in the first environment image respectively output by each of the target relocation models in the world coordinate system are obtained; The second position where the vehicle is currently located is determined according to the first position coordinates of each pixel point in the first environment image respectively output by each of the target relocation models.

5. The method according to claim 4, characterized in that The second position comprises a first global pose of the vehicle in a world coordinate system; Determining the second position of the vehicle according to the first position coordinates of each pixel point in the first environment image respectively output by each target relocation model comprises: For each of the target relocation models, determining a plurality of undetermined global poses of the vehicle according to the first position coordinates of each pixel point in the first environment image output by the target relocation model; Determining the pose consistency of the plurality of pending global poses, wherein the pose consistency is used to characterize the degree of consistency of the plurality of pending global poses; The first global pose is determined according to the pose consistency of the multiple pending global poses corresponding to each of the target relocation models, wherein the first global pose is a pose determined according to the pending global pose with the highest pose consistency.

6. The method according to claim 1, characterized in that Determining the target relocation model corresponding to the first position includes: The relocation model corresponding to the preset area where the first position is located is used as the target relocation model, and different preset areas correspond to different relocation models.

7. The method according to claim 1, characterized in that After training the preset relocation model according to the plurality of second environment images and the second global pose, obtaining the target relocation model corresponding to the preset position comprises: For each second environment image, after inputting the second environment image into the preset relocation model, obtain the second position coordinates of each pixel point in the second environment image output by the preset relocation model in the world coordinate system; Reprojecting the second position coordinates of each pixel point in the second environment image into an image coordinate system according to the second global pose and the camera internal and external parameters corresponding to the plurality of second environment images, to obtain a reprojection point corresponding to each pixel point; For each of the pixel points, a reprojection error is constructed according to the reprojection point corresponding to the pixel point and the pixel point, and the preset relocation model is trained according to the reprojection error to obtain the target relocation model.

8. The method according to claim 7, characterized in that The preset relocation model includes a feature extraction network and a position prediction network connected to the feature extraction network; After the preset relocation model is trained according to the reprojection error, obtaining the target relocation model comprises: After model training is performed on the position prediction network according to the reprojection error, the target relocation model is obtained.

9. A vehicle control device, characterized in that: include: An acquisition module is configured to acquire a first position where the vehicle is currently located and a first environment image of the vehicle in a driving direction at a current moment; A determination module is configured to determine a target relocation model corresponding to the first position; according to the first environment image, determine a second position where the vehicle is currently located by using the target relocation model, wherein the position accuracy of the second position is higher than that of the first position; the target relocation model is an implicit reconstruction of the real scene where the first position is located, and different positions correspond to different relocation models; the target relocation model is a pre-trained deep learning model; a control module, configured to control the vehicle to perform a preset driving operation according to the second position; The target relocation model is pre-trained in the following way: Acquire a plurality of second environment images of the vehicle at a preset position, and a second global position of the vehicle in a world coordinate system when the vehicle acquires the second environment image, wherein different second environment images correspond to different image acquisition angles, and the preset position includes the first position; After model training is performed on a preset relocation model according to the plurality of second environment images and the second global pose, the target relocation model corresponding to the preset position is obtained.

10. A vehicle, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to execute the steps of the method according to any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

12. A computer program product, characterized in that The invention comprises a computer program, which implements the steps of the method according to any one of claims 1 to 8 when being executed by a processor.

Images