Vehicle control method and device, vehicle and storage medium

Abstract

The present disclosure provides a vehicle control method and device, a vehicle and a storage medium. The vehicle comprises a camera. The method comprises: controlling the camera to capture an image of a scene in which the vehicle is located; obtaining projection information of a projection pattern in the image; determining deformation information corresponding to the projection pattern according to the projection information; and performing motion control on the vehicle according to the deformation information. Through the present disclosure, the projection information recognized can be used to assist in determining the scene information, so that the related scene information in the scene in which the vehicle is located can be effectively recognized, the accuracy of the vehicle motion control can be effectively improved, the safety of the vehicle operation can be improved, and the robustness and applicability of the vehicle control method can be improved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of intelligent driving, and in particular, to a vehicle control method and device, a vehicle, and a storage medium. BACKGROUND

[0002] With the development of intelligent driving technology, it is of great significance to identify the scene information in which the vehicle is located. Scene information identification (such as identifying the terrain of the driving road surface, obstacle information, etc.) can be used to assist in vehicle perception, obstacle avoidance, positioning, and control.

[0003] In related technologies, laser radar, ultrasonic sensors, and deep learning algorithms are usually used to identify the scene information in which the vehicle is located to assist in vehicle perception, obstacle avoidance, positioning, and control.

[0004] In this way, the identification of scene information is cumbersome, which may affect the robustness of the vehicle control method and thus the safety of vehicle operation. SUMMARY

[0005] The present disclosure aims to at least partially address one of the technical problems in the related art.

[0006] To this end, the present disclosure aims to provide a vehicle control method, device, vehicle, electronic device, and storage medium, which can use identified projection information to assist in determining scene information, thereby effectively assisting in identifying relevant scene information in the scene in which the vehicle is located, effectively improving the accuracy of vehicle motion control, improving the safety of vehicle operation, and improving the robustness and applicability of the vehicle control method.

[0007] The vehicle control method according to the first aspect of the present disclosure includes a camera device, and the method includes: controlling the camera device to capture an image of a scene in which the vehicle is located; obtaining projection information of a projection pattern in the image; determining deformation information corresponding to the projection pattern according to the projection information; and performing motion control on the vehicle according to the deformation information.

[0008] In some embodiments of the present disclosure, the number of camera devices is multiple, and the obtaining of the projection information of the projection pattern in the image includes:

[0009] determining information of a reference pixel point in a reference image, the reference image belonging to the multiple images;

[0010] analyzing and processing the information of the reference pixel point to obtain reference projection information of the projection pattern in the reference image, the reference projection information being projection information captured by a camera device that captures the reference image.

[0011] In some embodiments of the present disclosure, the determining the deformation information corresponding to the projection pattern according to the projection information comprises:

[0012] determining target projection information according to the reference projection information, the target projection information being projection information obtained by a camera capturing the projection pattern when capturing a target image, the target image being different from the reference image;

[0013] determining the deformation information corresponding to the projection pattern according to the reference projection information and the target projection information.

[0014] In some embodiments of the present disclosure, the determining the target projection information according to the reference projection information comprises:

[0015] determining the target projection information according to the reference projection information and a preset correspondence relationship;

[0016] The preset correspondence relationship comprises target projection information corresponding to the reference projection information, and the preset correspondence relationship is obtained by calibrating the projection pattern according to reference camera parameters corresponding to a camera capturing the reference image and target camera parameters of a camera capturing the target image.

[0017] In some embodiments of the present disclosure, the vehicle further comprises a projection device, and before the controlling the camera to capture the image of the scene where the vehicle is located, the method further comprises:

[0018] controlling the projection device to project the projection pattern to the scene.

[0019] In some embodiments of the present disclosure, the reference projection information is reference pixel coordinates based on a first camera coordinate system, and the first camera coordinate system is a coordinate system of a camera capturing the reference image.

[0020] The target projection information is target pixel coordinates based on a second camera coordinate system, and the second camera coordinate system is a coordinate system of a camera capturing the target image.

[0021] The target projection information is target pixel coordinates based on a second camera coordinate system, and the second camera coordinate system is a coordinate system of a camera capturing the target image.

[0022] In some embodiments of the present disclosure, the reference pixel point and the target pixel point respectively correspond to a position point in a scene space.

[0023] The determining the deformation information corresponding to the projection pattern according to the reference projection information and the target projection information comprises:

[0024] coordinate-convert the reference pixel coordinates according to the reference camera parameters to obtain first point coordinates of the position point based on a vehicle coordinate system, the vehicle coordinate system being a coordinate system in which the vehicle is located;

[0025] coordinate-convert the target pixel coordinates according to the target camera parameters to obtain second point coordinates of the position point based on the vehicle coordinate system;

[0026] determine deformation information corresponding to the projection pattern according to the first point coordinates and the second point coordinates.

[0027] In some embodiments of the present disclosure, the determining of the deformation information corresponding to the projection pattern according to the first point coordinates and the second point coordinates comprises:

[0028] generating point cloud data corresponding to the position point according to the first point coordinates and the second point coordinates;

[0029] determining deformation information corresponding to the projection pattern according to a plurality of point cloud data respectively corresponding to a plurality of position points in the scene.

[0030] In some embodiments of the present disclosure, the position point is located on the ground in the scene.

[0031] The motion control of the vehicle according to the deformation information comprises:

[0032] determining terrain information of the ground according to the deformation information;

[0033] controlling the motion of the vehicle according to the terrain information.

[0034] In some embodiments of the present disclosure, the determining of the terrain information of the ground according to the deformation information comprises:

[0035] determining ground flatness of the ground according to the deformation information; and / or

[0036] determining ground slope of the ground according to the deformation information; and / or

[0037] determining traffic sign information in the ground according to the deformation information, wherein the ground flatness, and / or the ground slope, and / or the traffic sign information are taken as the terrain information.

[0038] In some embodiments of the present disclosure, the position point is located on a target object in the scene,

[0039] The motion control of the vehicle according to the deformation information comprises:

[0040] According to the deformation information, determine the shape information and the position information of the target object;

[0041] According to the shape information and the position information, perform obstacle avoidance control on the vehicle.

[0042] The vehicle control method provided by the first aspect of the present disclosure can realize the use of the recognized projection information to assist in determining the scene information, thereby effectively assisting in recognizing the relevant scene information in the scene where the vehicle is located, effectively improving the accuracy of vehicle motion control, improving the safety of vehicle operation, and improving the robustness and applicability of the vehicle control method.

[0043] The vehicle control device provided by the second aspect of the present disclosure includes a camera device, and the device includes: an acquisition module configured to control the camera device to acquire an image of a scene where the vehicle is located; an obtaining module configured to obtain projection information of a projection pattern in the image; a determination module configured to determine deformation information corresponding to the projection pattern according to the projection information; and a control module configured to perform operation control on the vehicle according to the deformation information.

[0044] In some embodiments of the present disclosure, the number of camera devices is multiple.

[0045] The obtaining module is specifically configured to:

[0046] determine information of a reference pixel point in a reference image, the reference image belonging to the multiple images;

[0047] perform analysis processing on the information of the reference pixel point to obtain reference projection information of a projection pattern in the reference image, the reference projection information being projection information obtained by a camera device that acquires the reference image capturing the projection pattern.

[0048] In some embodiments of the present disclosure, the determination module includes:

[0049] a first determination submodule configured to determine target projection information according to the reference projection information, the target projection information being projection information obtained by a camera device that acquires a target image capturing the projection pattern, the target image belonging to the multiple images, and the reference image and the target image being different;

[0050] a second determining sub-module, configured to determine deformation information corresponding to the projection pattern according to the reference projection information and the target projection information.

[0051] In some embodiments of the present disclosure, the first determining sub-module is specifically configured to:

[0052] determine the target projection information according to the reference projection information and a preset correspondence relationship;

[0053] The preset correspondence relationship includes target projection information corresponding to the reference projection information, and the preset correspondence relationship is obtained by calibrating the projection pattern according to reference camera parameters corresponding to a camera device used to collect the reference image and target camera parameters of a camera device used to collect the target image.

[0054] In some embodiments of the present disclosure, the vehicle further includes a projection device, and the vehicle further includes:

[0055] a projection module, configured to control the projection device to project the projection pattern to the scene.

[0056] In some embodiments of the present disclosure, the reference projection information is reference pixel coordinates based on a first camera coordinate system, and the first camera coordinate system is a coordinate system of a camera device used to collect the reference image.

[0057] The target projection information is target pixel coordinates based on a second camera coordinate system, and the second camera coordinate system is a coordinate system of a camera device used to collect the target image.

[0058] The target projection information is target pixel coordinates based on a second camera coordinate system, and the second camera coordinate system is a coordinate system of a camera device used to collect the target image.

[0059] In some embodiments of the present disclosure, the reference pixel point and the target pixel point correspond to a position point in a scene space respectively.

[0060] The second determining sub-module is specifically configured to:

[0061] perform coordinate conversion on the reference pixel coordinates according to the reference camera parameters to obtain first point coordinates of the position point based on a vehicle coordinate system, and the vehicle coordinate system is a coordinate system of the vehicle.

[0062] perform coordinate conversion on the target pixel coordinates according to the target camera parameters to obtain second point coordinates of the position point based on the vehicle coordinate system.

[0063] determine deformation information corresponding to the projection pattern according to the first point coordinates and a plurality of second point coordinates.

[0064] In some embodiments of the present disclosure, the second determining sub-module is specifically configured to:

[0065] generate point cloud data corresponding to the position point according to the first point coordinate and the plurality of second point coordinates;

[0066] determine deformation information corresponding to the projection pattern according to a plurality of point cloud data respectively corresponding to a plurality of position points in the scene.

[0067] In some embodiments of the present disclosure, the position point is located on the ground in the scene.

[0068] The control module is specifically configured to:

[0069] determine terrain information of the ground according to the deformation information;

[0070] perform obstacle avoidance control on the vehicle according to the terrain information.

[0071] In some embodiments of the present disclosure, the control module is specifically configured to

[0072] determine ground flatness of the ground according to the deformation information; and / or

[0073] determine ground slope of the ground according to the deformation information; and / or

[0074] determine traffic sign information in the ground according to the deformation information, wherein the ground flatness, and / or the ground slope, and / or the traffic sign information are taken as the terrain information.

[0075] In some embodiments of the present disclosure, the position point is located on a target object in the scene,

[0076] The control module is specifically configured to:

[0077] determine shape information and position information of the target object according to the deformation information;

[0078] perform obstacle avoidance control on the vehicle according to the shape information and the position information.

[0079] The vehicle control device provided in the second aspect of the present disclosure can collect images of the scene where the vehicle is located through the camera device, acquire projection information of the projection pattern in the images, determine deformation information corresponding to the projection pattern according to the projection information, and perform motion control on the vehicle according to the deformation information, so as to assist in determining the scene information by using the recognized projection information, thereby effectively assisting in recognizing the relevant scene information in the scene where the vehicle is located, effectively improving the accuracy of the vehicle motion control, improving the safety of the vehicle operation, and improving the robustness and applicability of the vehicle control method.

[0080] The vehicle provided in the third aspect of the present disclosure includes a body, a plurality of camera devices configured to collect images of the scene, and a control device connected to the plurality of camera devices and configured to perform the vehicle control method according to any one of the above.

[0081] In some embodiments of the present disclosure, the vehicle further includes:

[0082] A projection device connected to the control device.

[0083] The control device is further configured to control the projection device to project a projection pattern to the scene, and the projection pattern is used for operation control of the vehicle.

[0084] In some embodiments of the present disclosure, the vehicle further includes:

[0085] A vehicle lamp configured to illuminate.

[0086] The projection device is integrated with the vehicle lamp.

[0087] The vehicle provided in the third aspect of the present disclosure includes a body, a plurality of camera devices configured to collect images of the scene, and a control device connected to the plurality of camera devices and configured to perform the vehicle control method according to any one of the above.

[0088] The electronic device provided in the fourth aspect of the present disclosure includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor performs the program to implement the vehicle control method according to the first aspect of the present disclosure.

[0089] The fifth aspect of the present disclosure provides a non-transitory computer-readable storage medium, which stores a computer program. The computer program is executed by a processor to implement the vehicle control method according to the first aspect of the present disclosure.

[0090] The sixth aspect of the present disclosure provides a computer program product. When the instructions in the computer program product are executed by a processor, the vehicle control method according to the first aspect of the present disclosure is performed.

[0091] Additional aspects and advantages of the present disclosure will be made apparent from the following description of embodiments, which proceeds with reference to the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0092] The above and / or additional aspects and advantages of the present disclosure will become apparent and be more readily understood from the following description, taken in conjunction with the accompanying drawings, in which:

[0093] Figure 1 is a flowchart of a vehicle control method according to an embodiment of the present disclosure;

[0094] Figure 2 is a schematic diagram of a random pattern according to an embodiment of the present disclosure;

[0095] Figure 3 is a schematic diagram of a sine wave pattern according to an embodiment of the present disclosure;

[0096] Figure 4 is a schematic diagram of a two-dimensional code pattern according to an embodiment of the present disclosure;

[0097] Figure 5 is a schematic diagram of a grid pattern according to an embodiment of the present disclosure;

[0098] Figure 6 is a flowchart of a vehicle control method according to another embodiment of the present disclosure;

[0099] Figure 7 is a flowchart of a vehicle control method according to another embodiment of the present disclosure;

[0100] Figure 8 is a schematic diagram of an application scenario of a vehicle control method according to an embodiment of the present disclosure;

[0101] Figure 9 is a schematic diagram of a vehicle control device according to an embodiment of the present disclosure;

[0102] Figure 10 is a schematic diagram of a vehicle control device according to another embodiment of the present disclosure;

[0103] Figure 11is a structural schematic diagram of a vehicle according to an embodiment of the present disclosure.

[0104] Figure 12 is a structural schematic diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0105] Embodiments of the present disclosure are described in detail below with reference to the accompanying drawings, examples of which are shown in the drawings, wherein the same or similar notations represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present disclosure and cannot be understood as a limitation of the present disclosure. On the contrary, the embodiments of the present disclosure include all changes, modifications and equivalents falling within the spirit and scope of the appended claims.

[0106] Figure 1 is a flowchart of a vehicle control method according to an embodiment of the present disclosure.

[0107] The present embodiment is exemplified by the vehicle control method being configured in a vehicle control device.

[0108] The vehicle control method in the present embodiment can be configured in a vehicle control device, which can be arranged in a server or also in a mobile terminal, and the present embodiment of the present disclosure does not limit this.

[0109] It should be noted that the execution subject of the present embodiment can be, for example, a central processing unit (CPU) in a server or a mobile terminal in hardware, and can be, for example, a related background service in a server or a mobile terminal in software, and the present disclosure does not limit this.

[0110] As shown in Figure 1 The vehicle control method comprises:

[0111] S101: Control the camera to collect an image of a scene in which the vehicle is located.

[0112] In some embodiments, before the camera is controlled to collect the image of the scene in which the vehicle is located, the projection device can be controlled to project a projection pattern to the scene.

[0113] In the present embodiment of the present disclosure, the projection device can be, for example, a light source with an optical modulation function arranged in the vehicle (the number of light sources can be adaptively configured according to the needs of the actual business scenario, and the present disclosure does not limit this), which can also be jointly integrated with the vehicle lamp. The wavelength band of the light source can be visible light or invisible light such as infrared light, and the present disclosure does not limit this.

[0114] In some other embodiments, the projection device can be integrated into one camera device, so that the camera device can realize both the projection function and the image capturing function, or the projection device can be separated from the vehicle, and no limitation is made in this regard.

[0115] The projection device can project a corresponding pattern to the scene where the vehicle is located, which can be referred to as a projection pattern. The projection pattern can be a grid pattern, a random coding pattern, a sine wave pattern, a triangular wave pattern, a two-dimensional code pattern, etc. (as shown in Figure 2 、 Figure 3 、 Figure 4 、 Figure 5 , Figure 2 is a schematic diagram of a random pattern according to an embodiment of the present disclosure, Figure 3 is a schematic diagram of a sine wave pattern according to an embodiment of the present disclosure, Figure 4 is a schematic diagram of a two-dimensional code pattern according to an embodiment of the present disclosure, Figure 5 is a schematic diagram of a grid pattern according to an embodiment of the present disclosure, and no limitation is made in this regard.

[0116] In this embodiment, the image of the scene where the vehicle is located can be obtained by capturing the projection pattern projected by the projection device to the scene, that is, the projection device can project a set projection pattern to the projection area in the scene, and then the camera device captures an image of the scene picture containing the projection area, which can be referred to as the image of the scene where the vehicle is located.

[0117] The number of images of the scene where the vehicle is located can be one or more, and no limitation is made in this regard.

[0118] In some embodiments, one camera device can be configured to capture multiple images of the scene, or multiple camera devices can be configured to capture images of the scene respectively to obtain multiple images, and no limitation is made in this regard.

[0119] S102: Obtain projection information of the projection pattern in the image.

[0120] After the camera device is controlled to capture the image of the scene where the vehicle is located, the projection information of the projection pattern in the image can be obtained.

[0121] The image of the scene picture containing the projection area captured by the camera device can contain information related to the projection pattern, which can be referred to as projection information. The projection information can be specifically, for example, pixel information of the projection pattern, deformation information of the projection pattern, brightness information of the projection pattern, etc., and no limitation is made in this regard.

[0122] In some embodiments, the information of the projected pattern in the image can be obtained by image analysis of the image to obtain the projection information of the projected pattern in the image, for example, the projected pattern contained in the image can be analyzed to obtain a plurality of pixel coordinates of the projected pattern, and the plurality of pixel coordinates can be taken as the projection information, and no limitation is made thereto.

[0123] Alternatively, the image can also be input into a pre-trained deep learning model to obtain a plurality of pixel coordinates of the projected pattern in the image output by the deep learning model, and the plurality of pixel coordinates can be taken as the projection information, or any other possible way of obtaining the projection information of the projected pattern in the image can also be used, and no limitation is made thereto.

[0124] S103: determining the deformation information corresponding to the projected pattern according to the projection information.

[0125] After the image analysis of the image to obtain the projection information of the projected pattern in the image, the deformation information corresponding to the projected pattern can be determined according to the projection information.

[0126] In the embodiment, when the projection device is controlled to project the projected pattern into the scene where the vehicle is located, there can be an obstacle in the scene where the vehicle is located, so that the projected pattern projected by the projection device will be deformed to different degrees when falling on the obstacle due to the influence of the obstacle. Accordingly, the information for describing the deformation of the projected pattern can be referred to as deformation information, and the deformation information can specifically be, for example, the offset of the pixel coordinates of the projected pattern, the offset angle of the projected pattern, the offset size of the projected pattern, and no limitation is made thereto.

[0127] In some embodiments, the deformation information corresponding to the projected pattern can be determined according to the projection information by comparing the projected pattern with the projected pattern projected by the projection device without deformation to obtain the change information of the projected pattern compared with the projected pattern without deformation, and taking the change information as the deformation information.

[0128] For example, taking the pattern projected by the projection device as a grid pattern, the deformed grid pattern and the grid pattern without deformation can be compared to judge the offset angle information of the grid, the offset size information of the grid, and the brightness information of the grid pattern, and the plurality of information obtained above can be taken as the deformation information, and no limitation is made thereto.

[0129] S104: performing motion control on the vehicle according to the deformation information.

[0130] After the deformation information corresponding to the projected pattern is determined according to the projection information, the motion control can be performed on the vehicle according to the deformation information.

[0131] That is to say, after the deformation information corresponding to the projection pattern is obtained, different types of motion control can be performed on the vehicle according to the deformation information of the projection pattern and the scene in which the vehicle is located. For example, in a parking scene, a driving scene, etc., a control strategy suitable for the current scene is given in combination with the deformation information of the projection pattern.

[0132] In some embodiments, the scene of the vehicle is judged, for example, whether there is an obstacle in the scene in which the vehicle is located and the position of the obstacle can be judged, and then the motion control of the vehicle can be performed, for example, the vehicle can be controlled to evade (for example, to turn left, to turn right, to make a U-turn, to change lanes) according to the position of the obstacle in the scene in which the vehicle is located, and the vehicle can also be controlled to slow down according to the flatness of the ground in the scene in which the vehicle is located. The corresponding control of the vehicle according to the deformation information described above can be referred to as motion control, which is not limited in this regard.

[0133] In some embodiments, the motion control of the vehicle according to the deformation information can be the motion control of the vehicle according to the brightness information of the image, for example, if the ground in the scene in which the vehicle is located is flat, the brightness of the projection pattern in the image is consistent, and if the ground in the scene in which the vehicle is located is not flat, the brightness of the projection pattern in the image can change accordingly. At this time, the brightness information corresponding to the projection pattern can be analyzed to control the motion of the vehicle.

[0134] For example, if the deformation information of the projection pattern indicates that the pattern projected onto a certain place of the road becomes dark, it can be considered that there is a depression at that place, and the vehicle can be controlled to evade accordingly. If the deformation information of the projection pattern indicates that the pattern projected onto a certain place of the road becomes bright, it can be considered that there is a convex obstacle at that place, and the vehicle can be controlled to evade accordingly, which is not limited in this regard.

[0135] In this embodiment, the image of the scene in which the vehicle is located is captured by controlling the camera, the projection information of the projection pattern in the image is obtained, the deformation information corresponding to the projection pattern is determined according to the projection information, and the motion control of the vehicle is performed according to the deformation information. The projection information recognized can be used to assist in determining the scene information, so as to effectively assist in recognizing the related scene information in the scene in which the vehicle is located, effectively improve the accuracy of the motion control of the vehicle, improve the safety of the operation of the vehicle, and improve the robustness and applicability of the vehicle control method.

[0136] Figure 6 is a flowchart of a vehicle control method according to another embodiment of the present disclosure.

[0137] As shown in Figure 6 , the vehicle control method comprises:

[0138] S601: Control the camera to collect an image of a scene where the vehicle is located.

[0139] The description of S601 can be specifically referred to the above embodiments, which will not be repeated here.

[0140] S602: Determine the information of a reference pixel point in a reference image, the reference image belongs to multiple images.

[0141] The vehicle can include multiple cameras, which are used to collect multiple images of the scene where the vehicle is located. For example, the multiple cameras can include a first camera (which is a camera for collecting a reference image) and a second camera (which is a camera for collecting a target image). The images collected by the first camera and the second camera are different, i.e., the image collected by the first camera can be referred to as a reference image, and the image collected by the second camera can be referred to as a target image. The reference image and the target image can have a corresponding association relationship, which can be used to assist in controlling the vehicle. However, this is not limited.

[0142] In the embodiments of the present disclosure, at least part of the cameras can be configured with different collection angles. Therefore, when multiple cameras are used to collect corresponding images, the multiple cameras can be controlled to collect images at corresponding multiple collection angles (the multiple images in the embodiments of the present disclosure can include images of the scene collected by each camera), or other different camera parameters (such as camera intrinsic parameters or camera extrinsic parameters) can be configured for at least part of the cameras. Therefore, the multiple cameras can be controlled to collect images at corresponding multiple camera parameters, so as to capture multiple images of the scene where the vehicle is located based on different camera parameters, thereby effectively improving the control effect of the vehicle based on the images of the scene where the vehicle is located.

[0143] The image collected by the first camera (which is a camera for collecting a reference image) can be referred to as a reference image. The number of reference images can be one or more, which is not limited.

[0144] In the embodiments, the reference image can be obtained by capturing a projection pattern projected by a projection device to the scene, i.e., the projection device can project a set projection pattern to a projection area in the scene, and then the camera captures a pattern of the scene picture containing the projection area, which can contain information related to the projection image.

[0145] The reference image can include a plurality of pixel points, which can be referred to as reference pixel points. Correspondingly, information used to describe the reference pixel points can be referred to as reference pixel point information. The reference pixel point information can include information of the pixel points themselves and information related to the projection pattern around the pixel points, and the like. For example, the reference pixel point information can include reference pixel point coordinates based on the first camera coordinate system, and the like. No limitation is imposed herein.

[0146] In some embodiments, determining the reference pixel point information in the reference image can include determining a reference pixel point in the reference image and determining reference pixel point coordinates of the reference pixel point based on the first camera coordinate system. The reference pixel point coordinates can be used as the reference pixel point information. No limitation is imposed herein.

[0147] S603: Analyzing the reference pixel point information to obtain reference projection information of the projection pattern in the reference image. The reference projection information is projection information of the projection pattern captured by the camera device that collects the reference image.

[0148] The reference projection information is reference pixel coordinates based on the first camera coordinate system. The first camera coordinate system is a coordinate system of the camera device that collects the reference image.

[0149] After the reference pixel point information in the reference image is determined, the reference pixel point information can be analyzed to obtain reference projection information of the projection pattern in the reference image. The reference projection information is projection information of the projection pattern captured by the camera device that collects the reference image. In this way, the accuracy and reliability of the reference projection information can be effectively improved, so that the target projection information and the deformation information can be more accurately determined based on the reference projection information.

[0150] That is, after the reference pixel point information in the reference image is determined, the reference pixel point information can be analyzed to obtain reference pixel coordinates based on the first camera coordinate system. The coordinates can be used as the reference projection information. No limitation is imposed herein.

[0151] S604: Determining target projection information based on the reference projection information. The target projection information is projection information of the projection pattern captured by the camera device that collects the target image. The target image belongs to a plurality of images, and the reference image and the target image are different.

[0152] After the reference pixel point information is analyzed to obtain the reference projection information of the projection pattern in the reference image, the target projection information can be determined based on the reference projection information. In this way, the accuracy of the determination of the target pixel information can be effectively improved based on the reference projection information, and the reference value of the target pixel information in the vehicle control scenario can be improved.

[0153] Among them, the target projection information is the projection information obtained by the camera device that captures the projection pattern of the target image. The target image belongs to multiple images, and the reference image and the target image are different.

[0154] The image captured by the second camera (which is a camera device for acquiring target images) can be referred to as the target image. The number of target images can be one or more, and there is no limit to this.

[0155] Among them, the target projection information is the target pixel coordinates based on the second camera coordinate system, which is the coordinate system of the camera device that acquires the target image.

[0156] In this embodiment, determining the target projection information based on the reference projection information can be achieved by determining the reference pixel coordinates of the first camera coordinate system based on the reference pixel coordinates of the first camera coordinate system.

[0157] Optionally, in some embodiments, the target projection information is determined based on the reference projection information. This can be done by determining the target projection information based on the reference projection information and a preset correspondence. Since the target projection information is determined based on the preset correspondence, the accuracy and reliability of the target projection information can be effectively improved. This can effectively assist in improving the accuracy of the target pixel information determination and enhance the reference value of the target pixel information in scene modeling.

[0158] The preset correspondence includes target projection information corresponding to the reference projection information. The preset correspondence is obtained by calibrating the projection pattern in advance based on the reference camera parameters corresponding to the camera device that acquires the reference image and the target camera parameters of the camera device that acquires the target image.

[0159] For example, in some embodiments, the camera device for acquiring the reference image and the camera device for acquiring the target image can be calibrated to obtain a corresponding preset relationship, which is:

[0160]

[0161] Among them, K _cam It is the intrinsic parameter matrix of the camera device, T _cam_vehicle It is the extrinsic parameter matrix of the camera device, P _i It is a point in the vehicle coordinate system (i.e., a position point in the scene based on the main coordinates of the vehicle coordinate system), P -cam These are the coordinates of the location point in the camera device coordinate system (P in the first camera coordinate system). -cam Corresponding to the reference pixel coordinates, P in the second camera coordinate system -cam (corresponding to target pixel coordinates), UV _iis the coordinate of the projection pattern (i.e., projection information, when the projection information is captured by the first camera, it can be referred to as reference projection information, when the projection information is captured by the second camera, it can be referred to as target projection information).

[0162] In some other embodiments, the projection device can also be regarded as a camera, and the projection device can be calibrated to obtain a preset relationship, which is:

[0163]

[0164] wherein K _proj is an intrinsic matrix of the projection device, T _proj_vehicle is an extrinsic matrix of the projection device, P _i is a point in the vehicle coordinate system, P _proj is the coordinate of the point in the coordinate system of the projection device, and UV _i is the coordinate of the projection pattern.

[0165] After the above preset relationship is determined, the coordinate UV _i of the projection pattern captured by the second camera can be obtained according to the above preset relationship, and the coordinate is taken as the target projection information.

[0166] In some other embodiments, the corresponding point matching method can also be used to determine the target pixel coordinate based on the second camera coordinate system according to the target projection information, and the target pixel coordinate is taken as the target projection information, which is not limited.

[0167] For example, for a reference pixel coordinate in the first camera coordinate system, since the reference projection information of the projection pattern is projected around UV _a , the reference pixel coordinate UV _a in the second camera coordinate system can be determined by template matching, intensity matching, etc. _b In addition, since the optical model of the projection device and the camera is consistent, the projection device can be regarded as a camera participating in the point-to-point matching, and thus all pixel points are matched to obtain the reference pixel coordinate UV _a in the second camera coordinate system. _b The matching target pixel coordinate UV

[0168] After the above corresponding point matching method is used to determine the target projection information (target pixel coordinate based on the second camera coordinate system) according to the reference projection information (reference pixel coordinate based on the first camera coordinate system), the matching point group IR _i (UV _a ,UV _b, UV _c ,...) are recorded, wherein a, b... represent different cameras, and one point group should contain at least 2 points, and no limitation is made to this.

[0169] S605: According to the reference projection information and the target projection information, the deformation information corresponding to the projection pattern is determined.

[0170] Optionally, in some embodiments, as shown in Figure 7 , the method further includes: Figure 7 is a schematic diagram according to another embodiment of the disclosure, and the method further includes:

[0171] S701: The reference pixel coordinates are converted according to the reference camera parameter to obtain first point coordinates of the position point based on the vehicle coordinate system.

[0172] After the reference pixel coordinates are converted according to the reference camera parameter, the coordinates of the position point based on the vehicle coordinate system can be obtained, which can be referred to as the first point coordinates.

[0173] In this embodiment, the reference pixel coordinates are converted according to the reference camera parameter to obtain the first point coordinates of the position point based on the vehicle coordinate system, which can be determined according to the association relationship between the reference camera parameter, the reference pixel coordinates and the first point coordinates, and the association relationship is as follows:

[0174] uv a = K a · T a · P i ;

[0175] P i is the first point coordinates of the position point based on the vehicle coordinate system, T a is the extrinsic calibration matrix of the first camera, K a is the intrinsic calibration matrix of the first camera, the extrinsic calibration matrix T a and the intrinsic calibration matrix K a can be referred to as the reference camera parameter mentioned above, and uv a represents the reference pixel coordinates of the position point based on the first camera coordinate system.

[0176] S702: The target pixel coordinates are converted according to the target camera parameter to obtain second point coordinates of the position point based on the vehicle coordinate system.

[0177] After the target pixel coordinates are converted according to the target camera parameter, the coordinates of the position point based on the vehicle coordinate system can be obtained, which can be referred to as the second point coordinates.

[0178] In this embodiment, the target pixel coordinates are converted according to the target camera parameters to obtain the second point coordinates of the position point based on the vehicle coordinate system. The second point coordinates of the position point based on the vehicle coordinate system can be determined according to the association between the target camera parameters, the target pixel coordinates and the second point coordinates, and the association is as follows:

[0179] uv b = K b · T b · p i ;

[0180] wherein P i is the second point coordinates of the position point based on the vehicle coordinate system, T b is the external parameter calibration matrix of the second camera, K b is the internal parameter calibration matrix of the second camera, the external parameter calibration matrix T b and the internal parameter calibration matrix K b can be referred to as the target camera parameters described above, and uv b represents the reference pixel coordinates of the position point based on the second camera coordinate system.

[0181] S703: Determine the deformation information corresponding to the projection pattern according to the first point coordinates and the plurality of second point coordinates.

[0182] Optionally, in some embodiments, determining the deformation information corresponding to the projection pattern according to the first point coordinates and the plurality of second point coordinates can be generating point cloud data corresponding to the position point according to the first point coordinates and the plurality of second point coordinates, and determining the deformation information corresponding to the projection pattern according to the plurality of point cloud data corresponding to the plurality of position points in the scene. Since the deformation information corresponding to the projection pattern is determined according to the point cloud data corresponding to the position point, a new dimension is provided for the determination of the deformation information, thereby effectively improving the accuracy and reliability of the deformation information.

[0183] wherein the set of point data can be referred to as point cloud data.

[0184] In some embodiments, generating point cloud data corresponding to the position point according to the first point coordinates and the plurality of second point coordinates can be generating a point cloud map corresponding to the position point according to the first subject coordinates and the plurality of second subject coordinates, and taking the point cloud map as the point cloud data corresponding to the position point, without limitation.

[0185] After the point cloud data corresponding to the position point is generated according to the first point coordinates and the plurality of second point coordinates, the deformation information corresponding to the projection pattern can be determined according to the plurality of point cloud data corresponding to the plurality of position points in the scene.

[0186] In this embodiment, a three-dimensional model of the scene can be constructed according to the plurality of point cloud data corresponding to the plurality of position points in the scene, so as to determine the deformation information corresponding to the projection pattern through the three-dimensional model of the scene, and no limitation is made in this regard.

[0187] Of course, other arbitrary possible ways can also be adopted to determine the deformation information corresponding to the projection pattern according to the first point coordinate and the plurality of second point coordinates, such as artificial intelligence, engineering, mathematical analysis, etc., and no limitation is made in this regard.

[0188] In this embodiment, the first point coordinate of the position point based on the vehicle coordinate system is obtained by performing coordinate conversion on the reference pixel coordinate according to the reference camera parameter, the vehicle coordinate system is the coordinate system in which the vehicle is located, the second point coordinate of the position point based on the vehicle coordinate system is obtained by performing coordinate conversion on the target pixel coordinate according to the target camera parameter, and the deformation information corresponding to the projection pattern is determined according to the first point coordinate and the plurality of second point coordinates, which can effectively improve the accuracy of the deformation information, so that the obstacles in the scene where the vehicle is located can be judged based on the deformation information, and the control effect of the vehicle can be effectively improved.

[0189] S606: Determine the ground terrain information according to the deformation information.

[0190] After the projection information corresponding to the projection pattern is determined, the ground terrain information can be determined according to the deformation information.

[0191] Optionally, in some embodiments, determining the ground terrain information according to the deformation information can be determining the ground flatness according to the deformation information, and / or determining the ground slope according to the deformation information, and / or determining the traffic sign information in the ground according to the deformation information, wherein the ground flatness, and / or the ground slope, and / or the traffic sign information are taken as the terrain information, thereby effectively improving the accuracy of the terrain information determination, so that the motion control effect of each link of the vehicle driving can be improved based on the terrain information.

[0192] S607: Perform motion control on the vehicle according to the terrain information.

[0193] In some embodiments, performing motion control on the vehicle according to the terrain information includes: controlling the motion speed of the vehicle according to the determined ground flatness to improve the comfort of driving; providing a reference for acceleration and torque control in the motion of the vehicle according to the determined ground slope; and providing the determined traffic sign information (such as lane lines, stop lines, arrows, etc.) to support the driving decision of the vehicle turning, stopping, etc.

[0194] In this embodiment, the ground topography information is determined according to the deformation information, and the vehicle is controlled according to the ground topography information. Since the vehicle is controlled according to the ground topography information, the vehicle can make more accurate decisions when facing different ground topography information, and the safety of vehicle operation and the control effect of the vehicle are effectively improved.

[0195] Optionally, in another embodiment, the vehicle is controlled according to the deformation information, the shape information and the position information of the target object are determined, and the vehicle is controlled according to the shape information and the position information. Thus, the perception, obstacle avoidance and control of the vehicle can be significantly improved, the obstacle avoidance effect of the vehicle in different operation scenarios can be effectively improved, and the driving comfort of the vehicle can be effectively improved.

[0196] The target object refers to an obstacle in the scene where the vehicle is located (for example, a road curb, a wall, a tree trunk, etc.), and the information reflecting the shape of the target object can be referred to as shape information (the shape information can be specifically, for example, the height, width and concave degree of the obstacle), and the information reflecting the position of the target object can be referred to as position information (the position information can be specifically, for example, the direction of the obstacle and the distance of the obstacle).

[0197] That is, the shape information and the position information of the obstacle can be determined according to the deformation information, and the vehicle can be controlled according to the shape information and the position information in different vehicle operation scenarios, for example:

[0198] (1) In a parking scenario, the shape information and the position information of the obstacle including a wheel arch, a road curb, a wall, a tree trunk, etc. are determined according to the deformation information, so as to help the vehicle to reasonably avoid obstacles and achieve precise parking.

[0199] (2) In a driving scenario, the shape information and the position information of the obstacle including a road edge, a manhole cover, a road pit, a fence and a close-range vehicle are determined according to the deformation information, so as to help the vehicle to reasonably avoid obstacles and improve driving comfort.

[0200] In this embodiment, by controlling the camera to collect the image of the scene where the vehicle is located, after determining the information of the reference pixel points in the reference image, the information of the reference pixel points can be analyzed and processed to obtain the reference projection information of the projection pattern in the reference image. The reference projection information is the projection information obtained by the camera capturing the projection pattern. Thus, the accuracy and reliability of the reference projection information can be effectively improved, so that the target projection information and the deformation information can be more accurately determined based on the reference projection information. After analyzing and processing the information of the reference pixel points to obtain the reference projection information of the projection pattern in the reference image, the target projection information can be determined according to the reference projection information. Thus, the accuracy of the determination of the target pixel information can be effectively improved based on the reference projection information, and the referenceability of the target pixel information in the vehicle control scene can be improved. According to the reference projection information and the target projection information, the deformation information corresponding to the projection pattern is determined. According to the deformation information, the terrain information of the ground is determined. According to the terrain information, the vehicle is controlled to move. Since the vehicle is controlled according to the terrain information, the vehicle can make more accurate decisions when facing different terrain information, effectively improving the safety of vehicle operation and the control effect of the vehicle

[0201] In this embodiment, as shown in Figure 8 , Figure 8 is a schematic diagram of an application scenario of a vehicle control method in an embodiment of the present disclosure, which includes a vehicle (a coordinate system in which the vehicle is located, i.e., a vehicle coordinate system, which can be specifically, for example, a coordinate system established with a certain point on the vehicle as the origin), a projection device for projecting a projection pattern to the environment where the vehicle is located (as shown by the dashed line in the figure, the projection pattern is projected into the dashed box at a projection angle), and a camera device composed of a first camera (a coordinate system in which the first camera is located, i.e., a first camera coordinate system, which can be specifically, for example, a coordinate system established with the focal center of the first camera as the origin), a second camera (a coordinate in which the second camera is located, i.e., a second camera coordinate system, which can be specifically, for example, a coordinate system established with the focal center of the second camera as the origin), and the like. The camera device is used to capture multiple images of the projection pattern projected by the projection device, and then the motion control of the vehicle can be realized through the analysis and processing of the multiple images.

[0202] Figure 9 is a structural schematic diagram of a vehicle control device according to an embodiment of the present disclosure.

[0203] The vehicle includes a camera device.

[0204] As shown in Figure 9 , the vehicle control device 90 includes:

[0205] The acquisition module 901 is used to control the camera device to acquire images of the scene where the vehicle is located;

[0206] The acquisition module 902 is used to acquire the projection information of the projected pattern in the image;

[0207] The determination module 903 is used to determine the deformation information corresponding to the projected pattern based on the projection information;

[0208] The control module 904 is used to control the operation of the vehicle based on deformation information.

[0209] In some embodiments of this disclosure, the number of camera devices is multiple;

[0210] Specifically, the acquisition module 902 is used for:

[0211] Determine the information of reference pixels in a reference image, which belongs to multiple images;

[0212] The information of the reference pixels is parsed and processed to obtain the reference projection information of the projected pattern in the reference image. The reference projection information is the projection information obtained by the camera device that acquires the reference image and captures the projected pattern.

[0213] In some embodiments of this disclosure, such as Figure 10 As shown, Figure 10 This is a schematic diagram of the structure of a vehicle control device according to another embodiment of the present disclosure, wherein the determining module 903 includes:

[0214] The first determining submodule 9031 is used to determine the target projection information based on the reference projection information. The target projection information is the projection information obtained by the camera device that acquires the target image to capture the projection pattern. The target image belongs to multiple images, and the reference image and the target image are different.

[0215] The second determining submodule 9032 is used to determine the deformation information corresponding to the projection pattern based on the reference projection information and the target projection information.

[0216] In some embodiments of this disclosure, the first determining submodule 9031 is specifically used for:

[0217] The target projection information is determined based on the reference projection information and the preset correspondence.

[0218] The preset correspondence includes target projection information corresponding to the reference projection information. The preset correspondence is obtained by calibrating the projection pattern in advance based on the reference camera parameters corresponding to the camera device that acquires the reference image and the target camera parameters of the camera device that acquires the target image.

[0219] In some embodiments of the present disclosure, the vehicle further comprises a projection device, such as Figure 10 As shown, the device 90 further comprises:

[0220] A projection module 905 is configured to control the projection device to project the projection pattern to the scene.

[0221] In some embodiments of the present disclosure, wherein,

[0222] The reference projection information is reference pixel coordinates based on a first camera coordinate system, and the first camera coordinate system is a coordinate system of a camera device that captures the reference image.

[0223] The target projection information is target pixel coordinates based on a second camera coordinate system, and the second camera coordinate system is a coordinate system of a camera device that captures the target image.

[0224] In some embodiments of the present disclosure, the reference pixel points and the target pixel points correspond to position points in the scene space respectively.

[0225] The second determination submodule 9032 is specifically configured to:

[0226] According to the reference camera parameters, the reference pixel coordinates are converted to obtain first point coordinates of the position points based on a vehicle coordinate system, and the vehicle coordinate system is a coordinate system of the vehicle.

[0227] According to the target camera parameters, the target pixel coordinates are converted to obtain second point coordinates of the position points based on the vehicle coordinate system.

[0228] According to the first point coordinates and the plurality of second point coordinates, deformation information corresponding to the projection pattern is determined.

[0229] In some embodiments of the present disclosure, the second determination submodule 9032 is specifically configured to:

[0230] According to the first point coordinates and the plurality of second point coordinates, point cloud data corresponding to the position points is generated.

[0231] According to a plurality of point cloud data respectively corresponding to a plurality of position points in the scene, deformation information corresponding to the projection pattern is determined.

[0232] In some embodiments of the present disclosure, the position points are located on the ground in the scene.

[0233] The control module 904 is specifically configured to:

[0234] According to the deformation information, terrain information of the ground is determined.

[0235] According to the terrain information, the vehicle is controlled to avoid obstacles.

[0236] In some embodiments of the present disclosure, the control module 904 is specifically configured to

[0237] determine the ground flatness of the ground according to the deformation information; and / or

[0238] determine the ground slope of the ground according to the deformation information; and / or

[0239] determine the traffic sign information in the ground according to the deformation information, wherein the ground flatness, and / or the ground slope, and / or the traffic sign information are taken as the terrain information.

[0240] In some embodiments of the present disclosure, the position point is located on the target object in the scene,

[0241] wherein the control module 904 is specifically configured to:

[0242] determine the shape information and the position information of the target object according to the deformation information;

[0243] perform obstacle avoidance control on the vehicle according to the shape information and the position information.

[0244] It should be noted that the foregoing explanation and description of the vehicle control method embodiment are also applicable to the vehicle control device of the embodiment, which will not be described here.

[0245] In the present embodiment, the image of the scene where the vehicle is located is collected by controlling the camera device, and the projection information of the projection pattern in the image is obtained, the deformation information corresponding to the projection pattern is determined according to the projection information, and the vehicle is controlled to move according to the deformation information. The use of the recognized projection information to assist in determining the scene information can effectively assist in recognizing the relevant scene information in the scene where the vehicle is located, effectively improve the accuracy of vehicle motion control, improve the safety of vehicle operation, and improve the robustness and applicability of the vehicle control method.

[0246] Figure 11 is a structural schematic diagram of a vehicle according to an embodiment of the present disclosure, which comprises.

[0247] As shown in Figure 11 , the vehicle 110 comprises:

[0248] a body 1101;

[0249] a plurality of camera devices 1102 configured to collect images of a scene;

[0250] a control device 1103 connected to the plurality of camera devices respectively, configured to perform any of the vehicle control methods described above.

[0251] In some embodiments of the present disclosure, the vehicle 110 further comprises:

[0252] The projection device 1104 connected with the control device 1103;

[0253] The control device 1103 is further configured to control the projection device 1104 to project a projection pattern to the scene, and the projection pattern is used to control the operation of the vehicle 110.

[0254] In some embodiments of the present disclosure, the vehicle 110 further comprises:

[0255] The vehicle light 1105 is configured to illuminate.

[0256] The projection device 1104 is integrated with the vehicle light 1105.

[0257] It should be noted that the above description of the vehicle control method also applies to the vehicle 110 of this embodiment, which will not be described here.

[0258] In this embodiment, by configuring the vehicle with a body, a plurality of camera devices configured to capture images of the scene, and a control device connected with each of the camera devices, the control device is configured to execute the vehicle control method as described above, the projection information recognized can be used to assist in determining the scene information, thereby effectively assisting in recognizing the relevant scene information in the scene where the vehicle is located, effectively improving the accuracy of vehicle motion control, improving the safety of vehicle operation, and improving the robustness and applicability of the vehicle control method.

[0259] Figure 12 A structural schematic diagram of an electronic device according to an embodiment of the present disclosure is provided.

[0260] The electronic device comprises:

[0261] The electronic device comprises:

[0262] The processor 1202 executes the program to implement the vehicle control method provided in the above embodiments.

[0263] In a possible implementation, the electronic device further comprises:

[0264] The communication interface 1203 is configured to communicate between the memory 1201 and the processor 1202.

[0265] The memory 1201 is configured to store a computer program executable on the processor 1202.

[0266] The memory 1201 can include a high-speed RAM memory, and can also include a non-volatile memory, such as at least one disk memory.

[0267] The processor 1202 is configured to implement the vehicle control method according to the above-described embodiments when the processor 1202 executes a program.

[0268] If the memory 1201, the processor 1202 and the communication interface 1203 are implemented independently, the communication interface 1203, the memory 1201 and the processor 1202 can be connected with each other through a bus and complete communication between each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, Figure 12 In the figure, only one thick line is used to represent the bus, but it does not mean that there is only one bus or only one type of bus.

[0269] Optionally, in a specific implementation, if the memory 1201, the processor 1202 and the communication interface 1203 are integrated on a chip, the memory 1201, the processor 1202 and the communication interface 1203 can complete communication between each other through an internal interface.

[0270] The processor 1202 can be a Central Processing Unit (CPU), or an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present disclosure.

[0271] The embodiment further provides a computer readable storage medium, which stores a computer program. The program is executed by the processor to implement the vehicle control method as described above.

[0272] In order to implement the above-described embodiments, the present disclosure further provides a computer program product. When the instructions in the computer program product are executed by the processor, the vehicle control method according to the above-described embodiments is executed.

[0273] It should be noted that, in the description of the present disclosure, the terms “first”, “second” and the like are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance. In addition, in the description of the present disclosure, unless otherwise specified, the meaning of “a plurality of” is two or more.

[0274] Any procedural or methodological descriptions in flow charts or otherwise described herein can be understood to represent modules, segments, or portions of code that include executable instructions for implementing the specific logical functions or steps, and the scope of preferred embodiments of the present disclosure includes additional implementations in which the functions are performed in an order different from that shown or discussed, including substantially simultaneously, or in reverse order, as will be understood by those skilled in the art to which the embodiments of the present disclosure pertain.

[0275] It should be understood that portions of the present disclosure can be implemented in hardware, software, firmware, or combinations thereof. In the above-described embodiments, a number of steps or methods can be implemented in software or firmware that is stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, implementation can be with any or a combination of the following technologies, which are all well-known in the art: a discrete logic circuit having logic gates for implementing logic functions upon an application of data signals; an application specific integrated circuit having appropriate combinational logic gates; a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0276] Those skilled in the art can understand that all or part of the steps carried out by the above-mentioned embodiments can be completed by programs instructing related hardware, and the programs can be stored in a computer readable storage medium, and when executed, include one or a combination of steps of the method embodiments.

[0277] In addition, each functional unit in various embodiments of the present disclosure can be integrated into one processing module, or each unit can be physically present separately, or two or more units can be integrated into one module. The above-mentioned integrated module can be realized in the form of hardware or in the form of a software functional module. When the integrated module is realized in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer readable storage medium.

[0278] The above-mentioned storage medium can be a read-only memory, a magnetic disk or an optical disk, etc.

[0279] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present disclosure. In the present specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0280] Although the embodiments of the present disclosure have been shown and described above, it is understood that the above-described embodiments are exemplary, and it is not construed that the present disclosure is limited to the above-described embodiments, and a person of ordinary skill in the art can make changes, modifications, replacements, and variations to the above-described embodiments within the scope of the present disclosure.

Claims

1. A vehicle control method characterized by, The vehicle comprises a camera, and the method comprises: controlling the camera to capture an image of a scene in which the vehicle is located; obtaining projection information of a projection pattern in the image; determining deformation information corresponding to the projection pattern according to the projection information; controlling the vehicle according to the deformation information; the number of cameras is multiple, and the obtaining of the projection information of the projection pattern in the image comprises: determining information of a reference pixel point in a reference image; analyzing the information of the reference pixel point to obtain reference projection information of the projection pattern in the reference image, the reference projection information being projection information obtained by a camera capturing the projection pattern when capturing the reference image.

2. The method of claim 1, wherein, The reference image belongs to multiple images.

3. The method of claim 2, wherein, wherein the determining of the deformation information corresponding to the projection pattern according to the projection information comprises: determining target projection information according to the reference projection information, the target projection information being projection information obtained by a camera capturing the projection pattern when capturing a target image, the target image belonging to the multiple images, and the reference image and the target image being different; determining the deformation information corresponding to the projection pattern according to the reference projection information and the target projection information.

4. The method of claim 3, wherein, wherein the determining of the target projection information according to the reference projection information comprises: determining the target projection information according to the reference projection information and a preset correspondence relationship; wherein the preset correspondence relationship comprises target projection information corresponding to the reference projection information, and the preset correspondence relationship is obtained by calibrating the projection pattern according to reference camera parameters corresponding to the camera capturing the reference image and target camera parameters of the camera capturing the target image.

5. The method of claim 1, wherein, The vehicle further comprises a projection device, and before the controlling of the camera to capture the image of the scene in which the vehicle is located, the method further comprises: controlling the projection device to project the projection pattern to the scene.

6. The method of claim 4, wherein, wherein the reference projection information is reference pixel coordinates based on a first camera coordinate system, and the first camera coordinate system is a coordinate system of the camera capturing the reference image; the target projection information is target pixel coordinates based on a second camera coordinate system, and the second camera coordinate system is a coordinate system of the camera capturing the target image.

7. The method of claim 6, wherein, The reference pixel point and the target pixel point correspond to a position point in a scene space respectively. wherein the determining of the deformation information corresponding to the projection pattern according to the reference projection information and the target projection information comprises: performing coordinate conversion on the reference pixel coordinates according to the reference camera parameters to obtain first point coordinates of the position point based on a vehicle coordinate system, the vehicle coordinate system being a coordinate system in which the vehicle is located; performing coordinate conversion on the target pixel coordinates according to the target camera parameters to obtain second point coordinates of the position point based on the vehicle coordinate system; determining the deformation information corresponding to the projection pattern according to the first point coordinates and the multiple second point coordinates.

8. The method of claim 7, wherein, The determining the deformation information corresponding to the projection pattern according to the first point coordinate and the plurality of second point coordinates comprises: generating point cloud data corresponding to the position point according to the first point coordinate and the plurality of second point coordinates; determining the deformation information corresponding to the projection pattern according to the plurality of point cloud data corresponding to the plurality of position points in the scene.

9. The method of claim 7 or 8, wherein, The position point is located on the ground in the scene. The motion control of the vehicle according to the deformation information comprises: determining terrain information of the ground according to the deformation information; controlling the motion of the vehicle according to the terrain information.

10. The method of claim 9, wherein, The determining the terrain information of the ground according to the deformation information comprises: determining ground flatness of the ground according to the deformation information; and / or determining ground slope of the ground according to the deformation information; and / or determining traffic sign information in the ground according to the deformation information, wherein the ground flatness, and / or the ground slope, and / or the traffic sign information are taken as the terrain information.

11. The method of claim 7 or 8, wherein, The position point is located on a target object in the scene, The motion control of the vehicle according to the deformation information comprises: determining shape information and position information of the target object according to the deformation information; controlling the obstacle avoidance of the vehicle according to the shape information and the position information.

12. A vehicle control device characterized by comprising: The vehicle comprises a camera device, and the device comprises: a collection module configured to control the camera device to collect an image of a scene in which the vehicle is located; an acquisition module configured to acquire projection information of a projection pattern in the image; a determination module configured to determine deformation information corresponding to the projection pattern according to the projection information; a control module configured to control the motion of the vehicle according to the deformation information. The number of the camera devices is a plurality, and the acquisition module is specifically configured to: determine information of a reference pixel point in a reference image; and analyze the information of the reference pixel point to obtain reference projection information of the projection pattern in the reference image, the reference projection information being projection information obtained by a camera device that collects the reference image capturing the projection pattern.

13. The apparatus of claim 12, wherein, The reference image belongs to the plurality of images.

14. The apparatus of claim 13, wherein, The determination module comprises: a first determination submodule configured to determine target projection information according to the reference projection information, the target projection information being projection information obtained by a camera device that collects a target image capturing the projection pattern, the target image belonging to the plurality of images, and the reference image and the target image being different; a second determination submodule configured to determine the deformation information corresponding to the projection pattern according to the reference projection information and the target projection information. The first determination submodule is specifically configured to:

15. The apparatus of claim 14, wherein, determine the target projection information according to the reference projection information and a preset correspondence relationship. ​ ​ The preset correspondence includes: target projection information corresponding to the reference projection information, and the preset correspondence is obtained by calibrating the projection pattern according to reference camera parameters corresponding to the camera device for collecting the reference image and target camera parameters of the camera device for collecting the target image.

16. The apparatus of claim 12, wherein, The vehicle further comprises a projection device, and the projection device comprises: A projection module is configured to control the projection device to project the projection pattern to the scene.

17. The apparatus of claim 15, wherein, The reference projection information is reference pixel coordinates based on a first camera coordinate system, and the first camera coordinate system is a coordinate system of a camera device for collecting the reference image. The target projection information is target pixel coordinates based on a second camera coordinate system, and the second camera coordinate system is a coordinate system of a camera device for collecting the target image. The reference pixel point and the target pixel point correspond to a position point in the scene space respectively.

18. The apparatus of claim 17, wherein, The second determination sub-module is specifically configured to: perform coordinate conversion on the reference pixel coordinates according to the reference camera parameters to obtain first point coordinates of the position point based on a vehicle coordinate system, and the vehicle coordinate system is a coordinate system of the vehicle; perform coordinate conversion on the target pixel coordinates according to the target camera parameters to obtain second point coordinates of the position point based on the vehicle coordinate system; determine deformation information corresponding to the projection pattern according to the first point coordinates and the plurality of second point coordinates. The second determination sub-module is specifically configured to:

19. The apparatus of claim 18, wherein, generate point cloud data corresponding to the position point according to the first point coordinates and the plurality of second point coordinates; determine deformation information corresponding to the projection pattern according to a plurality of point cloud data respectively corresponding to a plurality of position points in the scene. The position point is located on the ground in the scene.

20. The apparatus of claim 18 or 19, wherein, The control module is specifically configured to: determine terrain information of the ground according to the deformation information; perform obstacle avoidance control on the vehicle according to the terrain information. The control module is specifically configured to 21. The apparatus of claim 20, wherein, determine ground flatness of the ground according to the deformation information; and / or determine ground slope of the ground according to the deformation information; and / or determine traffic sign information in the ground according to the deformation information, wherein the ground flatness, and / or the ground slope, and / or the traffic sign information are taken as the terrain information. The position point is located on a target object in the scene.

22. The apparatus of claim 18 or 19, wherein, The control module is specifically configured to: determine shape information and position information of the target object according to the deformation information; perform obstacle avoidance control on the vehicle according to the shape information and the position information. The vehicle comprises:

23. A vehicle characterized by comprising: a body; a plurality of camera devices configured to collect images of the scene; a control device connected to the plurality of camera devices, and the control device is configured to execute the vehicle control method according to any one of claims 1-11. The vehicle further comprises:

24. The vehicle of claim 23, wherein, a projection device connected to the control device. ​ The control device is further configured to control the projection device to project a projection pattern to the scene, and the projection pattern is used for operation control of the vehicle.

25. The vehicle of claim 24, wherein, Further comprising: a vehicle light configured to illuminate; The projection device is integrated with the vehicle light.

26. A computer readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the vehicle control method according to any one of claims 1-11.

Images