Vehicle control method and device, electronic equipment and storage medium

Abstract

The invention relates to a vehicle control method and device, electronic equipment and a storage medium, and the vehicle control method comprises the steps: obtaining a collection image and a radar signal, obtaining first distance measurement information and obstacle type information according to the collection image, and obtaining second distance measurement information according to the radar signal; the first distance measurement information and the second distance measurement information are fused to obtain target distance information, so that environment closeness information is obtained according to the target distance information and the obstacle type information, then the vehicle is controlled based on the environment closeness information, and a vehicle control result is obtained; therefore, adaptive control is carried out on the vehicle by utilizing the environment sealing degree information, the problem that in the prior art, when the vehicle is parked in the environment with the high sealing degree, tail gas and noise generated by the vehicle cause adverse effects on a user in the vehicle is solved, and the user experience can be effectively improved.

Description

Technical Field

[0001] The present application relates to the field of vehicle control, and in particular to a vehicle control method, device, electronic device and storage medium. Background Art

[0002] With the development of science and technology, cars are no longer just a means of transportation, but have gradually become a mobile living space for users, especially in the era of intelligence, cars can not only meet basic transportation needs, but also provide users with more functions such as rest and entertainment; however, in parking scenarios, if the car is in a highly closed environment, the operation of the engine, air conditioning and other devices will continue to generate exhaust gas and noise. Due to the high degree of environmental closure, the exhaust gas and noise cannot be well diffused outward, which will affect the users in the car, and the users cannot promptly and effectively judge whether the vehicle is in a highly closed environment, which makes the exhaust gas and noise continue to affect the users in the car, that is, when parking in a highly closed environment, the exhaust gas and noise generated by the vehicle have an adverse effect on the users in the vehicle. Summary of the invention

[0003] In order to solve the problem in the prior art that the exhaust gas and noise generated by the vehicle when parking in a highly enclosed environment have an adverse effect on the users in the vehicle, the present application provides a vehicle control method, device, electronic device and storage medium.

[0004] In a first aspect, the present application provides a vehicle control method, comprising:

[0005] Acquire collected images and radar signals;

[0006] Obtaining first ranging information and obstacle type information according to the collected image, and obtaining second ranging information according to the radar signal;

[0007] The first ranging information and the second ranging information are integrated to obtain target distance information;

[0008] Obtaining environmental closure information according to the target distance information and the obstacle type information;

[0009] The vehicle is controlled based on the environmental enclosure information to obtain a vehicle control result.

[0010] Optionally, obtaining the first ranging information and obstacle type information according to the collected image includes:

[0011] Performing semantic segmentation on the collected image to obtain image segmentation result information;

[0012] Determining the obstacle type information corresponding to each obstacle according to the image segmentation result information;

[0013] Scan and measure the distance of the collected image to obtain the first distance measurement information.

[0014] Optionally, scanning and measuring the distance of the acquired image to obtain the first distance measurement information includes:

[0015] Based on the collected image, determine the camera light spot position and image area angle;

[0016] Obtaining a preset scanning distance and a preset scanning number corresponding to the vehicle;

[0017] Calculating according to the image area angle and the preset number of scans to obtain a scan angle;

[0018] Based on the camera light spot position, scanning and measuring the range of the collected image according to the scanning angle to determine whether there is a target scanning point, where the target scanning point is a point on the obstacle surface within a preset scanning distance from the camera light spot position;

[0019] In the case where the target scanning point exists, determining a camera ranging value, wherein the camera ranging value is a distance value between the target scanning point and the vehicle;

[0020] In the absence of the target scanning point, determining a preset distance measurement value as the camera distance measurement value;

[0021] Based on the camera ranging value, the first ranging information is generated.

[0022] Optionally, the first ranging information and the second ranging information are fused to obtain target distance information, including:

[0023] Extracting a target scanning point and a camera ranging value corresponding to the target scanning point from the first ranging information;

[0024] Based on the target scanning point, radar ranging is performed in combination with the radar signal to obtain second ranging information corresponding to the target scanning point, where the second ranging information includes a radar ranging value;

[0025] According to a preset fusion coefficient, the camera distance measurement value and the radar distance measurement value are fused and calculated to obtain a target distance value;

[0026] The target distance information is generated according to the target distance value.

[0027] Optionally, performing radar ranging based on the target scanning point and in combination with the radar signal to obtain second ranging information corresponding to the target scanning point includes:

[0028] determining whether there is a target radar point in the radar signal that matches the target scanning point;

[0029] In a case where the target radar point matching the target scanning point exists in the radar signal, determining a radar ranging value corresponding to the target radar point based on a distance value between the target radar point and the vehicle;

[0030] In a case where the target radar point matching the target scanning point does not exist in the radar signal, determining the camera ranging value as the radar ranging value;

[0031] The second ranging information is generated according to the radar ranging value.

[0032] Optionally, obtaining environment closure information according to the target distance information and the obstacle type information includes:

[0033] Extracting a target distance value from the target distance information;

[0034] Determining a type weight corresponding to the obstacle type information;

[0035] The target distance value and the type weight are used to perform weighted calculation to obtain environmental closure information.

[0036] Optionally, the step of performing weighted calculation using the target distance value and the type weight to obtain the environmental closure information includes:

[0037] Determine a camera identifier corresponding to the target distance information;

[0038] Obtaining a preset camera weight corresponding to the camera identifier;

[0039] Based on the camera weight, a weighted calculation is performed in combination with the target distance value and the type weight to obtain the environment closure information.

[0040] In a second aspect, a vehicle control device is provided, comprising:

[0041] An acquisition module, used for acquiring images and radar signals;

[0042] A processing module, used to obtain first ranging information and obstacle type information according to the collected image, and obtain second ranging information according to the radar signal;

[0043] A fusion module, used for fusing the first ranging information and the second ranging information to obtain target distance information;

[0044] An environment closure module, used to obtain environment closure information according to the target distance information and the obstacle type information;

[0045] A control module is used to control the vehicle based on the environmental enclosure information to obtain a vehicle control result.

[0046] In a third aspect, an electronic device is provided, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0047] Memory, used to store computer programs;

[0048] The processor is used to implement the vehicle control method described in any one of the first aspects when executing the program stored in the memory.

[0049] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the vehicle control method as described in any one of the first aspects is implemented.

[0050] The embodiment of the present application obtains a captured image and a radar signal, obtains first ranging information and obstacle type information according to the captured image, and obtains second ranging information according to the radar signal, and fuses the first ranging information and the second ranging information to obtain target distance information, so as to obtain environmental enclosure information according to the target distance information and the obstacle type information, and then controls the vehicle based on the environmental enclosure information to obtain a vehicle control result; thereby, the vehicle is adaptively controlled using the environmental enclosure information, avoiding the problem in the existing related technology that the exhaust gas and noise generated by the vehicle when parking in a highly enclosed environment have an adverse effect on the user in the vehicle, and can effectively improve the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0052] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0053] Figure 1 A schematic diagram of a vehicle control method provided in an embodiment of the present application;

[0054] Figure 2 A schematic diagram of an application scenario of a vehicle control method provided in an embodiment of the present application;

[0055] Figure 3 A schematic diagram of the structure of a vehicle control device provided in an embodiment of the present application;

[0056] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0058] As a product that is used frequently in daily life, vehicles often need to be parked for rest, such as parking in the basement, parking for people, etc. When the vehicle is parked, in order to avoid being too hot or bored in the car, most users will turn on the engine, air conditioning, in-car multimedia and other devices for cooling and entertainment. If more devices are turned on or the power of the turned devices is large, the exhaust gas and noise generated by the vehicle will also increase proportionally. For scenes with open environments, the exhaust gas and noise will diffuse to the surroundings in time, and the impact on users in the car will be relatively small. If the environment is relatively closed, the exhaust gas and noise cannot diffuse to the surroundings in time, the noise will be reflected and the exhaust gas will further affect the users in the car through circulation; most users cannot observe whether the surrounding environment is closed in time and accurately when parking, and will continue to be affected by exhaust gas and noise. That is, when parking in a highly closed environment, the exhaust gas and noise generated by the vehicle will have an adverse effect on users in the vehicle, which can effectively improve the user experience.

[0059] In order to solve the problem of low efficiency of vehicle information query in the prior art, the present application provides a vehicle control method, device, electronic device and storage medium, by acquiring a captured image and a radar signal, obtaining first ranging information and obstacle type information according to the captured image, and obtaining second ranging information according to the radar signal, and fusing the first ranging information and the second ranging information to obtain target distance information, so as to obtain environmental enclosure information according to the target distance information and the obstacle type information, and then controlling the vehicle based on the environmental enclosure information to obtain a vehicle control result; thereby using the environmental enclosure information to adaptively control the vehicle, avoiding the problem in the prior related technology that the exhaust gas and noise generated by the vehicle when parking in a highly enclosed environment has an adverse effect on the user in the vehicle, and can effectively improve the user experience.

[0060] Figure 1A flow chart of a method for determining a verification method provided in an embodiment of the present application. This method can be applied to one or more electronic devices such as vehicles, clients, and servers. In addition, the execution subject of this method can be hardware or software. When the above-mentioned execution subject is hardware, the execution subject can be one or more of the above-mentioned electronic devices. For example, a single electronic device can execute this method, or multiple electronic devices can cooperate with each other to execute this method. When the above-mentioned execution subject is software, this method can be implemented as multiple software or software modules, or as a single software or software module. It is not specifically limited here.

[0061] like Figure 1 As shown, the present application discloses an embodiment, providing a vehicle control method, which may specifically include the following steps:

[0062] Step S110: Acquire collected images and radar signals.

[0063] Among them, the captured image can represent the image captured by the camera equipment pre-configured in the vehicle, such as the image captured by the four-channel vehicle-mounted fisheye camera, the eight-channel vehicle-mounted fisheye camera, the infrared camera device and other camera equipment, and the radar signal can represent the radar signal collected by the vehicle-mounted ultrasonic radar.

[0064] It should be noted that vehicles are usually pre-configured with multiple cameras. At this time, the captured image can also represent the stitched image of the panoramic monitoring imaging system (Around View Monitor, AVM). The AVM stitched image is the image of the vehicle's surrounding environment stitched together from the images captured by all cameras in the vehicle.

[0065] Step S120: obtaining first ranging information and obstacle type information according to the collected image, and obtaining second ranging information according to the radar signal.

[0066] Specifically, after obtaining the acquired image, the acquired image can be processed to obtain first ranging information and obstacle type information, wherein the first ranging information can represent the ranging information of the distance between each obstacle and the vehicle in the acquired image, and the obstacle type information can represent the type of each obstacle in the acquired image. The specific type represented by the obstacle type information can be one of multiple pre-configured obstacle types, wherein the pre-configured obstacle types can include walls, vehicles, columns, etc., which are not specifically limited in this example; and second ranging information can also be obtained according to the radar signal, and the second ranging information can represent the ranging information of the distance between each obstacle and the vehicle measured by the radar signal.

[0067] It should be noted that the method of performing image processing on the captured image in this embodiment may include multiple processing steps, such as image semantic segmentation processing, image ranging processing and other steps. The image semantic segmentation processing can perform semantic segmentation on the captured image to obtain the segmentation result corresponding to the obstacle, so that the type of obstacle, that is, the obstacle type information, can be obtained based on the segmentation result corresponding to the obstacle, and the image ranging processing can identify and measure the image to determine the distance between each obstacle and the vehicle, and then obtain the first ranging information; of course, the above is only an example for illustration, and it can be adaptively adjusted according to the scenario and needs during specific implementation, and this embodiment does not make specific limitations on this.

[0068] In addition, the first ranging information in this embodiment can represent the ranging information of the distance between each obstacle and the vehicle in the captured image, that is, the first ranging information can include the ranging information of the distance between multiple obstacles and the vehicle, and the first ranging information can also represent the ranging information of the distance between each obstacle surface point and the vehicle in the captured image, and an obstacle can include one or more obstacle surface points.

[0069] Step S130: The first distance measurement information and the second distance measurement information are integrated to obtain target distance information.

[0070] Specifically, after obtaining the first ranging information and the second strategy information, a fusion calculation can be performed based on the first ranging information and combined with the first ranging information to obtain the target distance information. The target distance information can represent the calculated distance between each obstacle and the vehicle, and the fusion calculation can be a fusion calculation of the distance represented by the first ranging information and the distance identified by the second ranging information. The specific fusion calculation method can adopt a preset neural network model or a preset formula model, etc., which is not specifically limited in this embodiment.

[0071] It should be noted that in the process of fusing the first ranging information and the second ranging information to obtain the target distance information, the first ranging information and the second ranging information used have a matching relationship, which can indicate that the distances targeted by the first ranging information and the second ranging information are for the same subject.

[0072] Step S140: Obtaining environment closure information according to the target distance information and obstacle type information.

[0073] Specifically, after obtaining the target distance information, the environmental enclosure information can be obtained based on the target distance information and the obstacle type information. The environmental enclosure information can indicate the enclosure of the environment in which the current vehicle is located. The specific method for obtaining the environmental enclosure information may be weighted calculation of the target distance information and the obstacle type information, algorithm model processing, neural network model processing, etc. This embodiment does not make any specific limitation on this.

[0074] In addition, the environmental closure information in this embodiment can characterize the closure of the vehicle's current environment, that is, the more obstacles there are in the vehicle's surrounding environment, the larger the obstacles are, the higher the environmental closure will be for the user and the vehicle, and the fewer obstacles there are in the vehicle's surrounding environment, the smaller the obstacles are, the lower the environmental closure will be for the user and the vehicle; for example, when the vehicle is in an environment where three of the four sides are walls, the environmental closure of the vehicle's environment is relatively high, and a high degree of closure can be determined as the environmental closure information; when the vehicle is in an environment where two of the four sides are walls, the environmental closure of the vehicle's environment is relatively medium, and a medium degree of closure can be determined as the environmental closure information; when the vehicle is in an environment where there are no obstacles around, the vehicle's environment has no environmental closure, and no environmental closure can be determined as the environmental closure information; of course, the above is only an example, and this embodiment does not make any specific limitations on this.

[0075] It can be seen that in this embodiment, since the obstacle type information represents the type of each obstacle, and the target distance information is combined with the obstacle type information for weighted calculation, it can play the role of combining the obstacle distance represented by the target distance information and the obstacle type represented by the obstacle type information to obtain the degree of enclosure of the obstacle to the vehicle, and then all the enclosures can be weighted calculated to obtain the environmental enclosure information corresponding to the vehicle.

[0076] It should be noted that in this embodiment, when different methods are used to obtain the environmental closure information, the environmental closure information can specifically represent different values. For example, when normalized weighted calculation is used, the environmental closure information can be a normalized value; when percentage weighted calculation is used, the environmental closure information can be a percentage value; when the environmental closure information is obtained by closure degree calculation, the environmental closure information can be a high, medium or other closure degree; of course, other methods can also be used for weighted calculation, and this embodiment does not specifically limit this.

[0077] Step S150: Control the vehicle based on the environmental enclosure information to obtain a vehicle control result.

[0078] Specifically, after obtaining the environmental enclosure information, the vehicle can be controlled based on the environmental enclosure information to obtain a vehicle control result, which indicates whether the vehicle control is completed, that is, the vehicle control result can be a vehicle control completion result or a vehicle control incomplete control result. When an abnormality or authority or other influences occur during the vehicle control process, resulting in the inability to control the vehicle, a vehicle control result of an incomplete vehicle control result can be generated. When no abnormality or authority or other situations occur during the vehicle control process, a vehicle control result of a completed vehicle control result can be generated; and the vehicle control method can include but is not limited to engine control, range extender control, air-conditioning control, etc., wherein the vehicle control method and the vehicle control result have a corresponding relationship, that is, the vehicle control result includes the relevant results corresponding to each vehicle control. For example, when the vehicle control method is engine control, the vehicle control result can include the engine control result, and the engine control result can indicate whether the engine control is completed; when the vehicle control method is engine control and air-conditioning control, the vehicle control result can include the engine control result and the air-conditioning control result, and the air-conditioning control result can indicate whether the air-conditioning control is completed; this embodiment does not make specific limitations on this.

[0079] It should be noted that, in the process of controlling the vehicle based on the environmental enclosure information and obtaining the vehicle control result, the target control strategy corresponding to the environmental enclosure information can also be determined. The target vehicle control strategy represents a preset control strategy that matches the environmental enclosure information. The preset control strategy is a pre-configured strategy for controlling the vehicle, such as turning off the range extender and the air conditioner, turning off the range extender and turning on the air conditioner, limiting the output power of the range extender, and other strategies; in the process of determining the target control strategy, the environmental enclosure score can be extracted from the environmental enclosure information. The environmental enclosure score represents the specific environmental enclosure corresponding to the environmental enclosure information, so that the target score interval to which the environmental enclosure score belongs can be determined, and then the preset control strategy corresponding to the target score interval can be determined as the target control strategy, and the correspondence between the preset control strategy and the score interval can be pre-configured, so as to achieve the effect of controlling the vehicle in different ways according to different environmental enclosure information.

[0080] In one example, after obtaining the environmental enclosure information, the environmental enclosure score is extracted from the environmental enclosure information. If the environmental enclosure score is greater than 0.8, the vehicle is controlled to turn off the range extender and the air conditioner; if the environmental enclosure score is less than 0.8 and greater than 0.5, the vehicle is controlled to turn off the range extender and only turn on the air conditioner; if the environmental enclosure score is less than 0.5 and greater than 0.2, the vehicle is controlled to limit the output power of the range extender; if the environmental enclosure score is less than 0.2, the vehicle is controlled not to limit the operation of the air conditioner and the range extender; of course, the above is only an example for illustration, and adaptive adjustments can be made according to the scenario and needs during specific implementation, and this embodiment does not make specific limitations on this.

[0081] It can be seen that in this embodiment, by acquiring a captured image and a radar signal, first ranging information and obstacle type information are obtained according to the captured image, and second ranging information is obtained according to the radar signal, and the first ranging information and the second ranging information are fused to obtain target distance information, so as to obtain environmental enclosure information according to the target distance information and the obstacle type information, and then the vehicle is controlled based on the environmental enclosure information to obtain a vehicle control result; thereby, the vehicle is adaptively controlled using the environmental enclosure information, avoiding the problem in the existing related technology that the exhaust gas and noise generated by the vehicle when parking in a highly closed environment have an adverse effect on the user in the vehicle, and can effectively improve the user experience.

[0082] In the process of judging whether there are obstacles around the vehicle, it is usually judged based on the distance between the vehicle and other surrounding objects. In related related technologies, users usually make visual observations based on their experience. At this time, the judgment is mainly based on the user's experience and feeling, which means that there will be a problem of inaccurate distance judgment.

[0083] In an optional embodiment of the present application, step S120 obtains the first ranging information and obstacle type information based on the captured image, and specifically may include the following sub-steps: performing semantic segmentation on the captured image to obtain image segmentation result information; determining the obstacle type information corresponding to each obstacle based on the image segmentation result information; and performing scanning ranging on the captured image to obtain the first ranging information.

[0084] In this embodiment, after obtaining the captured image, semantic segmentation can be performed on the captured image to obtain image segmentation result information. The image segmentation result information represents the image information obtained after semantic segmentation. Semantic segmentation is to identify and mark the captured image. In this embodiment, semantic segmentation can identify obstacles in the captured image and mark the specific types of obstacles in the captured image. That is, the obstacle type corresponding to each obstacle in the captured image can be determined based on the image segmentation result information, and then the obstacle type information can be obtained, so that the obstacle type information can include each obstacle in the captured image and its corresponding type; then the captured image can be scanned and measured to obtain the first ranging information, and the first ranging information represents the information of the distance between the vehicle and the surrounding obstacles measured based on the captured image.

[0085] In this embodiment, semantic segmentation can be performed by collecting images to determine the obstacle type information corresponding to each obstacle, and scanning and measuring the distance of the collected images can obtain the first ranging information representing the distance between the vehicle and the surrounding obstacles, thereby avoiding the problem of inaccurate distance judgment by users based on personal experience and feeling in related technologies, improving the accuracy of distance judgment, and obtaining the type corresponding to each obstacle.

[0086] It should be noted that the principle of semantic segmentation is based on deep learning, especially convolutional neural networks. By training a large amount of labeled image data, the model can learn the mapping relationship from pixel level to high-level semantic information, so that the trained model can be used in practical applications to infer new unlabeled images and divide them into multiple semantic regions; the semantic segmentation in this embodiment can adopt the deeplabv3 semantic segmentation model, such as Figure 2 As shown, Figure 2 The left side in the middle is the vehicle AVM mosaic image, that is, the collected image, and the right side is the image segmentation result information obtained through semantic segmentation.

[0087] In an optional embodiment of the present application, scanning and measuring the distance of the captured image to obtain the first ranging information may specifically include the following sub-steps: based on the captured image, determining the camera light spot position and the image area angle; obtaining a preset scanning distance and a preset scanning number corresponding to the vehicle; calculating according to the image area angle and the preset scanning number to obtain the scanning angle; based on the camera light spot position, scanning and measuring the distance of the captured image according to the scanning angle to determine whether there is a target scanning point, the target scanning point being a point on the obstacle surface within the preset scanning distance from the camera light spot position; if there is a target scanning point, determining the camera ranging value, the camera ranging value being the distance value between the target scanning point and the vehicle; if there is no target scanning point, determining the preset ranging value as the camera ranging value; based on the camera ranging value, generating the first ranging information.

[0088] In this embodiment, in the process of scanning and measuring the distance of the collected image, the camera light spot position and the image area angle can be determined based on the collected image. The camera light spot position represents the optical center position of the camera in the collected image, and the image area angle represents the shooting area angle of the camera in the collected image; thereby, the preset scanning distance and preset scanning times corresponding to the vehicle can be obtained. The preset scanning distance represents the distance pre-configured to scan the distance between the vehicle and the vehicle. For example, when the preset scanning distance is 2 meters, the vehicle is scanned within a distance of 2 meters, and the preset scanning times represents the number of scans required for each camera corresponding screen; and then the scanning angle can be calculated based on the image area angle and the preset scanning times, and the scanning angle can represent the angle of each scanning interval, wherein the specific calculation method can be to divide the image area angle by the preset scanning times for calculation, and the result is calculated as is the scanning angle. Of course, other calculation methods can also be used. This embodiment does not make specific restrictions on this. Subsequently, based on the camera light spot position, the captured image can be scanned and measured according to the scanning angle to determine whether there is a target scanning point. The target scanning point is an obstacle surface point within a preset scanning distance from the camera light spot position, that is, a ray can be made with the camera light spot position as the center, and scanning can be started from one end of the camera corresponding screen toward the other end to determine whether there is an obstacle surface point within the preset scanning distance. In this way, a scan is performed once at each scanning angle, and a preset number of scans are performed for each camera corresponding screen, wherein the obstacle surface point indicates whether the ray made with the camera light spot position as the center within the preset scanning distance intersects with the obstacle surface. If the ray within the preset scanning distance intersects with the obstacle surface, the intersection point is determined as the target scanning point.

[0089] When there is a target scanning point, the camera ranging value can be determined, and the camera ranging value is the distance value between the target scanning point and the vehicle. More specifically, the camera ranging value can be the distance between the camera light spot position and the target scanning point. When there is no target scanning point, it means that there is no obstacle within the preset scanning distance at the scanning angle. At this time, the preset ranging value can be determined as the camera ranging value, wherein the preset ranging value represents a pre-configured ranging value, which can be pre-configured according to the target distance information obtained by subsequent different fusion calculations. The required measurement value in this case; and the captured image may include multiple camera corresponding screens. At this time, the aforementioned scanning ranging step can be performed on each camera corresponding screen to obtain the camera ranging value corresponding to each camera corresponding screen, and the first ranging information is generated by integrating all the camera ranging values.

[0090] In one example, if Figure 2 As shown in FIG. 1 , when the captured image is an AVM stitched image, the captured image may include corresponding images of multiple cameras, such as Figure 2 The captured image shown includes the corresponding images of the cameras in the front, back, left and right directions, where A is the vehicle area, A1 is the camera configured on the vehicle, and as shown in the figure, the vehicle includes four cameras, through which the relevant images are captured, and the relevant images captured by the four cameras are spliced ​​to generate an AVM spliced ​​image, i.e., the captured image; at this time, the aforementioned scanning and ranging steps can be performed for the corresponding images of the four cameras respectively to obtain the camera ranging value corresponding to the corresponding image of each camera; the specific scanning process is explained by taking the corresponding image of the camera in the right direction as an example, Figure 2 The position of the triangle symbol on the left side is the position of camera A1, which can represent the position of the camera light spot of camera A1; taking the camera on the right side of the vehicle as an example, the image acquisition angle of the camera can be represented by the dotted line angle in the figure, and the dotted line angle can be the image area angle recorded as α, so as to scan the image acquired by the camera. The specific scanning method can be to scan according to a preset scanning distance and a preset number of times. For example, the preset scanning distance is 2 meters, and the preset scanning number is n times. Then, the scanning angle can be calculated based on the image area angle and the preset scanning number, that is, it can be calculated according to the formula "scanning angle = α / n". When scanning, a ray can be made with the camera light spot position as the center, starting from one end of the image area angle to the other end, and each scan is scanned once at an interval of α / n, for a total of n scans, and it is determined whether there is a target scanning point in each scan, such as Figure 2If ray T does not intersect with any obstacle within the preset scanning distance of 2 meters, there is no target scanning point. If ray Y intersects with the obstacle column, there is a target scanning point. Then, each scan can be recorded. When there is a target scanning point, the camera ranging value is determined. The camera ranging value is the distance between the target scanning point and the vehicle. When there is no target scanning point, the preset ranging value is determined as the camera ranging value. That is, in this embodiment, the same number of camera ranging values ​​as the preset number of scans will be obtained. Then, the first ranging information can be generated based on integration of all camera ranging values, so that the first ranging information can include the camera ranging values ​​corresponding to each camera.

[0091] It can be seen that in this embodiment, the preset scanning distance is used to determine the distance between each obstacle and the vehicle within the preset scanning distance, that is, the first ranging information can reflect whether the vehicle is in a closed environment; in addition, different scenes or users may have different judgment criteria for closure. At this time, the preset scanning distance can be determined based on vehicle size, user configuration, user data and other information.

[0092] In the existing related technologies, the distance between the vehicle and the obstacle is usually determined by a single method, such as infrared measurement, visual measurement, etc. However, the use of a single measurement method may result in inaccurate calculations such as calculation anomalies or calculation errors.

[0093] In an optional embodiment of the present application, step S130 fuses the first ranging information and the second ranging information to obtain target distance information, which may specifically include the following sub-steps: extracting the target scanning point and the camera ranging value corresponding to the target scanning point from the first ranging information; performing radar ranging based on the target scanning point in combination with the radar signal to obtain the second ranging information corresponding to the target scanning point, wherein the second ranging information includes the radar ranging value; performing a fusion calculation on the camera ranging value and the radar ranging value according to a preset fusion coefficient to obtain the target distance value; and generating the target distance information based on the target distance value.

[0094] In this embodiment, after obtaining the first ranging information, the target scanning point and the camera ranging value corresponding to the target scanning point can be extracted from the first ranging information, and radar ranging is performed based on the target scanning point in combination with the radar signal to obtain second ranging information corresponding to the target scanning point. The second ranging information includes the radar ranging value corresponding to each target scanning point. The radar ranging value can indicate the distance between the adjacent point adjacent to the target scanning point in the radar signal and the vehicle. The adjacent point can be an ultrasonic radar point whose distance difference with the target scanning point is less than a preset distance difference threshold. The ultrasonic radar point can be a radar point included in the radar signal. Then, according to the preset fusion system The camera ranging value and the radar ranging value are fused and calculated to obtain the target distance value. The fusion coefficient can be a first coefficient pre-configured to match the camera ranging value and a second coefficient pre-configured to match the radar ranging value. When performing the fusion calculation, the camera ranging value can be combined with the first coefficient, and the radar ranging value can be combined with the second coefficient, and finally weighted to obtain the target distance value; since multiple target scanning points can be included, the aforementioned fusion calculation steps can be performed for each target scanning point, then the target distance value corresponding to each target scanning point can be obtained, and then the target distance value corresponding to each target scanning point can be integrated to generate target distance information.

[0095] In one example, when the fusion coefficient includes a first coefficient of 0.5 and a second coefficient of 0.5, i target scanning points are obtained, and for each target scanning point, the adjacent point corresponding to each target scanning point is determined by the radar signal. So we can base it on the nearby points calculate Distance to vehicle That is, the radar ranging value, which can then be fused and calculated according to the following formula:

[0096]

[0097] in is the target distance value.

[0098] It can be seen that in this embodiment, the target distance value is obtained by fusing the camera ranging value and the radar ranging value, which plays the role of using two ranging methods for ranging and performing weighted calculation, thereby avoiding the problems of inaccurate calculations such as calculation anomalies or calculation errors that exist in related technologies that only use a single measurement method, and can effectively improve the measurement accuracy.

[0099] In an optional embodiment of the present application, radar ranging is performed based on the target scanning point in combination with the radar signal to obtain second ranging information corresponding to the target scanning point, which may specifically include the following sub-steps: determining whether there is a target radar point matching the target scanning point in the radar signal; if there is a target radar point matching the target scanning point in the radar signal, determining the radar ranging value corresponding to the target radar point based on the distance value of the target radar point from the vehicle; if there is no target radar point matching the target scanning point in the radar signal, determining the camera ranging value as the radar ranging value; and generating the second ranging information based on the radar ranging value.

[0100] In this embodiment, in the process of obtaining the second ranging information corresponding to the target scanning point, it can be determined whether there is a target radar point matching the target scanning point in the radar signal. The target radar point can represent an ultrasonic radar point whose distance difference with the target scanning point is less than a preset distance difference threshold, and the preset distance difference threshold can be pre-configured according to the scenario and needs, such as 2 cm, 1 cm, etc.; when there is a target radar point matching the target scanning point in the radar signal, the radar ranging value corresponding to the target radar point can be determined based on the distance value of the target radar point from the vehicle, and when there is no target radar point matching the target scanning point in the radar signal, the camera ranging value can be determined as the radar ranging value. Since multiple target scanning points may be included, the aforementioned steps can be performed for each target scanning point, so as to determine the radar ranging value corresponding to each target scanning point, and then the second ranging information can be generated by integrating the radar ranging value corresponding to each target scanning point.

[0101] In one example, when the fusion coefficient includes a first coefficient of 0.5 and a second coefficient of 0.5, i target scanning points are obtained, and for each target scanning point, it is determined whether there is a target radar point matching the target scanning point in the radar signal. When there is a target radar point matching the target scanning point in the radar signal, such as the target radar point is At this time, based on the target radar point The distance value from the vehicle determines the radar ranging value corresponding to the target radar point. For example, the radar ranging value is If there is no target radar point matching the target scanning point in the radar signal, the camera distance value D i Determined as the radar ranging value, that is, the radar ranging value At this time, if the formula for calculating the target distance value is Then the target distance value corresponding to each target scanning point can be calculated. In the process of calculating each target point, Can be The distance value from the vehicle can also be D i; Thus, when there is no target radar point matching the target scanning point in the radar signal, the camera ranging value can be directly used as the target distance value.

[0102] In an optional embodiment of the present application, step S140 obtains environmental closure information based on the target distance information and the obstacle type information, and may specifically include the following sub-steps: extracting the target distance value from the target distance information; determining the type weight corresponding to the obstacle type information; and performing weighted calculation using the target distance value and the type weight to obtain the environmental closure information.

[0103] In this embodiment, after obtaining the target distance information, the target distance value can be extracted from the target distance information, and the number of extracted target distance values ​​is multiple, and the specific number can be consistent with the preset number of scans; then the type weight corresponding to the obstacle type information is determined, and the type weight is the weight corresponding to each obstacle type pre-configured, so that the target distance value and the type weight can be used for weighted calculation to obtain the environmental closure information, wherein the type weight corresponding to the target distance value can be determined in the process of weighted calculation, and specifically, the target scanning point corresponding to the target distance value can be determined, and the obstacle type information corresponding to the target scanning point can be obtained, so that the type weight corresponding to the obstacle type information can be determined, and then the type weight and the target distance value can be weighted calculated to obtain the environmental closure score corresponding to each target distance value, and then the environmental closure score corresponding to each target distance value can be integrated to obtain the environmental closure information.

[0104] In addition, in the process of using the target distance value and type weight for weighted calculation to obtain the environmental closure information, the environmental closure coefficient can also be obtained. Based on the target distance value and type weight, combined with the environmental closure coefficient, weighted calculation is performed to obtain the environmental closure information, where the environmental closure coefficient is a pre-configured constant.

[0105] In one example, the environmental closure information can be obtained by weighted calculation using the following formula:

[0106]

[0107] Where σ is the environmental closure coefficient, i is the position of the current target distance value, n is the total number of target distance values, and A i is the type weight corresponding to the current target distance value, is the current target distance value, Enclose cam is the environmental closure information; it can be seen that in this embodiment, the environmental closure information is the value obtained by weighted averaging the environmental closure scores corresponding to each target distance value.

[0108] In an optional embodiment of the present application, a weighted calculation is performed using the target distance value and the type weight to obtain the environment closure information, which may specifically include the following sub-steps: determining the camera identifier corresponding to the target distance information; obtaining the preset camera weight corresponding to the camera identifier; and performing a weighted calculation based on the camera weight in combination with the target distance value and the type weight to obtain the environment closure information.

[0109] In this embodiment, in the process of using the target distance value and type weight to perform weighted calculation to obtain the environmental closure information, since the captured image may contain camera images captured by multiple cameras, the camera identifier corresponding to the target distance information can be determined. The camera identifier can represent an identifier used to represent the camera, and the camera is a camera used to obtain the camera image corresponding to the current target distance information. Then, the preset camera weight corresponding to the camera identifier can be obtained. The camera weight represents the weight of the current camera's influence on the environmental closure. Different weights can be configured in advance for each camera, so that based on the camera weight, combined with the target distance value and type weight, a weighted calculation is performed to obtain the environmental closure score corresponding to each camera, and then the environmental closure information is obtained by integration based on the environmental closure score corresponding to each camera.

[0110] Since obstacles at different positions have different environmental enclosure effects on users or vehicles, for example, for exhaust gas, the rear obstacle has a greater environmental enclosure effect on exhaust gas than the front obstacle. At this time, the present embodiment can use the camera weight to adjust the target distance value and type weight corresponding to the camera image captured by each camera, so as to achieve the purpose of improving the accuracy of environmental enclosure information.

[0111] In one example, the environmental closure information can be obtained by weighted calculation using the following formula:

[0112]

[0113] Among them, Enclose all is the environmental closure information, w j is the camera weight, j is the current camera ID, and 4 means that the vehicle contains 4 cameras. In this case, the camera IDs can be j=1, j=2, j=3, and j=4. j The environmental closure score corresponding to the camera image of the current camera identifier, also known as Enclose j The camera image can be The calculated value is the weighted average of the environmental closure scores corresponding to each target distance value in the same camera image.

[0114] like Figure 3 As shown, the present application also discloses an embodiment, providing a vehicle control device, including:

[0115] An acquisition module 310 is used to acquire collected images and radar signals;

[0116] A processing module 320, configured to obtain first ranging information and obstacle type information according to the collected image, and obtain second ranging information according to the radar signal;

[0117] A fusion module 330, configured to fuse the first ranging information and the second ranging information to obtain target distance information;

[0118] An environment closure module 340, configured to obtain environment closure information according to the target distance information and the obstacle type information;

[0119] The control module 350 is used to control the vehicle based on the environmental enclosure information to obtain a vehicle control result.

[0120] In an optional embodiment of the present application, the processing module 320 may include:

[0121] A semantic segmentation unit, used to perform semantic segmentation on the collected image to obtain image segmentation result information;

[0122] A first determining unit, configured to determine the obstacle type information corresponding to each obstacle according to the image segmentation result information;

[0123] The scanning and ranging unit is used to scan and measure the distance of the collected image to obtain the first ranging information.

[0124] In an optional embodiment of the present application, the scanning ranging method may include:

[0125] A first determination subunit, used to determine the camera light spot position and the image area angle based on the collected image;

[0126] A first acquisition subunit is used to acquire a preset scanning distance and a preset scanning number corresponding to the vehicle;

[0127] A calculation subunit, configured to calculate according to the image area angle and the preset number of scans to obtain a scan angle;

[0128] A scanning and ranging subunit, configured to scan and measure the acquired image according to the scanning angle based on the camera light spot position to determine whether there is a target scanning point, wherein the target scanning point is a point on the surface of an obstacle within a preset scanning distance from the camera light spot position;

[0129] A second determining subunit is used to determine a camera ranging value when the target scanning point exists, wherein the camera ranging value is a distance value between the target scanning point and the vehicle;

[0130] A third determining subunit is used to determine a preset distance measurement value as the camera distance measurement value when the target scanning point does not exist;

[0131] The first generating subunit is used to generate the first ranging information based on the camera ranging value.

[0132] In an optional embodiment of the present application, the fusion module 330 may include:

[0133] A first extraction unit, configured to extract a target scanning point and a camera ranging value corresponding to the target scanning point from the first ranging information;

[0134] A radar ranging unit, configured to perform radar ranging based on the target scanning point and in combination with the radar signal to obtain second ranging information corresponding to the target scanning point, wherein the second ranging information includes a radar ranging value;

[0135] A fusion calculation unit, used to perform fusion calculation on the camera ranging value and the radar ranging value according to a preset fusion coefficient to obtain a target distance value;

[0136] A generating unit is used to generate the target distance information according to the target distance value.

[0137] In an optional embodiment of the present application, the radar ranging unit may include:

[0138] a fourth determination subunit, configured to determine whether there is a target radar point matching the target scanning point in the radar signal;

[0139] a fifth determining subunit, configured to determine, when the target radar point matching the target scanning point exists in the radar signal, a radar ranging value corresponding to the target radar point based on a distance value between the target radar point and the vehicle;

[0140] a sixth determining subunit, configured to determine the camera ranging value as the radar ranging value when there is no target radar point matching the target scanning point in the radar signal;

[0141] The second generating subunit is used to generate the second ranging information according to the radar ranging value.

[0142] In an optional embodiment of the present application, the environment enclosure module 340 may include:

[0143] A second extraction unit, configured to extract a target distance value from the target distance information;

[0144] A second determining unit, configured to determine a type weight corresponding to the obstacle type information;

[0145] The weighted calculation unit is used to perform weighted calculation using the target distance value and the type weight to obtain environmental closure information.

[0146] In an optional embodiment of the present application, the weighted calculation unit may include:

[0147] A seventh determining subunit, configured to determine a camera identifier corresponding to the target distance information;

[0148] A second acquisition subunit is used to acquire a preset camera weight corresponding to the camera identifier;

[0149] The weighted calculation subunit is used to perform weighted calculation based on the camera weight in combination with the target distance value and the type weight to obtain environmental closure information.

[0150] The implementation process of the functions and effects of each module in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, which will not be repeated here.

[0151] like Figure 4 As shown, an embodiment of the present application provides an electronic device, including a processor 410, a communication interface 420, a memory 430 and a communication bus 440, wherein the processor 410, the communication interface 420, and the memory 430 communicate with each other through the communication bus 440;

[0152] Memory 430, for storing computer programs;

[0153] In one embodiment of the present application, the processor 410 is used to execute the program stored in the memory 430 to implement the vehicle control method provided by any of the aforementioned method embodiments, by acquiring a captured image and a radar signal, obtaining first ranging information and obstacle type information according to the captured image, and obtaining second ranging information according to the radar signal, and fusing the first ranging information and the second ranging information to obtain target distance information, so as to obtain environmental closure information according to the target distance information and the obstacle type information, and then controlling the vehicle based on the environmental closure information to obtain a vehicle control result; thereby, the vehicle is adaptively controlled using the environmental closure information, avoiding the problem in the prior art that the exhaust gas and noise generated by the vehicle when parking in a highly closed environment have an adverse effect on the user in the vehicle, and can effectively improve the user experience.

[0154] The embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the vehicle control method provided by any of the aforementioned method embodiments is implemented, by acquiring a captured image and a radar signal, obtaining first ranging information and obstacle type information according to the captured image, and obtaining second ranging information according to the radar signal, and fusing the first ranging information and the second ranging information to obtain target distance information, so as to obtain environmental enclosure information according to the target distance information and the obstacle type information, and then controlling the vehicle based on the environmental enclosure information to obtain a vehicle control result; thereby, the vehicle is adaptively controlled using the environmental enclosure information, avoiding the problem in the prior art that the exhaust gas and noise generated by the vehicle when parking in a highly enclosed environment has an adverse effect on the user in the vehicle, and can effectively improve the user experience.

[0155] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0156] The above describes specific embodiments of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0157] The foregoing is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A vehicle control method, characterized in that: include: Acquire collected images and radar signals; Obtaining first ranging information and obstacle type information according to the collected image, and obtaining second ranging information according to the radar signal; The first ranging information and the second ranging information are integrated to obtain target distance information; Obtaining environmental closure information according to the target distance information and the obstacle type information; The vehicle is controlled based on the environmental enclosure information to obtain a vehicle control result.

2. The vehicle control method according to claim 1, characterized in that: The obtaining the first ranging information and the obstacle type information according to the collected image includes: Performing semantic segmentation on the collected image to obtain image segmentation result information; Determining the obstacle type information corresponding to each obstacle according to the image segmentation result information; Scan and measure the distance of the collected image to obtain the first distance measurement information.

3. The vehicle control method according to claim 1 or 2, characterized in that: The scanning and ranging of the acquired image to obtain the first ranging information includes: Based on the collected image, determine the camera light spot position and image area angle; Obtaining a preset scanning distance and a preset scanning number corresponding to the vehicle; Calculating according to the image area angle and the preset number of scans to obtain a scan angle; Based on the camera light spot position, scanning and measuring the range of the collected image according to the scanning angle to determine whether there is a target scanning point, where the target scanning point is a point on the obstacle surface within a preset scanning distance from the camera light spot position; In the case where the target scanning point exists, determining a camera ranging value, wherein the camera ranging value is a distance value between the target scanning point and the vehicle; In the absence of the target scanning point, determining a preset distance measurement value as the camera distance measurement value; Based on the camera ranging value, the first ranging information is generated.

4. The vehicle control method according to claim 1, characterized in that: The first ranging information and the second ranging information are merged to obtain target distance information, including: Extracting a target scanning point and a camera ranging value corresponding to the target scanning point from the first ranging information; Based on the target scanning point, radar ranging is performed in combination with the radar signal to obtain second ranging information corresponding to the target scanning point, where the second ranging information includes a radar ranging value; According to a preset fusion coefficient, the camera distance measurement value and the radar distance measurement value are fused and calculated to obtain a target distance value; The target distance information is generated according to the target distance value.

5. The vehicle control method according to claim 4, characterized in that: The performing radar ranging based on the target scanning point and in combination with the radar signal to obtain second ranging information corresponding to the target scanning point includes: determining whether there is a target radar point in the radar signal that matches the target scanning point; In a case where the target radar point matching the target scanning point exists in the radar signal, determining a radar ranging value corresponding to the target radar point based on a distance value between the target radar point and the vehicle; In a case where the target radar point matching the target scanning point does not exist in the radar signal, determining the camera ranging value as the radar ranging value; The second ranging information is generated according to the radar ranging value.

6. The vehicle control method according to any one of claims 1 to 5, characterized in that: The obtaining of environment closure information according to the target distance information and the obstacle type information includes: Extracting a target distance value from the target distance information; Determining a type weight corresponding to the obstacle type information; The target distance value and the type weight are used to perform weighted calculation to obtain environmental closure information.

7. The vehicle control method according to claim 6, characterized in that: The step of performing weighted calculation using the target distance value and the type weight to obtain the environmental closure information includes: Determine a camera identifier corresponding to the target distance information; Obtaining a preset camera weight corresponding to the camera identifier; Based on the camera weight, a weighted calculation is performed in combination with the target distance value and the type weight to obtain the environment closure information.

8. A vehicle control device, characterized in that: include: An acquisition module, used for acquiring images and radar signals; A processing module, used to obtain first ranging information and obstacle type information according to the collected image, and obtain second ranging information according to the radar signal; A fusion module, used for fusing the first ranging information and the second ranging information to obtain target distance information; An environment closure module, used to obtain environment closure information according to the target distance information and the obstacle type information; A control module is used to control the vehicle based on the environmental enclosure information to obtain a vehicle control result.

9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; The processor is used to implement the vehicle control method described in any one of claims 1 to 7 when executing the program stored in the memory.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the vehicle control method according to any one of claims 1 to 7 is implemented.

Images