Environment sensing method and device, vehicle, storage medium and program product

Abstract

The invention relates to an environment sensing method and device, a vehicle, a storage medium and a program product. The method comprises the steps that first vehicle-mounted sensing information collected by a sensor of a target vehicle is acquired; under the condition that the target area exists around the target vehicle, auxiliary equipment is controlled to conduct environment sensing on the target area, and first airborne sensing information is obtained; and determining a target perception map based on the first vehicle-mounted perception information and the first airborne perception information. Therefore, if a target area in which the target vehicle-mounted sensing information does not meet the reliability condition exists around the target vehicle, the auxiliary equipment is controlled to perform environment sensing on the target area to obtain first airborne sensing information, and a target sensing map is determined based on the first vehicle-mounted sensing information and the first airborne sensing information; the accuracy of the determined target sensing map can be improved, so that the running safety of the target vehicle is improved in the process of controlling the running state of the target vehicle based on the target sensing map.

Description

Technical Field

[0001] This application relates to the technical field of environmental perception, and particularly relates to an environmental perception method, device, vehicle, storage medium, and program product. Background Art

[0002] Nowadays, with the gradual implementation of the systematic construction of urban digitalization and intelligence, various transportation systems and vehicle forms have developed rapidly, and the intelligent levels of many traffic participants have also been continuously improved. The driving levels of many vehicles in scenarios such as automatic start / stop and automatic parking have become increasingly enhanced. Functions such as automatic position finding and automatic parking have become new trends in scenario applications.

[0003] In related technologies, the perception system of autonomous vehicles mainly performs environmental perception based on in-vehicle sensor systems. Since the environmental perception and simultaneous localization and mapping (SLAM) are realized from the first perspective, the accuracy is relatively low, and the reliability of environmental perception is usually low due to the accuracy of vehicle sensors and the complex environment around the vehicle, thus affecting the safe operation of autonomous vehicles. Summary of the Invention

[0004] This application provides an environmental perception method, device, vehicle, storage medium, and program product. This method can improve the safety of the target vehicle during the process of controlling the running state of the vehicle based on the target perception map.

[0005] The technical solution of this application is realized as follows: An embodiment of this application provides a control method, including: obtaining first in-vehicle perception information collected by sensors of a target vehicle; when there is a target area around the target vehicle, controlling an auxiliary device to perform environmental perception on the target area to obtain first airborne perception information; the target in-vehicle perception information corresponding to the target area does not meet the reliability condition, and the target in-vehicle perception information corresponding to the target area is determined based on the first in-vehicle perception information; determining a target perception map based on the first in-vehicle perception information and the first airborne perception information; and controlling the running state of the target vehicle based on the target perception map.

[0006] According to the above technical means, when it is determined that there is a target area around the target vehicle where the target in-vehicle perception information does not meet the reliability condition, by controlling the auxiliary device to perform environmental perception on the target area to obtain the first airborne perception information, and determining the target perception map based on the first in-vehicle perception information and the first airborne perception information, the accuracy of the determined target perception map can be improved, thereby improving the safety of the target vehicle during the process of controlling the running state of the vehicle based on the target perception map.

[0007] In some embodiments, the control auxiliary device performs environmental perception on the target area to obtain the first airborne perception information, including: determining the first position information of the target area; sending the first position information to the auxiliary device so that the auxiliary device performs environmental perception based on the position information; receiving the first airborne perception information fed back by the auxiliary device based on the first position information.

[0008] According to the above technical means, by sending the position information of the target area to the auxiliary device, it is convenient for the auxiliary device to determine the target area that needs to perform environmental perception based on the position information, and improve the environmental perception efficiency of the auxiliary device.

[0009] In some embodiments, the auxiliary device includes a flight device; sending the position information of the target area to the auxiliary device so that the auxiliary device performs environmental perception based on the position information includes: sending the position information to the flight device so that the flight device flies above the target area based on the position information and performs environmental perception on the target area from a top-down perspective.

[0010] According to the above technical means, by sending the position information of the target area to the flight device, for scenarios where the obstacles in the target area are relatively close to the target vehicle or the target area corresponds to a narrow environment, etc., the flight device can fly above the target area based on the position information to perform more accurate and comprehensive environmental perception on the target area, thereby improving the accuracy of the first airborne perception information.

[0011] In some embodiments, determining the target perception map based on the first vehicle-mounted perception information and the first airborne perception information includes: creating a first perception map based on the first vehicle-mounted perception information; creating a second perception map based on the first airborne perception information; performing a fusion process on the first perception map and the second perception map based on the first weight corresponding to the first perception map and the second weight corresponding to the second perception map to obtain the target perception map.

[0012] According to the above technical means, performing a fusion process on the first perception map and the second perception map based on the first weight corresponding to the first perception map and the second weight corresponding to the second perception map can improve the accuracy of the target perception map.

[0013] In some embodiments, the environmental perception method further includes: obtaining the first configuration information of the sensors of the target vehicle; determining the first weight and / or the second weight based on the first configuration information.

[0014] According to the above technical means, determine the first weight corresponding to the first perception map and / or the second weight corresponding to the second perception map based on the configuration information of the sensors of the target vehicle, so that the weights used in the fusion process of the two perception maps are associated with the configuration information of the vehicle-mounted sensors, and the configuration information of the vehicle sensors can affect the accuracy of the created perception map. The first weight and the second weight can determine the contribution ratio of the respective perception maps in the fusion process. Therefore, based on the configuration information of the sensors of the target vehicle, the rationality and accuracy of the determined first weight and second weight can be improved.

[0015] In some embodiments, the environmental perception method further includes: determining the reliability degree of the target vehicle-mounted perception information corresponding to the target area; based on the reliability degree, determining the first weight and / or the second weight.

[0016] According to the above technical means, determine the first weight corresponding to the first perception map and / or the second weight corresponding to the second perception map based on the reliability degree of the target vehicle-mounted perception information corresponding to the target area, so that the weights used in the fusion process of the two perception maps are associated with the reliability degree of the vehicle-mounted perception information, thereby can improve the rationality and accuracy of the determined first weight and second weight.

[0017] In some embodiments, the environmental perception method further includes: based on the first vehicle-mounted perception information, determining the target vehicle-mounted perception information corresponding to at least one target candidate area around the target vehicle; in the case where the target vehicle-mounted perception information corresponding to the target candidate area does not meet the reliability condition, determining the target candidate area as the target area.

[0018] According to the above technical means, by determining, from at least one target candidate area around the target vehicle, the target candidate area where the target vehicle-mounted perception information does not meet the reliability condition as the target area, the accuracy of the identified target area can be improved.

[0019] In some embodiments, the first vehicle-mounted perception information includes the first images corresponding to at least one first candidate area around the target vehicle, and the target vehicle-mounted perception information includes the target image; based on the first vehicle-mounted perception information, determining the target vehicle-mounted perception information corresponding to at least one target candidate area around the target vehicle includes: based on the first images corresponding to at least one first candidate area, determining the target images corresponding to at least one target candidate area.

[0020] According to the above technical means, by determining the second images corresponding to at least one target candidate area through the first images corresponding to at least one first candidate area, it is possible to more accurately determine the target area from each target candidate area based on the second images corresponding to at least one target candidate area subsequently.

[0021] In some embodiments, at least one target candidate region includes at least one second candidate region and / or at least one third candidate region. The second candidate region includes the region where at least two first candidate regions overlap, and the third candidate region includes the region in the first candidate regions that does not overlap with other first candidate regions.

[0022] According to the above technical means, since the second candidate region is usually located near the edge of the corresponding first candidate region, there may be a difference in the image perception ability of the vehicle's sensor for this part and the image perception ability of the third candidate region. By using the second candidate region and / or the third candidate region as the target candidate region, the area around the target vehicle can be divided with finer granularity, so that the target area around the target vehicle can be identified more accurately.

[0023] In some embodiments, the target vehicle-mounted perception information corresponding to the target candidate region includes the target distance between the target vehicle and at least one obstacle in the target candidate region. The environmental perception method further includes: when the target distance corresponding to the target candidate region is less than the distance threshold, determining that the target vehicle-mounted perception information corresponding to the target candidate region does not meet the reliability condition.

[0024] According to the above technical means, if the target distance corresponding to the target candidate region is less than the distance threshold, it indicates that the vehicle-mounted perception information collected by the vehicle-mounted sensor may be unreliable, or more perception information of the target candidate region needs to be obtained in the current scenario to improve the reliability of the perception information. Therefore, in this case, determining that the target vehicle-mounted perception information corresponding to the target candidate region does not meet the reliability condition can improve the accuracy of judging the reliability degree of the target vehicle-mounted perception information.

[0025] In some embodiments, controlling the running state of the target vehicle based on the target perception map includes: determining the position information of the target parking space available for the target vehicle to park in the current scenario of the target vehicle; and controlling the target vehicle to park in the target parking space based on the target perception map and the position information of the target parking space.

[0026] According to the above technical means, since the target perception map integrates vehicle-mounted perception information and airborne perception information and has high accuracy, when the position information of the target parking space in the current scenario of the target vehicle is determined, controlling the target vehicle to park based on the target perception map and the position information of the target parking space can improve the accuracy and efficiency of the target vehicle parking in the target parking space.

[0027] In some embodiments, determining the position information of a target parking space available for a target vehicle in the scene where the target vehicle is currently located includes: sending a control instruction to an auxiliary device to control the auxiliary device to perform environmental perception on the scene where the target vehicle is currently located; obtaining second on-vehicle perception information fed back by the auxiliary device based on the control instruction; and determining the position information of the target parking space based on the second on-vehicle perception information and second on-board perception information obtained by the sensors of the target vehicle for environmental perception around the target vehicle.

[0028] According to the above technical means, by sending a control instruction to the auxiliary device, the second on-board perception information obtained by the auxiliary device after performing environmental perception on the scene where the target vehicle is currently located can be obtained. Further, based on the combination of the second on-board perception information and the second on-vehicle perception information obtained by the sensors of the target vehicle for environmental perception around the target vehicle, the position information of the target parking space is determined, which can improve the accuracy of the determined position of the target parking space.

[0029] In some embodiments, the environmental perception method further includes: determining a moving path of the target vehicle based on the second on-board perception information; and controlling the target vehicle to move following the auxiliary device based on the moving path, and controlling the sensors of the target vehicle to perform environmental perception around the target vehicle during the process of the target vehicle moving following the auxiliary device to obtain second on-vehicle perception information.

[0030] According to the above technical means, the moving path of the target vehicle is determined according to the second on-board perception information sensed by the auxiliary device, and the target vehicle is controlled to move following the auxiliary device based on the moving path. During the process of controlling the sensors of the target vehicle to perform environmental perception around the target vehicle while the target vehicle is moving following the auxiliary device, second on-vehicle perception information is obtained, so that the target parking space can be determined based on the fusion result of the second on-vehicle perception information and the second on-board perception information subsequently, thereby improving the accuracy of the determined target parking space.

[0031] An embodiment of the present application provides an environmental perception device, including: A first acquisition module, configured to acquire first on-vehicle perception information collected by sensors of a target vehicle; A first control module, configured to control an auxiliary device to perform environmental perception on the target area to obtain first on-board perception information when it is determined that there is a target area around the target vehicle; the target on-vehicle perception information corresponding to the target area does not meet the reliability condition; the target on-vehicle perception information corresponding to the target area is determined based on the first on-vehicle perception information; A first determination module, configured to determine a target perception map based on the first on-vehicle perception information and the first on-board perception information.

[0032] An embodiment of the present application provides a vehicle, including a sensor, a memory, and a processor, where, the sensor is configured to collect first vehicle-mounted perception information; the memory stores a computer program that can run on the processor, and when the processor executes the program, the steps in the above method are implemented.

[0033] An embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps in the above method are implemented.

[0034] An embodiment of the present application provides a computer program product, including a computer program or instruction, and when the computer program or instruction is executed by a processor, the steps in the above method are implemented.

[0035] Advantages of the present application: According to the above technical means, in the case where it is determined that there is a target area around the target vehicle where the target vehicle-mounted perception information does not meet the reliability condition, by controlling the auxiliary device to perform environmental perception on the target area to obtain the first airborne perception information, and determining the target perception map based on the first vehicle-mounted perception information and the first airborne perception information, the accuracy of the determined target perception map can be improved, and thus the safety of the target vehicle operation can be improved during the process of controlling the vehicle operation state based on the target perception map.

[0036] It can facilitate the auxiliary device to determine the target area that needs to perform environmental perception based on the position information, and improve the environmental perception efficiency of the auxiliary device.

[0037] The accuracy of the target perception map can be improved.

[0038] Make the weights used in the fusion process of the two perception maps associated with the configuration information of the vehicle-mounted sensors, and the configuration information of the vehicle sensors can affect the accuracy of the created perception map. The first weight and the second weight can determine the contribution ratio of the respective perception maps in the fusion process. Therefore, based on the configuration information of the sensors of the target vehicle, the rationality and accuracy of the determined first weight and second weight can be improved.

[0039] Make the weights used in the fusion process of the two perception maps associated with the reliability degree of the vehicle-mounted perception information, so that the rationality and accuracy of the determined first weight and second weight can be improved.

[0040] The accuracy of the identified target area can be improved.

[0041] Make it possible to more accurately determine the target area from each target candidate area based on the second images respectively corresponding to at least one target candidate area subsequently.

[0042] The area around the target vehicle can be divided into finer granularity, so that the target area around the target vehicle can be identified more accurately.

[0043] The accuracy of judging the reliability of the target vehicle-mounted perception information can be improved.

[0044] The accuracy and efficiency of the target vehicle parking in the target parking space can be improved.

[0045] The accuracy of the position of the determined target parking space can be improved. Description of the Drawings

[0046] Figure 1 A schematic diagram of an automatic parking environment perception fusion mapping provided by an embodiment of the present application; Figure 2 A schematic flowchart of an environment perception method provided by an embodiment of the present application; Figure 3 A schematic diagram of an automatic parking enhanced perception system architecture based on vehicle-machine collaboration provided by an embodiment of the present application; Figure 4 A schematic diagram of an environment perception enhanced technology architecture provided by an embodiment of the present application; Figure 5 A schematic diagram of a vehicle networking collaboration technology architecture provided by an embodiment of the present application; Figure 6 A schematic diagram of the structure of an environment perception device provided by an embodiment of the present application. Detailed Embodiments

[0047] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the drawings in the embodiments of the present application.

[0048] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in conjunction with the drawings. The described embodiments should not be regarded as limitations of the present application. All other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present application.

[0049] In the following description, reference is made to "some embodiments / other embodiments", which describe subsets of all possible embodiments. However, it can be understood that "some embodiments / other embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0050] In the following description, the terms "first / second" only distinguish similar objects and do not represent a specific order for the objects. Understandably, "first / second" can be interchanged with a specific order or sequence when permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0051] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0052] Today, with the gradual implementation of the systematic construction of urban digitalization and intelligence, various transportation systems and vehicle forms have developed rapidly, and the intelligence levels of many transportation participants have also been continuously improved. The automation levels of many vehicles in scenarios such as automatic start / stop, parking in and out have been increasingly enhanced, and functions such as automatic position finding and automatic parking have become new trends in scenario applications.

[0053] Today, in the context of vehicle intelligence and networking, automatic parking technology, as one of the core technologies of autonomous driving, has received extensive attention from many OEM (Original Equipment Manufacturer) manufacturers and researchers in the field of intelligent driving. Its purpose is to assist or replace the driver's operations during the parking process to reduce the difficulties and pressures faced by the driver in complex parking environments such as narrow parking spaces and dim lighting. On the one hand, with the improvement of sensor performance levels, the industry has put forward higher requirements for the parking efficiency and effect of parking systems. On the other hand, automatic parking technology has gradually developed from traditional semi-automatic parking (Semi-Automatic Parking Assist, S-APA), full-automatic parking (Full-Automatic Parking Assist, F-APA), remote control parking (Remote Parking Assist, RPA) to memory parking (Home-Zone Parking Assist, HPA) and even automated valet parking (Automated Valet Parking, AVP).

[0054] The AVP (Automated Valet Parking) automated valet parking system, as an application of autonomous driving in the parking scenario, realizes the full-automatic valet parking function. Its ultimate goal is to replace traditional manual valet parking. It is the functional ceiling of the current parking scenario, involving complex sub-scenarios and high technical difficulties.

[0055] SLAM is based on Simultaneous Localization and Mapping. It refers to a subject equipped with specific sensors that, without prior environmental information, builds a model of the surrounding environment during movement and simultaneously estimates its own motion state. It is usually used in the fields of autonomous driving vehicles or autonomous mobile robots, mainly solving the problems of positioning and perception mapping. In the field of sensor-based SLAM, there are mainly LiDAR-based SLAM and Vision-based SLAM.

[0056] Vehicle-to-Everything (V2X) collaborative technology generally refers to the technology that uses modern network communication technology to achieve information exchange and sharing between vehicles (V2V), vehicles and roadside infrastructure (V2I), vehicles and pedestrians (Vehicle to Pedestrian, V2P), vehicles and the network (V2N), and vehicles and the cloud (V2C).

[0057] With the continuous progress of drone technology and the continuous expansion of travel application scenarios, in-vehicle drone technology has gradually been installed in some high-end vehicle products, and will also present a broader application prospect in the future. In-vehicle drone technology combines the flexibility of drones and the long-distance mobility of the host vehicle, with broad application prospects and huge market potential. In addition, in addition to the application of drone technology in traditional passenger cars, in the future, in-vehicle drone systems will be involved in many transportation fields such as flying cars, intelligent logistics vehicles, and autonomous mobile robots to achieve assisted driving.

[0058] In-vehicle drones belong to an emerging application field with relatively few related patents. Currently, there are patents based on the mechanical platform and structure of in-vehicle drones, while drone collaboration mainly focuses on target recognition and path planning for drone swarms. A method for collaborative path planning of a vehicle and an in-vehicle drone is mainly used for flight path planning to and from a supply point, focusing on researching the optimal path planning algorithm, which is different from the research direction of this application; a system and method for in-vehicle drone-assisted autonomous driving mainly solve the problem of enhancing high-altitude perspective information when there are obstacles or complex road conditions in front during the process of obtaining environmental information from a low-altitude perspective of the vehicle. It is mainly used to assist in obtaining road condition information during the autonomous driving process of autonomous vehicles, only briefly describing the collaborative operation process without designing a specific collaborative architecture; an image processing method for in-vehicle drones of an autonomous driving vehicle designs the recognition image processing and fusion algorithm for pedestrians and roadblocks in multiple directions during the driving process of the autonomous driving vehicle, focusing on theoretical modeling, which is significantly different from the architecture design and application scenario design of this application. Figure 1Schematic diagram of the fusion and mapping of the automatic parking environment perception provided by the embodiments of the present application. During the process of generating a panoramic top view based on the surround view mapping, the panoramic top view is mainly composed of four top views obtained by four fisheye cameras on the vehicle body based on the collected images after distortion correction and top view transformation, and the stitching seams are eliminated based on the processing of the image fusion algorithm. As Figure 1 shown, the surround view mapping mainly forms the front view area 102, the rear view area 103, the left view area 104, the right view area 105 (4 front view areas) of the vehicle 101, and the left front common area 106, the right front common area 107, the left rear common area 108, the right rear common area 109 (4 common stitching areas). As mentioned above, the 4 common stitching areas usually need to be fused, such as the weighted average fusion algorithm based on distance, etc. However, due to problems such as complex environment and low accuracy of the collected image sources themselves in the common stitching areas, the fusion accuracy is usually relatively poor, and it is impossible to construct a high-precision perception positioning SLAM map in complex and narrow environments, ultimately resulting in the failure of automatic vehicle parking.

[0059] In summary, on the one hand, the traditional automatic parking perception system mainly relies on the vehicle-mounted sensor system for environment perception. Since the environment perception and SLAM mapping are realized from the first perspective, the accuracy is relatively low, and parking failure usually occurs due to low perception reliability in some complex parking space environments; on the other hand, traditional automatic parking requires the vehicle to autonomously find the parking space and plan the path based on SLAM perception. Since the vehicle speed is low and the flexibility is poor, it is impossible to efficiently find the parking space and plan the path.

[0060] Based on the problems existing in the related technologies, the embodiments of the present application provide an environment perception method, which can be applied to autonomous driving vehicles, semi-autonomous driving vehicles, non-autonomous driving vehicles, or other remote devices, etc. As Figure 2 shown, it is a schematic flowchart of an environment perception method provided by the embodiments of the present application. The method includes the following steps S201 to step S203: Step S201, obtain the first vehicle-mounted perception information collected by the sensors of the target vehicle.

[0061] Here, the target vehicle can be an autonomous driving vehicle, a semi-autonomous driving vehicle, a non-autonomous driving vehicle, etc., and the sensors of the target vehicle can include vehicle-mounted cameras, vehicle-mounted radars, etc. The first vehicle-mounted perception information can include the position information of obstacles around the vehicle collected by the vehicle-mounted radar, and images collected by the vehicle-mounted cameras, etc.

[0062] In some embodiments, the first vehicle-mounted perception information can be collected in real time by the sensors of the target vehicle, and the first vehicle-mounted perception information can be obtained when receiving the vehicle-mounted perception information acquisition instruction sent by the control device or remote device of the target vehicle.

[0063] In step S202, when there is a target area around the target vehicle, control the auxiliary device to perform environmental perception on the target area to obtain first on-vehicle perception information.

[0064] Here, the target area may include at least one of the left area, right area, front area, rear area, left front area, left rear area, right front area, and right rear area of the target vehicle, etc. The target on-vehicle perception information corresponding to the target area does not meet the reliability condition, and the target on-vehicle perception information corresponding to the target area can be determined based on the first on-vehicle perception information. For example, by performing fusion or splicing processing on the first on-vehicle perception information to obtain the target on-vehicle perception information corresponding to the target area. The auxiliary device may include, but is not limited to, flying information collection devices such as flying devices or flying apparatuses (such as drones), and the auxiliary device may also include, but is not limited to, information collection devices such as robots and other vehicles different from the target vehicle.

[0065] In some embodiments, the first on-vehicle perception information may include images of different areas around the target vehicle collected by on-vehicle sensors, and may also include the position information of obstacles in different areas around the target vehicle collected by on-vehicle radars. It is possible to identify the clarity of the images of different areas collected, and determine whether there is a target area around the target vehicle according to the clarity of the images; it is also possible to analyze the position information of obstacles in different areas around the target vehicle, and determine whether there is a target area around the target vehicle according to the distance between the target vehicle and the obstacles around it. The description of the first on-vehicle perception information here, and determining whether there is a target area around the target vehicle based on the first on-vehicle perception information is only an exemplary illustration, and this application is not limited thereto.

[0066] In some embodiments, when it is determined that there is a target area around the target vehicle, a control instruction can be sent to the auxiliary device through the on-vehicle control system of the auxiliary device, so as to start the auxiliary device and control the auxiliary device to move to the position corresponding to the target area for environmental perception.

[0067] In some embodiments, the auxiliary device is configured with sensors such as on-vehicle cameras and on-vehicle radars. The first on-vehicle perception information includes image information collected by the on-vehicle camera and the position information of obstacles around the target vehicle collected by the on-vehicle radar. The auxiliary device can move to positions such as the side, above (directly above or obliquely above) of the target area to perform environmental perception on the target area, and obtain the image information within the target area and the position information of obstacles around the target vehicle.

[0068] In step S203, determine a target perception map based on the first on-vehicle perception information and the first on-vehicle perception information.

[0069] In some embodiments, the first vehicle-mounted perception information may include image information around the target vehicle and the position information of obstacles around the target vehicle. The first airborne perception information may include image information within a target area around the target vehicle and the position information of obstacles within the target area around the target vehicle. The position information of obstacles within the target area around the target vehicle can be converted into position information in a vehicle body coordinate system established with the center of the target vehicle as the origin, and based on the converted position information of obstacles within the target area around the target vehicle and the position information of obstacles around the target vehicle in the first vehicle-mounted perception information, the target position information of obstacles around the target vehicle is determined; based on the image information around the target vehicle in the first vehicle-mounted perception information and the image information within the target area around the target vehicle, the target image information around the target vehicle is determined; based on the target position information of obstacles around the target vehicle and the target image information around the target vehicle, a target perception map is created.

[0070] In some other embodiments, a first perception map may be created based on the image information around the target vehicle and the position information of obstacles around the target vehicle in the first vehicle-mounted perception information, and a second perception map may be created based on the image information within the target area around the target vehicle and the position information of obstacles within the target area around the target vehicle in the first airborne perception information. The first perception map and the second perception map are fused to determine the target perception map.

[0071] In some embodiments, the operating state of the target vehicle may be controlled based on the target map information. For example, during the autonomous driving of the target vehicle, the moving path of the target vehicle can be determined based on the target perception map, and the target vehicle is controlled to move based on the moving path. Specifically, during the process of controlling the target to move from the current position to the target position, the moving path from the current position to the target position can be determined according to the target perception map, and the vehicle is controlled to move from the current position to the target position based on the moving path.

[0072] In some embodiments, in an automatic parking scenario, the target vehicle can be controlled to perform automatic parking based on the target perception map; for a non-automatic or semi-automatic parking scenario, the target perception map can be output to the in-vehicle display interface for display, thereby guiding the driver to park the vehicle.

[0073] In an embodiment of the present application, first vehicle-mounted perception information collected by sensors of a target vehicle is obtained; when it is determined that there is a target area around the target vehicle, an auxiliary device is controlled to perform environmental perception on the target area to obtain first airborne perception information; the target vehicle-mounted perception information corresponding to the target area does not meet the reliability condition; the target vehicle-mounted perception information corresponding to the target area is determined based on the first vehicle-mounted perception information; a target perception map is determined based on the first vehicle-mounted perception information and the first airborne perception information. In this way, when it is determined that there is a target area around the target vehicle where the target vehicle-mounted perception information does not meet the reliability condition, by controlling the auxiliary device to perform environmental perception on the target area to obtain the first airborne perception information, and determining the target perception map based on the first vehicle-mounted perception information and the first airborne perception information, the accuracy of the determined target perception map can be improved, so as to improve the safety of the target vehicle during the process of controlling the vehicle running state based on the target perception map.

[0074] In some embodiments, controlling the auxiliary device to perform environmental perception on the first area to obtain the first airborne perception information in step S202 may include the following steps S2021 to S2023: Step S2021, determine the position information of the target area.

[0075] In some embodiments, the position information of the target area may be the position coordinates of the target area in a vehicle body coordinate system established with the center of the target vehicle as the origin. The position coordinates may include the position coordinates of the center of the target area and the position coordinates of multiple points on the edge of the target area. The size and position of the target area can be uniquely determined through the position coordinates.

[0076] In some embodiments, since the first vehicle-mounted perception information includes the image information around the target vehicle and the position information of the obstacles around the target vehicle, the position information of the target area can be determined by combining the image information corresponding to the target area and the position information of the obstacles in the target area. For example, the target obstacle in the target area can be first determined through the image information corresponding to the target area, and then the position information of the target area can be determined by combining the position information of the target obstacle.

[0077] Step S2022, send the position information of the target area to the auxiliary device so that the auxiliary device performs environmental perception based on the position information.

[0078] In some embodiments, after the position information of the target area is determined, the position information of the target area can be sent to the auxiliary device. After receiving the position information, the auxiliary device can convert the position information into the first position information in the coordinate system established with its own center as the origin, and then move to the area corresponding to the first position information to realize the environmental perception of the target area.

[0079] In some embodiments, the target area may include one or more. When the target area includes one, the auxiliary device can be controlled to move above the target area to perform environmental perception on the target area and obtain the first airborne perception information. When the target area includes multiple, the target vehicle or the remote device can set the environmental perception order of the auxiliary device for each target area and send the environmental perception order to the auxiliary device, so that the auxiliary device performs environmental perception on each target area based on the environmental perception order. Alternatively, the auxiliary device itself can also determine the environmental perception order of each target area, and the auxiliary device performs environmental perception on each target area based on the self-set environmental perception order.

[0080] Step S2023: Receive the first airborne perception information fed back by the auxiliary device based on the position information of the target area.

[0081] In some embodiments, after the auxiliary device performs environmental perception on the target area and obtains the first airborne perception information, it can actively send the first airborne perception information to the target vehicle or the remote device.

[0082] In some other embodiments, the target vehicle or the remote device can also send an airborne perception information acquisition request to the auxiliary device after a preset time after sending the position information of the target area to the auxiliary device. The auxiliary device feeds back the collected first airborne perception information based on the airborne perception information acquisition request.

[0083] In the above embodiments, by sending the position information of the target area to the auxiliary device, it is convenient for the auxiliary device to determine the target area that needs to perform environmental perception based on the position information, and improve the environmental perception efficiency of the auxiliary device.

[0084] In some embodiments, the auxiliary device includes a flying device. Sending the position information of the target area to the auxiliary device in step S2022 to enable the auxiliary device to perform environmental perception based on the position information may include the following step S20221: Step S20221: Send the position information to the flying device, so that the flying device flies above the target area based on the position information and performs environmental perception on the target area from a top-down perspective.

[0085] In some embodiments, the flying device can fly directly above or obliquely above the target area based on the position information sent by the target vehicle to perform environmental perception on the target area; or, the flying device can first fly to any position above the target area, and then adjust its own position or pose based on its height from the target area, the distribution of obstacles in the target area, etc., to better perform environmental perception on the target area.

[0086] In some other embodiments, the target vehicle may also send the position information of the target area and the acquisition parameters of the target area to the flying device. Among them, the acquisition parameters may be the flight altitude, acquisition azimuth, acquisition viewing angle, etc. for the flying device to perform environmental perception on the target area, and the acquisition parameters may be determined by the target vehicle based on the target vehicle-mounted perception information corresponding to the target area.

[0087] In the above embodiments, by sending the position information of the target area to the flying device, for scenarios where the obstacles in the target area are relatively close to the target vehicle or the target area corresponds to a narrow environment, etc., the flying device can fly to the upper part of the target area based on this position information to perform more accurate and comprehensive environmental perception on the target area, thereby improving the accuracy of the first airborne perception information.

[0088] In some embodiments, determining the target perception map based on the first vehicle-mounted perception information and the first airborne perception information in step S203 may include the following steps S2031 to S2033: Step S2031, creating a first perception map based on the first vehicle-mounted perception information.

[0089] In some embodiments, the first perception map may be created according to the image information around the target vehicle and the position information of the obstacles around the target vehicle in the first vehicle-mounted perception information. For example, the SLAM technology may be used to associate the image information around the target vehicle and the position information of the obstacles around the target vehicle, and establish the first perception map.

[0090] Step S2032, creating a second perception map based on the first airborne perception information.

[0091] In some embodiments, the second perception map may be created according to the image information corresponding to the first area in the first airborne perception information and the position information of the obstacles in the first area in the first airborne perception information.

[0092] In some other embodiments, the position information of the obstacles in the first area in the first airborne perception information may be converted into the position coordinates in the same coordinate system corresponding to the position information of the obstacles around the target vehicle in the first vehicle-mounted perception information, and then the second perception map may be created based on the converted position coordinates. For example, when the position information of the obstacles around the target vehicle in the first vehicle-mounted perception information is the position coordinates in the vehicle body coordinate system established with the vehicle body center as the origin, the position information of the obstacles around the target vehicle in the first vehicle-mounted perception information may be kept unchanged, and the position information of the obstacles in the first area in the first airborne perception information may be converted into the position coordinates in this vehicle body coordinate system. Then, based on the converted position coordinates and the image information in the first area in the first airborne perception information, the second perception map may be established.

[0093] Step S2033: Based on the first weight corresponding to the first perception map and the second weight corresponding to the second perception map, perform a fusion process on the first perception map and the second perception map to obtain a target perception map.

[0094] Here, both the first weight and the second weight can be pre-determined, and the first weight can be less than or equal to the second weight.

[0095] In some embodiments, the first perception map and the second perception map may include image information around the target vehicle and position information of obstacles around the target vehicle. The fusion of the first perception map and the second perception map may include the fusion of the position information of obstacles around the target vehicle in the first perception map and the corresponding position information in the second perception map, and the fusion of the image information around the target vehicle in the first perception map and the corresponding image information in the second perception map. Among them, the first weight can correct the position information and / or pixel values of the image in the first perception map, and the second weight can correct the second perception map and / or pixel values of the image.

[0096] In some embodiments, when the first perception map and the second perception map include position information in the same coordinate system, the first perception map and the second perception map can be directly fused based on the first weight and the second weight to obtain a target perception map.

[0097] In some embodiments, when the first perception map and the second perception map include position information in different coordinate systems, the position information in the two perception maps can be first converted into position information in the same coordinate system to obtain the converted perception maps, and then the two converted perception maps are fused to obtain a target perception map.

[0098] In the above embodiments, based on the first weight corresponding to the first perception map and the second weight corresponding to the second perception map, performing a fusion process on the first perception map and the second perception map can improve the accuracy of the target perception map.

[0099] In some embodiments, the above method may further include the following steps S204 to S205: Step S204: Obtain the configuration information of the sensors of the target vehicle.

[0100] Here, the configuration information may include the type, quantity of the sensors of the target vehicle, and the configuration level of the sensors, etc. The configuration level of the sensors of the target vehicle can be determined according to the type, quantity, etc. of the sensors of the target vehicle. For example, the more types and quantities of sensors, the higher the corresponding sensor configuration level.

[0101] In some embodiments, the categories and quantities of sensors of a target vehicle can be obtained, and based on a preset correspondence relationship established in advance between the categories and quantities of vehicle sensors and the sensor configuration levels, the sensor configuration level of the target vehicle can be determined.

[0102] Step S205: Based on the configuration information, determine the first weight and / or the second weight.

[0103] In some embodiments, the first weight corresponding to the first perception map can be determined according to the configuration information of the sensors of the target vehicle, the second weight corresponding to the second perception map can also be determined according to the configuration information of the sensors of the target vehicle, or the first weight corresponding to the first perception map and the second weight corresponding to the second perception map can be determined simultaneously according to the configuration information of the sensors of the target vehicle.

[0104] In some embodiments, the first weight can be determined according to the correspondence relationship established in advance between the vehicle sensor configuration level and the weight of the perception map, and the sensor configuration level of the target vehicle; or the first weight can be calculated according to the correlation relationship model (such as a pre-trained neural network model) established in advance between the vehicle sensor configuration level and the weight of the perception map, and the sensor configuration level of the target vehicle.

[0105] In some embodiments, the sum of the first weight and the second weight can be a fixed parameter. After the first weight is determined, the second weight can be determined according to this fixed parameter and the first weight. For example, if the first weight is 0.4 and the fixed parameter is 1, then the second weight can be determined to be 0.6.

[0106] In the above embodiments, the first weight corresponding to the first perception map and / or the second weight corresponding to the second perception map are determined based on the configuration information of the sensors of the target vehicle, so that the weights used in the fusion process of the two perception maps are associated with the configuration information of the vehicle-mounted sensors, and the configuration information of the vehicle sensors can affect the accuracy of the created perception maps. The first weight and the second weight can determine the contribution ratio of the respective perception maps in the fusion process. Therefore, based on the configuration information of the sensors of the target vehicle, the rationality and accuracy of the determined first weight and second weight can be improved.

[0107] In some embodiments, the above method may further include the following steps S206 to S207: Step S206: Determine the reliability degree of the target vehicle-mounted perception information corresponding to the target area.

[0108] Here, the reliability degree of the target vehicle-mounted perception information can represent the accuracy degree of the target vehicle-mounted perception information.

[0109] In some embodiments, the target vehicle-mounted perception information may include a first distance between the target vehicle and the nearest obstacle within the target area. The first airborne perception information may include a first reference distance between the target vehicle and the nearest obstacle within the first area. The ratio of the first distance to the first reference distance may be determined as the reliability degree of the target perception information. Alternatively, the ratio of the first distance to a second reference distance may be determined as the reliability degree of the target perception information, where the second reference distance may be the distance between the target vehicle and the nearest obstacle within the target area perceived by other electronic devices different from the auxiliary device.

[0110] In some embodiments, the target vehicle-mounted perception information may include a first clarity of an image within the target area. The first airborne perception information may include a first reference clarity of the image within the target area. The ratio of the first clarity to the first reference clarity may be determined as the reliability degree of the target perception information. Alternatively, the ratio of the first clarity to a second reference clarity may be determined as the reliability degree of the target perception information, where the second reference clarity may be the clarity of the image within the target area perceived by other electronic devices different from the auxiliary device.

[0111] Step S207: Determine the first weight and / or the second weight based on the reliability degree.

[0112] In some embodiments, the first weight may be determined based on the reliability degree of the target vehicle-mounted perception information. The second weight may be determined based on the reliability degree of the target vehicle-mounted perception information. Alternatively, the first weight and the second weight may be determined simultaneously based on the reliability degree of the target vehicle-mounted perception information.

[0113] In some embodiments, the first weight may be determined according to the pre-established correspondence between the reliability degree of the sensor perception information and the weight of the perception map, and the reliability degree of the target vehicle-mounted perception information corresponding to the target area. Alternatively, the first weight may be calculated according to the pre-established correlation relationship model between the reliability degree of the sensor perception information and the weight of the perception map, and the reliability degree of the target vehicle-mounted perception information corresponding to the target area.

[0114] In some embodiments, the sum of the first weight and the second weight may be a fixed parameter. After the first weight is determined, the second weight may be determined according to the fixed parameter and the first weight.

[0115] In some embodiments, the value range of the reliability level can be [0, 100%]. After determining the reliability level of the target vehicle perception information corresponding to the target area, the reliability level can also be directly determined as the first weight, and the second weight can be determined based on the first weight and a fixed parameter (the sum of the first weight and the second weight). For example, if the determined reliability level corresponding to the target vehicle perception information is 35%, the first weight can be determined as 35% and the second weight can be determined as 75%.

[0116] In the above embodiments, based on the reliability level of the target vehicle perception information corresponding to the target area, the first weight corresponding to the first perception map and / or the second weight corresponding to the second perception map are determined, so that the weights used in the fusion process of the two perception maps are associated with the reliability degree of the vehicle perception information, thereby the rationality and accuracy of the first weight and the second weight can be determined and improved.

[0117] In some embodiments, the above method may further include the following steps S208 to S209: Step S208, based on the first vehicle perception information, determine the target vehicle perception information corresponding to at least one target candidate area around the target vehicle.

[0118] Here, the target candidate area may be one or more of the left area, right area, front area, rear area, left front area, left rear area, right front area, and right rear area of the target vehicle, etc.

[0119] In some embodiments, the first vehicle perception information may include the target vehicle perception information corresponding to the left area, right area, front area, and rear area of the target vehicle respectively. The target vehicle perception information corresponding to the left front area of the target vehicle can be determined by the target vehicle perception information corresponding to the left area of the target vehicle and the target vehicle perception information corresponding to the front area of the target vehicle; the target vehicle perception information corresponding to the left rear area of the target vehicle can be determined by the target vehicle perception information corresponding to the left area of the target vehicle and the target vehicle perception information corresponding to the rear area of the target vehicle; the target vehicle perception information corresponding to the right front area of the target vehicle can be determined by the target vehicle perception information corresponding to the right area of the target vehicle and the target vehicle perception information corresponding to the front area of the target vehicle; the target vehicle perception information corresponding to the right rear area of the target vehicle can be determined by the target vehicle perception information corresponding to the right area of the target vehicle and the target vehicle perception information corresponding to the rear area of the target vehicle.

[0120] Step S209, in the case where the target vehicle perception information corresponding to the target candidate area does not meet the reliability condition, determine the target candidate area as the target area.

[0121] In some embodiments, the target vehicle perception information corresponding to the target candidate region may include an image corresponding to the target candidate region. Whether the target vehicle perception information meets the reliability condition may be determined according to the relationship between the clarity of the image and the image clarity threshold. For example, if the clarity of the image corresponding to the target candidate region is less than the image clarity threshold, it may be determined that the target vehicle perception information corresponding to the target candidate region does not meet the reliability condition.

[0122] In some embodiments, when there are multiple target candidate regions, whether the vehicle perception information corresponding to each target candidate region meets the reliability condition may be determined in sequence, and finally the target region may be determined from the multiple target candidate regions. The target region may include one or more.

[0123] In the above embodiments, by determining, from at least one target candidate region around the target vehicle, the target candidate region whose target vehicle perception information does not meet the reliability condition as the target region, the accuracy of the identified target region can be improved.

[0124] In some embodiments, the first vehicle perception information includes first images respectively corresponding to at least one first candidate region around the target vehicle, and the target vehicle perception information includes target images. The step of determining the target vehicle perception information respectively corresponding to at least one target candidate region around the target vehicle based on the first vehicle perception information in step S208 may include the following step S2081: Step S2081: Determine the target images respectively corresponding to at least one target candidate region based on the first images respectively corresponding to at least one first candidate region.

[0125] Here, the first candidate region may be at least one of the left region, right region, front region, and rear region of the target vehicle, and the target candidate region may be at least one of the left region, right region, front region, rear region, left front region, left rear region, right front region, and right rear region of the target vehicle, etc.

[0126] In some embodiments, when the first candidate region and the target candidate region are the same, the first image corresponding to the first candidate region may be directly determined as the target image corresponding to the corresponding target candidate region; when there are multiple first candidate regions, the first images respectively corresponding to the multiple first candidate regions may be spliced or fused to obtain the target image corresponding to the corresponding target candidate region.

[0127] Exemplarily, if the first candidate region includes the left region and the front region of the target vehicle, and the target candidate region includes the left region, the front region, and the left front region of the target vehicle, then the first image corresponding to the left region in the first candidate region can be determined as the second image corresponding to the left region in the target candidate region, the first image corresponding to the front region in the first candidate region can be determined as the second image corresponding to the front region in the target candidate region, the first image corresponding to the left region in the first candidate region and the first image corresponding to the front region in the first candidate region are subjected to splicing processing, and the spliced common region is determined as the second image corresponding to the left front region in the target candidate region. It should be noted that the above description of the first candidate region and the target candidate region is only exemplary, and the present application does not limit this.

[0128] In the above embodiments, the second images corresponding to at least one target candidate region are determined through the first images corresponding to at least one first candidate region, so that the target region can be more accurately determined from each target candidate region based on the second images corresponding to at least one target candidate region subsequently.

[0129] In some embodiments, at least one target candidate region includes at least one second candidate region and / or at least one third candidate region. The second candidate region includes the region where at least two first candidate regions overlap, and the third candidate region includes the region in the first candidate region that does not overlap with other first candidate regions.

[0130] In some embodiments, when the first candidate region includes the left region, the right region, and the front region of the target vehicle, the second candidate region may be the overlapping region after splicing the left region and the front region (i.e., the left front region), and / or the overlapping region after splicing the right region and the front region (i.e., the right front region); when the first candidate region includes the left region, the right region, and the rear region of the target vehicle, the second candidate region may be the overlapping region after splicing the left region and the rear region (i.e., the left rear region), and / or the overlapping region after splicing the right region and the rear region (i.e., the right rear region). In some embodiments, the third candidate region may be one or more of the left region, the right region, the front region, and the rear region of the target vehicle.

[0131] In the above embodiments, since the second candidate region is usually located at the edge of the corresponding first candidate region, there may be a difference in the image perception ability of the vehicle's sensor for this part and the image perception ability of the third candidate region. By using the second candidate region and / or the third candidate region as the target candidate region, the area around the target vehicle can be divided into finer granularity, so that the target area around the target vehicle can be identified more accurately.

[0132] In some embodiments, the target vehicle perception information corresponding to the target candidate region includes the target distance between the target vehicle and at least one obstacle within the target candidate region; the above method may further include the following step S210: Step S210, when the target distance corresponding to the target candidate region is less than the distance threshold, determine that the target vehicle perception information corresponding to the target candidate region does not meet the reliability condition.

[0133] Here, the distance threshold may be a pre-determined safe distance between the vehicle and the obstacle.

[0134] In some embodiments, the target distance corresponding to the first target candidate region may be the distance between the target vehicle and any one obstacle within the target candidate region. The distances between the target vehicle and each obstacle within the target candidate region may be respectively compared with the distance threshold. If it is determined that there is a distance between the target vehicle and a certain obstacle within the target candidate region that is less than the distance threshold, it may be determined that the vehicle perception information corresponding to the target candidate region does not meet the reliability condition.

[0135] In the above embodiments, if the target distance corresponding to the target candidate region is less than the distance threshold, it indicates that the vehicle perception information collected by the vehicle-mounted sensor may be unreliable, or more perception information of the target candidate region needs to be obtained in the current scenario to improve the reliability of the perception information. Therefore, in this case, determining that the target vehicle perception information corresponding to the target candidate region does not meet the reliability condition can improve the accuracy of judging the reliability degree of the target vehicle perception information.

[0136] In some embodiments, the above method may further include the following steps S2041 to S2042: Step S2041, determine the position information of the target parking space available for the target vehicle to park in the scene where the target vehicle is currently located.

[0137] Here, the scene where the target vehicle is currently located may be an environment such as a garage or a parking lot where the target vehicle is located, and there may be no pre-established map data in the scene where the target vehicle is currently located.

[0138] In some embodiments, the position information of the target parking space available for the target vehicle to park in the environment where the target vehicle is currently located may be determined by an auxiliary device or other electronic devices, and the position information may include the position coordinates of the target parking space.

[0139] Step S2042, based on the target perception map and the position information of the target parking space, control the target vehicle to park in the target parking space.

[0140] In some embodiments, the target perception map can be continuously created according to the motion state of the target vehicle. During the movement of the target vehicle, the first in-vehicle perception information around the target vehicle and the first airborne perception information corresponding to the target area can be obtained in real time, and the target perception map can be determined based on the first in-vehicle perception information and the first airborne perception information obtained in real time.

[0141] In some embodiments, when the position information of the target parking space can be found in the target perception map, the movement path for the target vehicle to park in the target parking space can be determined by combining the target perception map and the position information of the target parking space. Based on this movement path, the target vehicle can be controlled to drive from the current position to the target parking space and park in the target parking space. In some embodiments, when the position information of the target parking space cannot be found in the target perception map either, the airborne perception information of the area between the target vehicle and the target parking space can be obtained through an auxiliary device, and a perception map of the area between the target parking space and the target vehicle can be created based on this airborne perception information. Based on this perception map and the target perception map, the movement path for the target vehicle to drive to the target parking space can be determined.

[0142] In the above embodiments, since the target perception map integrates in-vehicle perception information and airborne perception information and has high accuracy, when the position information of the target parking space in the current scene of the target vehicle is determined, controlling the target vehicle to park based on the target perception map and the position information of the target parking space can improve the accuracy and efficiency of the target vehicle parking in the target parking space.

[0143] In some embodiments, the determination of the position information of the target parking space available for the target vehicle to park in the current scene of the target vehicle described in step S2041 includes the following steps S20411 to S20413: Step S20411: Send a control instruction to the auxiliary device to control the auxiliary device to perform environmental perception on the current scene of the target vehicle.

[0144] In some embodiments, when it is determined that the target vehicle has a parking requirement, a control instruction can be sent to the auxiliary device. After receiving the control instruction, the auxiliary device can move to the current scene of the target vehicle to perform environmental perception and obtain the second airborne perception information.

[0145] Step S20412: Obtain the second airborne perception information fed back by the auxiliary device based on the control instruction.

[0146] In some embodiments, after obtaining the second on-vehicle sensing information, the auxiliary device may actively feedback the second on-vehicle sensing information to the target vehicle or the remote device; alternatively, the target vehicle or the remote device may send a request for obtaining on-vehicle sensing information to the auxiliary device after a preset duration of sending a control instruction to the auxiliary device, and after receiving the request for obtaining on-vehicle sensing information, the auxiliary device sends the second on-vehicle sensing information to the target vehicle or the remote device.

[0147] Step S20413: Based on the second on-vehicle sensing information and the second vehicle-mounted sensing information obtained by the sensors of the target vehicle through environmental sensing of the surroundings of the target vehicle, determine the position information of the target parking space.

[0148] In some embodiments, during the process of the auxiliary device performing environmental sensing on the current scene where the target vehicle is located, the target vehicle may remain at its current position unchanged, and the second vehicle-mounted sensing information collected by the target vehicle sensors also remains unchanged; in other embodiments, during the process of the auxiliary device performing environmental sensing on the current scene where the target vehicle is located, the target vehicle may also move following the auxiliary device and collect the second vehicle-mounted sensing information in real time during the movement.

[0149] In some embodiments, a third sensing map may be created based on the second on-vehicle sensing information, a fourth sensing map may be created based on the second vehicle-mounted sensing information, the third sensing map and the fourth sensing map may be fused to obtain a candidate sensing map, and the position information of the target parking space may be determined based on the candidate sensing map.

[0150] In the above embodiments, by sending a control instruction to the auxiliary device, the second on-vehicle sensing information obtained after the auxiliary device performs environmental sensing on the current scene where the target vehicle is located can be obtained. Further, based on the combination of the second on-vehicle sensing information and the second vehicle-mounted sensing information obtained by the sensors of the target vehicle through environmental sensing of the surroundings of the target vehicle, the position information of the target parking space is determined, which can improve the accuracy of the determined position of the target parking space.

[0151] In some embodiments, the above method may further include the following steps S211 to S212: Step S211: Based on the second on-vehicle sensing information, determine the movement path of the target vehicle.

[0152] In some embodiments, the auxiliary device may send the second on-vehicle sensing information it senses to the target vehicle or the remote device, and the target vehicle or the remote device may determine the movement path of the target vehicle based on the second on-vehicle sensing information. The movement path may be a driving path for the target vehicle to gradually approach directly below or obliquely below the auxiliary device.

[0153] In some other embodiments, the auxiliary device may send the second on-vehicle sensing information it perceives to the target vehicle or the remote device, and the target vehicle or the remote device determines the moving path of the target vehicle based on the second on-vehicle sensing information and the on-vehicle sensing information perceived by the target vehicle.

[0154] Step S212: Based on the moving path, control the target vehicle to move following the auxiliary device, and control the sensors of the target vehicle to perform environmental sensing around the target vehicle during the process of the target vehicle moving following the auxiliary device, so as to obtain the second on-vehicle sensing information.

[0155] In some embodiments, during the process of the target vehicle moving following the auxiliary device, it can perform environmental sensing around the target vehicle in real time, that is, the obtained second on-vehicle sensing information changes according to the moving position of the target vehicle.

[0156] In the above embodiments, the moving path of the target vehicle is determined according to the second on-vehicle sensing information perceived by the auxiliary device, and the target vehicle is controlled to move following the unmanned aerial vehicle based on this moving path, and during the process of controlling the sensors of the target vehicle to move following the auxiliary device, environmental sensing is performed around the target vehicle to obtain the second on-vehicle sensing information, so that the target parking space can be determined based on the fusion result of the second on-vehicle sensing information and the second on-vehicle sensing information subsequently, thereby improving the accuracy of the determined target parking space.

[0157] Next, the implementation process of the application embodiment in the actual application scenario will be introduced.

[0158] As Figure 3 shown, it is a schematic diagram of an automatic parking enhanced sensing system architecture based on vehicle-drone cooperation provided by an embodiment of the present application. An on-vehicle sensor system is newly added on the basis of the traditional on-vehicle sensor system. The new system architecture mainly includes on-vehicle sensing information 302 generated by the on-vehicle sensor system 301, and on-vehicle sensing information 304 generated by the on-vehicle sensor system 303. The enhanced environmental sensing information 305 is formed based on the fusion of the on-vehicle sensing information 302 and the on-vehicle sensing information 304, and based on the enhanced environmental sensing information 305, it guides the autonomous driving vehicle to perform path planning and vehicle status monitoring based on the decision-making and planning system 306, generate and issue control instructions based on the execution and control system 307, and perform control of driving, braking, and steering based on the vehicle drive system 308.

[0159] Among them, the vehicle-mounted sensor system 301 includes a vehicle-mounted ultrasonic radar 3011, a vehicle-mounted lidar 3012, a vehicle-mounted inertial measurement unit 3013, and a vehicle-mounted camera 3014. The vehicle-mounted perception information 302 includes a vehicle-mounted perception map 3023 created after performing two-dimensional target detection 3021 and three-dimensional target detection 3022 on the information collected by the vehicle-mounted sensor system; the airborne sensor system 303 includes an airborne ultrasonic radar 3031, an airborne lidar 3032, an airborne inertial measurement unit 3033, and an airborne camera 3034. The airborne perception information 304 includes an airborne perception map 3043 created after performing target monitoring and tracking 3041 and target re-identification 3042 on the information collected by the airborne sensor system; the enhanced environmental perception information 305 includes basic map data 3051, parking space information 3052, vehicle position 3053, and other position information 3054, etc. formed based on the fusion of the vehicle-mounted perception information 302 and the airborne perception information 304; the decision-making and planning system 306 includes a path planning module 3061 for performing path planning and a vehicle status monitoring module 3062 for monitoring the vehicle status; the execution and control system 307 includes a control instruction generation module 3071 for generating control instructions and an actuator control module 3072. The control instruction generation module 3071 sends the generated control instructions to the actuator control module 3072; the vehicle drive system 308 controls the drive module 3081, the brake module 3082, and the steering module 3083 of the vehicle to perform corresponding control actions based on the control instructions sent to the actuator control module 3072.

[0160] Figure 4A schematic diagram of an environmental perception enhancement technology architecture provided by an embodiment of this application. The perception enhancement is mainly achieved by fusing the vehicle-mounted perception basic global map and the airborne perception local map. The vehicle-mounted perception basic global map is obtained by respectively rectifying the front view original image 401 collected by the front view camera A, the right view original image 402 collected by the right view camera B, the left view original image 403 collected by the left view camera C, and the rear view original image 404 collected by the rear view camera D to obtain the front view rectified image 405, the right view rectified image 406, the left view rectified image 407, and the rear view rectified image 408. Then, after respectively performing coordinate transformation on the front view rectified image 405, the right view rectified image 406, the left view rectified image 407, and the rear view rectified image 408, four front view regions, namely the top view 409 of camera A (front view region), the top view 410 of camera B (right view region), the top view 411 of camera C (left view region), and the top view 412 of camera D (rear view region), and four common splicing regions, namely the left front common region view 413, the right front common region view 414, the left rear common region view 415, and the right rear common region view 416, are formed, totaling 8 vehicle-mounted perception base maps. The perception local map is obtained by the UAV performing image correction on the left front aircraft view 417 to obtain the left front rectified image 418, and then performing vehicle-mounted view coordinate transformation on the left front rectified image 418 to obtain the left front top view 419; performing image correction on the right front aircraft view 420 to obtain the right front rectified image 421, and then performing vehicle-mounted view coordinate transformation on the right front rectified image 421 to obtain the right front top view 422; performing image correction on the left rear aircraft view 423 to obtain the left rear rectified image 424, and then performing vehicle-mounted view coordinate transformation on the left rear rectified image 424 to obtain the left rear top view 425; performing image correction on the right rear aircraft view 426 to obtain the right rear rectified image 427, and then performing vehicle-mounted view coordinate transformation on the right rear rectified image 427 to obtain the right rear top view 428. The left front top view 419, the right front top view 422, the left rear top view 425, and the right rear top view 428 respectively correspond to the left front common region view 413, the right front common region view 414, the left rear common region view 415, and the right rear common region view 416. Based on the above fusion of the vehicle-mounted perception basic global map spectrum and the airborne perception local map spectrum, the perception enhancement of Figure 1 the low-precision area described in

[0161] Figure 5A schematic diagram of a vehicle - internet - of - things collaborative technology architecture provided by an embodiment of this application. This vehicle - internet - of - things collaborative technology architecture mainly elaborates on the collaborative mechanism, communication mechanism, and control mechanism between the vehicle - borne environment perception system and the airborne environment perception system, and finally forms enhanced vehicle - borne environment perception. During the automatic parking process of the autonomous vehicle 501, vehicle - borne environment perception is preferentially performed based on the vehicle - borne sensor system 502 to obtain vehicle - borne environment perception information 503. When the local judgment of the vehicle - borne environment perception information 503 is of low reliability, weak - vision perception feedback is performed and weak - vision perception points are fed back. The autonomous vehicle 501 starts the vehicle - borne drone through the vehicle - borne drone control system 504, and based on the vehicle - borne drone communication system 505, transmits the target control point information (weak - vision perception points) through vehicle - internet - of - things short - range collaborative communication technologies such as V2X, WiFi, Bluetooth, and UWB. The vehicle - borne drone flies to the weak - vision perception point and performs view acquisition on the weak - vision target point based on the airborne sensor system 506 to form local airborne environment perception information 507. The airborne environment perception information 507 includes a left - front top view 5071, a right - front top view 5072, a left - rear top view 5073, and a right - rear top view 5074. Similarly, the airborne environment perception information 507 is fed back to the autonomous vehicle 501 through the vehicle - borne drone communication system 505. The autonomous vehicle 501 performs fusion perception stitching 508 based on the vehicle - borne environment perception information 503 and the airborne environment perception information 507 to obtain enhanced environment perception information 509.

[0162] Next, the method of vehicle - machine collaborative perception provided by this application will be described in two scenarios: the scenario of automatically finding a parking space based on vehicle - machine collaboration and the scenario of automatically parking into a space based on enhanced perception. For other scenarios of autonomous vehicles involving automatic driving and fusion perception, this application is equally applicable.

[0163] This application provides a method for automatically finding a parking space based on vehicle - machine collaboration, and the method includes: Step 11: After the autonomous vehicle drives into a garage without a high - precision map, it can stop and wait in the safe area at the garage entrance. Step 12: The autonomous vehicle starts the vehicle - borne drone through the vehicle - borne drone control system and controls it to automatically cruise and search for available parking spaces in the garage. Step 13: The vehicle - borne drone leaves the autonomous vehicle and realizes SLAM path mapping and available parking space mapping based on its own ultrasonic radar, lidar, camera, etc. during the process of searching for available parking spaces. Step 14: The vehicle - borne drone transmits the SLAM perception data and available parking space data to the autonomous vehicle based on the vehicle - borne drone communication system. Step 15: The autonomous vehicle receives the perception information from the vehicle - borne drone end, performs position transformation and fusion processing, and forms vehicle - end - based perception information. Step 16: The autonomous vehicle conducts path planning and control based on the perceived information, activates the vehicle drive system, and autonomously drives to the target vacant parking space; Step 17: After the autonomous vehicle drives to the target vacant parking space, it can perform landing control or the next parking control based on the on-vehicle drone control system.

[0164] The method for automatically finding a parking space based on vehicle-drone collaboration provided in this application can realize finding a parking space based on an on-vehicle drone, improve the efficiency of finding a parking space while reducing the mapping and planning consumption of the vehicle itself, and has high flexibility. It solves the problem that traditional autonomous vehicles need to independently find a parking space and perform SLAM mapping with low mapping efficiency if there is no original high-precision map base in the garage before automatic parking.

[0165] This application provides an automatic parking control method based on enhanced perception, and this method includes: Step 21: After the autonomous vehicle drives to the target parking space, it perceives the environmental state of the parking space based on the on-vehicle environment perception system to obtain the on-vehicle perception basic global map; Step 22: If the perception result of the parking space environmental state in Step 21 is highly reliable and the parking difficulty is low, the autonomous vehicle only perceives according to the traditional method based on its own perception system and completes parking into the space; if the perception result of the parking space environmental state in Step 21 is less reliable and the parking difficulty is high, weak vision perception feedback is performed, and the weak vision perception target area is determined; Step 23: After the on-vehicle environment perception system performs weak vision perception feedback, the vehicle starts the on-vehicle drone through the on-vehicle drone control system; Step 24: The autonomous vehicle sends the position information of the weak vision perception target area to the on-vehicle drone based on the on-vehicle drone communication system; Step 25: The on-vehicle drone moves to the weak vision perception target area and completes on-board environment perception mapping based on the on-board sensor system to obtain the on-board perception local map; Step 26: The on-vehicle drone feeds back the on-board environment perception information (on-board perception local map) to the vehicle end through the on-vehicle drone communication system; Step 27: After the vehicle end of the autonomous vehicle receives the on-board environment perception information, it fuses the on-vehicle perception basic global map in Step 21 and the on-board perception local map in Step 26 to obtain enhanced environment perception information; Step 28: The autonomous vehicle performs dynamic parking based on the enhanced environment perception information fused in Step 27, conducts path planning control based on the enhanced environment perception information, and activates the vehicle drive system to automatically park into the target parking space; Step 29: After the autonomous vehicle drives to the target parking space, it performs landing control on it based on the on-vehicle drone control system.

[0166] The automatic parking control method based on enhanced perception provided by this application introduces physical third - perspective perception "command" based on vehicle - mounted UAV collaborative perception, realizes "blind - spot filling" for the construction of high - precision maps in SLAM for low - precision areas, improves the global perception accuracy, effectively improves the automatic parking accuracy, parking success rate and parking efficiency in various complex environments, and solves the problem that the traditional self - vehicle perception method for automatic parking is the physical first - perspective, with low perception accuracy and mapping efficiency in complex environments and distorted position areas.

[0167] In addition, this application is also applicable to non - fully automatic parking systems. During the manual parking process, the environmental perception of the vehicle - mounted UAV during parking can be directly sent to the vehicle cockpit in the form of a video stream, realizing 360° surround - view data and enhanced driver vision in complex parking space environments. At the same time, this application is also applicable to vehicle - surrounding environment shooting and self - vehicle following shooting in terms of scene functions; the airborne perception system described in this application can be used to make up for the fusion perception ability of vehicles without a vehicle - mounted perception system or with a low - configuration vehicle - mounted perception system. For example, the vehicle may not be equipped with surround - view cameras, and normal - environment perception can be achieved only based on the airborne perception system; the method described in this application is a general and universal method. In addition to realizing enhanced automatic parking perception in traditional passenger cars, it can also achieve enhanced environmental perception through technology replication and iteration in flying cars, intelligent logistics vehicles, autonomous mobile robots, and non - motor vehicles with similar automatic parking capabilities.

[0168] An embodiment of this application provides an environmental perception device, as Figure 6 shown. The environmental perception device 600 includes: A first acquisition module 601, configured to acquire first vehicle - mounted perception information collected by sensors of a target vehicle; A first control module 602, configured to control an auxiliary device to perform environmental perception on the target area to obtain first airborne perception information when it is determined that there is a target area around the target vehicle; the target vehicle - mounted perception information corresponding to the target area does not meet the reliability condition; the target vehicle - mounted perception information corresponding to the target area is determined based on the first vehicle - mounted perception information; A first determination module 603, configured to determine a target perception map based on the first vehicle - mounted perception information and the first airborne perception information.

[0169] In some embodiments, the first control module 602 includes: A first determination sub - module, configured to determine the position information of the target area; A first sending sub - module, configured to send the position information of the target area to the auxiliary device, so that the auxiliary device performs environmental perception based on the position information; The first receiving sub-module is configured to receive the first airborne perception information fed back by the auxiliary device based on the position information of the target area.

[0170] In some embodiments, the auxiliary device includes a flying device; the first sending sub-module is further configured to: send the position information to the flying device, so that the flying device flies above the target area based on the position information and performs environmental perception on the target area from an overhead perspective.

[0171] In some embodiments, the first determining module 603 includes: The first creating sub-module is configured to create a first perception map based on the first vehicle-mounted perception information; The second creating sub-module is configured to create a second perception map based on the first airborne perception information; The first fusion processing sub-module is configured to perform fusion processing on the first perception map and the second perception map based on the first weight corresponding to the first perception map and the second weight corresponding to the second perception map, to obtain the target perception map.

[0172] In some embodiments, the environmental perception device 600 further includes: The second obtaining module is configured to obtain the configuration information of the sensors of the target vehicle; The second determining module is configured to determine the first weight and / or the second weight based on the configuration information.

[0173] In some embodiments, the environmental perception device 600 further includes: The third determining module is configured to determine the reliability degree of the target vehicle-mounted perception information corresponding to the target area; The fourth determining module is configured to determine the first weight and / or the second weight based on the reliability degree.

[0174] In some embodiments, the environmental perception device 600 further includes: The fifth determining module is configured to determine the target vehicle-mounted perception information corresponding to at least one target candidate area around the target vehicle based on the first vehicle-mounted perception information; The sixth determining module is configured to determine the target candidate area as the target area when the target vehicle-mounted perception information corresponding to the target candidate area does not meet the reliability condition.

[0175] In some embodiments, the first vehicle-mounted perception information includes first images respectively corresponding to at least one first candidate area around the target vehicle, and the target vehicle-mounted perception information includes target images; the fifth determining module includes: A second determination sub-module, configured to determine target images corresponding to the at least one target candidate region based on first images respectively corresponding to the at least one first candidate region.

[0176] In some embodiments, the at least one target candidate region includes at least one second candidate region and / or at least one third candidate region. The second candidate region includes a region where at least two first candidate regions overlap, and the third candidate region includes a region in the first candidate regions that does not overlap with other first candidate regions.

[0177] In some embodiments, the target vehicle perception information corresponding to the target candidate region includes a target distance between the target vehicle and at least one obstacle within the target candidate region; the environment perception device 600 further includes: A seventh determination module, configured to determine that the target vehicle perception information corresponding to the target candidate region does not meet the reliability condition when the target distance corresponding to the target candidate region is less than a distance threshold.

[0178] In some embodiments, the environment perception device 600 further includes: An eighth determination module, configured to determine position information of a target parking space where the target vehicle can park in the scene where the target vehicle is currently located; A second control module, configured to control the target vehicle to park in the target parking space based on the target perception map and the position information of the target parking space.

[0179] In some embodiments, the eighth determination module includes: A second sending sub-module, configured to send a control instruction to the auxiliary device to control the auxiliary device to perform environment perception on the scene where the target vehicle is currently located; A first obtaining sub-module, configured to obtain second airborne perception information fed back by the auxiliary device based on the control instruction; A fourth determination sub-module, configured to determine the position information of the target parking space based on the second airborne perception information and second vehicle-mounted perception information obtained by the sensors of the target vehicle for environment perception around the target vehicle.

[0180] In some embodiments, the environment perception device 600 further includes: A ninth determination module, configured to determine a moving path of the target vehicle based on the second airborne perception information. A third control module, configured to control the target vehicle to move following the auxiliary device based on the moving path, and to control sensors of the target vehicle to perform environmental perception around the target vehicle during the process that the target vehicle moves following the auxiliary device, so as to obtain the second in-vehicle perception information.

[0181] An embodiment of the present application provides a vehicle, including sensors, a memory, and a processor, wherein, the sensors are configured to collect first in-vehicle perception information; the memory stores a computer program that can run on the processor, and when the processor executes the program, the steps in the above method are implemented.

[0182] An embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, some or all of the steps in the above method are implemented. The computer-readable storage medium may be transient or non-transient.

[0183] An embodiment of the present application provides a computer program product, including a computer program or instruction, and when the computer program or instruction is executed by a processor, the steps in the method in the above embodiment are implemented.

[0184] An embodiment of the present application provides a computer program product, the computer program product includes a non-transient computer-readable storage medium storing a computer program, and when the computer program is read and executed by a computer, some or all of the steps in the above method are implemented. The computer program product may be specifically implemented in a manner of hardware, software, or a combination thereof. In some embodiments, the computer program product is specifically embodied as a computer storage medium, and in other embodiments, the computer program product is specifically embodied as a software product, such as a Software Development Kit (SDK), etc.

[0185] It should be noted here that: the descriptions of the above embodiments tend to emphasize the differences between the embodiments, and their similarities or similarities can be referred to each other. The descriptions of the above device, equipment, vehicle, storage medium, and program product embodiments are similar to the descriptions of the above method embodiments and have beneficial effects similar to those of the method embodiments. For the technical details not disclosed in the device, equipment, vehicle, storage medium, and program product embodiments of the present application, please refer to the descriptions of the method embodiments of the present application for understanding.

[0186] It should be understood that the "one embodiment" or "an embodiment" mentioned throughout the specification means that the specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, the appearances of "in one embodiment" or "in an embodiment" throughout the specification do not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the magnitudes of the serial numbers of the above processes do not mean the order of execution, and the order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application. The serial numbers of the embodiments of the present application above are only for description and do not represent the advantages or disadvantages of the embodiments.

[0187] It should be noted that in this text, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, article or device including the element.

[0188] As described above, it is only to fully illustrate the implementation manners of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed by the present application should be covered within the protection scope of the present application.

Claims

1. A method for environmental perception, characterized in that: include: Acquire first vehicle-mounted perception information collected by a sensor of the target vehicle; In the case where there is a target area around the target vehicle, controlling the auxiliary device to perform environmental perception on the target area to obtain first onboard perception information; The target vehicle-mounted sensing information corresponding to the target area does not meet the reliability condition, and the target vehicle-mounted sensing information corresponding to the target area is determined based on the first vehicle-mounted sensing information; A target perception map is determined based on the first vehicle-mounted perception information and the first airborne perception information.

2. The environment perception method according to claim 1, characterized in that: The control auxiliary device performs environmental perception on the target area to obtain first airborne perception information, including: Determining location information of the target area; Sending the location information of the target area to the auxiliary device, so that the auxiliary device performs environmental perception based on the location information; The first airborne sensing information fed back by the auxiliary device based on the position information of the target area is received.

3. The environment perception method according to claim 2, characterized in that: The auxiliary equipment includes flight equipment; The sending the location information of the target area to the auxiliary device so that the auxiliary device performs environment perception based on the location information includes: The position information is sent to the flying device, so that the flying device flies above the target area based on the position information to perceive the environment of the target area from a bird's-eye view.

4. The environment perception method according to claim 1, characterized in that: The determining a target perception map based on the first vehicle-mounted perception information and the first airborne perception information includes: Creating a first perception map based on the first vehicle-mounted perception information; creating a second perception map based on the first onboard perception information; Based on a first weight corresponding to the first perception map and a second weight corresponding to the second perception map, the first perception map and the second perception map are fused to obtain the target perception map.

5. The environment perception method according to claim 4, characterized in that: Also includes: Acquire the configuration information of the sensor of the target vehicle; Based on the configuration information, the first weight and / or the second weight is determined.

6. The environment perception method according to claim 4, characterized in that: Also includes: Determining a reliability level of target vehicle-mounted perception information corresponding to the target area; Based on the reliability degree, the first weight and / or the second weight is determined.

7. The environment perception method according to claim 1, characterized in that: Also includes: Based on the first vehicle-mounted sensing information, determining target vehicle-mounted sensing information corresponding to at least one target candidate area around the target vehicle; When the target vehicle-mounted perception information corresponding to the target candidate area does not meet the reliability condition, the target candidate area is determined as the target area.

8. The environment perception method according to claim 7, characterized in that: The first vehicle-mounted perception information includes first images corresponding to at least one first candidate area around the target vehicle, and the target vehicle-mounted perception information includes a target image; The determining, based on the first vehicle-mounted perception information, target vehicle-mounted perception information corresponding to at least one target candidate area around the target vehicle includes: Based on the first image respectively corresponding to at least one of the first candidate regions, a target image respectively corresponding to the at least one target candidate region is determined.

9. The environment perception method according to claim 8, characterized in that: The at least one target candidate region includes at least one second candidate region and / or at least one third candidate region, wherein the second candidate region includes an area where at least two first candidate regions overlap, and the third candidate region includes an area in a first candidate region that does not overlap with other first candidate regions.

10. The environment perception method according to claim 7, characterized in that: The target vehicle sensing information corresponding to the target candidate area includes a target distance between the target vehicle and at least one obstacle in the target candidate area; The environment perception method further includes: When the target distance corresponding to the target candidate area is less than the distance threshold, it is determined that the target vehicle-mounted perception information corresponding to the target candidate area does not meet the reliability condition.

11. The environment perception method according to any one of claims 1 to 10, characterized in that: Also includes: Determine the location information of a target parking space available for parking the target vehicle in the scene where the target vehicle is currently located; Based on the target perception map and the position information of the target parking space, the target vehicle is controlled to park in the target parking space.

12. The environment perception method according to claim 11, characterized in that: The determining the position information of the target parking space available for parking the target vehicle in the scene where the target vehicle is currently located includes: Sending a control instruction to the auxiliary device to control the auxiliary device to perform environmental perception of the scene where the target vehicle is currently located; Acquire second onboard sensing information fed back by the auxiliary device based on the control instruction; The position information of the target parking space is determined based on the second onboard perception information and the second onboard perception information obtained by the sensor of the target vehicle sensing the environment around the target vehicle.

13. The environment perception method according to claim 12, characterized in that: Also includes: Determining a moving path of the target vehicle based on the second airborne sensing information; Based on the moving path, the target vehicle is controlled to move following the auxiliary device, and the sensor of the target vehicle is controlled to sense the environment around the target vehicle during the process of the target vehicle following the auxiliary device to obtain the second vehicle-mounted sensing information.

14. An environment sensing device, characterized in that: include: A first acquisition module, used to acquire first vehicle-mounted perception information collected by a sensor of a target vehicle; A first control module is used to control the auxiliary device to perform environmental perception on the target area and obtain first airborne perception information when it is determined that there is a target area around the target vehicle; The target vehicle-mounted sensing information corresponding to the target area does not meet the reliability condition; The target vehicle-mounted sensing information corresponding to the target area is determined based on the first vehicle-mounted sensing information; The first determination module is used to determine a target perception map based on the first vehicle-mounted perception information and the first airborne perception information.

15. A vehicle comprising a sensor, a memory and a processor, wherein: The sensor is used to collect first vehicle-mounted sensing information; The memory stores a computer program that can be run on a processor, wherein the processor implements the steps of the method according to any one of claims 1 to 13 when executing the program.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 13 are implemented.

17. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the steps in the method according to any one of claims 1 to 13 are implemented.

Images