Vehicle behavior detection method and device, electronic equipment and computer readable storage medium
Through multimodal information fusion, including point cloud and image data, and combining environmental context data to divide road areas, detect vehicle parking behavior, the accuracy and robustness of vehicle illegal parking detection in complex traffic scenarios in the prior art are solved. The problem of insufficient improvements in the accuracy and adaptability of detection.
Patent Information
- Application Number
- CN202411172342.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-26
- Publication Date
- 2024-12-03
AI Technical Summary
Existing vehicle illegal parking detection technology is insufficient in accuracy and robustness in complex traffic scenarios, especially in non-standard roads or lack of clear road markings, which makes it difficult to effectively integrate multimodal data, resulting in the reliability of vehicle stationary detection and Low accuracy affects the reliability of the autonomous driving system.
Using multimodal information, including point cloud data, image data and environmental context data, divide the road area through environmental context data, obtain road sub-region information, and combine point cloud data and image data to detect the location and parking behavior of the target vehicle to determine The vehicle's road surface sub-region is used to determine whether its parking behavior complies with traffic regulations.
It improves the accuracy and reliability of vehicle parking behavior detection, adapts to complex traffic environments and non-standard roads, reduces dependence on road markings, and enhances the applicability and real-timeness of intelligent traffic systems.
Smart Images

Figure CN119068435A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of intelligent transportation technology, and in particular to a vehicle behavior detection method, device, electronic device and computer-readable storage medium. Background Art
[0002] With the continuous development and application of intelligent technology, Intelligent Traffic System (ITS) has become one of the important means to solve urban traffic problems. Intelligent Traffic System uses advanced information and communication technology to monitor, manage and control vehicles, roads and personnel in the traffic system to improve traffic efficiency, reduce traffic congestion and improve traffic safety.
[0003] Among them, the detection of illegal parking of vehicles is one of the key monitoring items of intelligent transportation systems for roadside vehicles. The existing technology usually uses single sensor data, such as using laser radar or camera to identify road markings (such as road markings such as parking signs) and vehicle driving status, and then determines whether the vehicle is illegally parked. The application of such detection solutions in complex traffic scenarios has at least the following challenges:
[0004] 1) Limited by the limited perception range of a single sensor, it is unable to fully understand the complexity of the entire scene, and thus cannot accurately identify whether the vehicle is involved in illegal parking;
[0005] 2) In the case of non-standard roads or lack of clear road markings, the accuracy and robustness of illegal parking detection are insufficient, and may have a negative impact on the reliability of the autonomous driving system;
[0006] 3) The illegal parking detection method that relies on a single sensor is difficult to effectively integrate multimodal data, and the reliability and accuracy of the vehicle stationary detection results are low. Summary of the invention
[0007] Embodiments of the present application provide a vehicle behavior detection method, device, electronic device, and computer-readable storage medium to improve the accuracy and reliability of parking behavior detection.
[0008] The present application embodiment adopts the following technical solutions:
[0009] In a first aspect, an embodiment of the present application provides a vehicle behavior detection method, the method comprising:
[0010] Acquire multimodal information corresponding to the road area, wherein the multimodal information includes point cloud data, image data, and environmental context data;
[0011] Acquire road surface area division information of the road area according to the environmental context data, wherein the road surface area division information includes different types of road surface sub-areas;
[0012] Acquire target vehicle information according to the point cloud data and the image data, wherein the target vehicle information includes vehicle position information;
[0013] Determining a road surface sub-area of the target vehicle according to the vehicle position information of the target vehicle and the road surface area division information;
[0014] Parking behavior specification information of the target vehicle is obtained according to the road surface sub-area of the target vehicle.
[0015] In a second aspect, an embodiment of the present application further provides a vehicle behavior detection device, the device comprising:
[0016] A data acquisition unit, used to acquire multimodal information corresponding to the road area, wherein the multimodal information includes point cloud data, image data and environmental context data;
[0017] A road surface division unit, configured to obtain road surface area division information of the road area according to the environmental context data, wherein the road surface area division information includes different types of road surface sub-areas;
[0018] A vehicle detection unit, configured to obtain target vehicle information according to the point cloud data and the image data, wherein the target vehicle information includes vehicle position information;
[0019] A vehicle position determination unit, configured to determine a road surface sub-area of the target vehicle according to the vehicle position information of the target vehicle and the road surface area division information;
[0020] The parking behavior detection unit is used to obtain parking behavior specification information of the target vehicle according to the road surface sub-area of the target vehicle.
[0021] In a third aspect, an embodiment of the present application further provides an electronic device, including:
[0022] A processor; and a memory arranged to store computer executable instructions, which when executed cause the processor to perform a vehicle behavior detection method.
[0023] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, which stores one or more programs, and when the one or more programs are executed by an electronic device including multiple application programs, the electronic device performs a vehicle behavior detection method.
[0024] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects:
[0025] The embodiment of the present application utilizes multimodal information including point cloud data, image data and environmental context data to detect the parking behavior of the vehicle. On the one hand, the road surface area is divided by the environmental context data to obtain different types of road surface sub-areas. The road surface sub-areas provide a judgment basis for parking behavior detection, and the degree of refinement of the road surface sub-area division can control the refinement and accuracy of parking behavior detection. On the other hand, the target vehicle that is currently in a parking state is detected by point cloud data and image data to improve the detection accuracy of the target vehicle. The road surface sub-areas of the target vehicle are used to determine the parking behavior specification information of the target vehicle, and then whether the parking behavior of the vehicle complies with traffic regulations is judged based on the parking behavior specification information. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0027] Figure 1 A flow chart of a vehicle behavior detection method in an embodiment of the present application;
[0028] Figure 2 This is a schematic diagram of a point cloud data recognition result in an embodiment of the present application;
[0029] Figure 3 A schematic diagram of a road environment in an embodiment of the present application;
[0030] Figure 4 This is a structural schematic diagram of a vehicle behavior detection device in an embodiment of the present application;
[0031] Figure 5 This is a schematic diagram of the structure of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION
[0032] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.
[0033] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.
[0034] The embodiments of the present application provide a vehicle behavior detection method. The following embodiments of the present application use roadside equipment as the execution subject of the vehicle behavior detection method for example. It can be understood that the execution subject of the embodiments of the present application may also be other devices or systems, such as a roadside unit, a roadside edge computing system or a chip system, and the chip system includes at least one processor, and a communication interface or a memory, wherein the memory is used to store programs or instructions, and the communication interface is used to provide the processor with information or data input and output functions. The execution subject of the embodiments of the present application may also be that the roadside sends the acquired information to the cloud and executes it by the cloud. Alternatively, the execution subject of the vehicle behavior detection method of the embodiments of the present application may also be software or hardware, and different execution steps may be centralized or distributed.
[0035] like Figure 1 As shown, a flow chart of a vehicle behavior detection method in an embodiment of the present application is provided, and the vehicle behavior detection method at least includes the following steps S110 to S150:
[0036] Step S110, obtaining multimodal information corresponding to the road area, wherein the multimodal information includes point cloud data, image data and environmental context data.
[0037] When the roadside equipment performs vehicle behavior detection, it first obtains multimodal information of the road area through the roadside sensor. The multimodal information here includes current point cloud data and image data, as well as historical point cloud data and historical image data over a period of time as the environmental context data of this embodiment. The road area can be understood as the road section area within the monitoring range of the roadside sensor.
[0038] Step S120: acquiring road surface area division information of the road area according to the environmental context data, wherein the road surface area division information includes different types of road surface sub-areas.
[0039] The environmental context data of this embodiment can be understood as historical point cloud data and historical image data within a period of time. According to relevant data processing algorithms, such as semantic segmentation, visual recognition, neural network and other algorithms, a road environment model of the road area can be constructed. Figure 2 As shown, the road environment model can clearly define the road surface, lane lines, stop lines, drivable areas, traffic signs, traffic lights, green belts, buildings, trees, etc. within the road area.
[0040] The historical driving information corresponding to the road area can also be obtained through the environmental context data. The historical driving information can determine the historical parking area on the road. The historical parking area in this embodiment refers to the roadside area where parking is allowed based on the historical driving information. The application will explain in detail the determination of the historical parking area based on the historical driving information in the following embodiments.
[0041] In this way, combining the historical parking area and the road environment model and according to whether parking is allowed, the road surface area can be roughly divided into three types: the first type is the specified parking area at the parking sign, the second type is the suspected parking area without a parking sign but parking is allowed, and the third type is the non-parking area. Of course, in actual applications, more detailed area divisions can be performed, and those skilled in the art can flexibly divide the road surface area according to the traffic environment of the road area.
[0042] Step S130, acquiring target vehicle information according to the point cloud data and the image data, wherein the target vehicle information includes vehicle position information.
[0043] This embodiment performs target detection on point cloud data and image data respectively, obtains point cloud detection results and image detection results, and performs target fusion on the point cloud detection results and image detection results according to the joint calibration relationship between the camera and the lidar, so as to obtain vehicle detection information with higher accuracy. The vehicle detection information includes but is not limited to vehicle position information, vehicle size information, vehicle speed information, vehicle heading angle information, vehicle size information, vehicle type information, etc.
[0044] The parking state of the vehicle is detected according to the vehicle detection information, and the target vehicle information currently in the parking state can be obtained according to the parking state detection result.
[0045] It should be noted that the embodiment of the present application does not limit the execution order of step S120 and step S130. In other embodiments, step S120 and step S130 may be executed simultaneously, or step S130 may be executed first and then step S120.
[0046] Step S140, determining the road surface sub-area of the target vehicle according to the vehicle position information of the target vehicle and the road surface area division information.
[0047] This embodiment uses a point-in-polygon algorithm to determine the road sub-area of the target vehicle, and the point-in-polygon algorithm includes, for example, a ray method, an area method, etc. Of course, those skilled in the art can also determine the road sub-area of the target vehicle through other existing schemes, and this embodiment does not impose any particular limitation on this.
[0048] Step S150, obtaining parking behavior specification information of the target vehicle according to the road surface sub-area of the target vehicle.
[0049] The parking behavior specification information of this embodiment is used to indicate whether the vehicle is parked normally in the specified roadside area. Normal parking can be understood as the vehicle being completely parked in the specified roadside area. When the vehicle is parked in a suspected parking area or in a non-parking area, the parking behavior specification information also includes alarm information, and the roadside equipment sends the alarm information to the alarm management platform or related application platform of the intelligent transportation system.
[0050] like Figure 1 It can be seen from the roadside parking behavior detection method shown that the embodiment of the present application uses multimodal information including point cloud data, image data and environmental context data to detect the parking behavior of the vehicle. On the one hand, the road surface area is divided by the environmental context data to obtain road surface sub-areas of different parking types. The road surface sub-areas provide a judgment basis for parking behavior detection, and the degree of refinement of the road surface sub-area division can control the refinement and accuracy of parking behavior detection; on the other hand, the target vehicle that is currently in a parking state is detected through point cloud data and image data to improve the detection accuracy of the target vehicle, and the road surface sub-area of the target vehicle is used to determine the parking behavior specification information of the target vehicle, and then judge whether the parking behavior of the vehicle complies with traffic regulations based on the parking behavior specification information.
[0051] In some embodiments of the present application, obtaining the road surface area division information of the road area according to the environmental context data in the above step S120 specifically includes:
[0052] Acquire environmental semantic information and historical driving information of the road area through the environmental context data, wherein the environmental semantic information includes traffic sign information and road surface area information;
[0053] The road surface area information is divided into areas according to the traffic sign information, the road surface area information and the historical driving information to obtain the road surface sub-area information.
[0054] As mentioned above, the environmental context data of this embodiment includes historical point cloud data and historical image data for a period of time (e.g., a week, a month, or longer). By processing the historical point cloud data through a semantic segmentation algorithm, environmental semantic information can be obtained, such as Figure 2As shown, the environmental semantic information at least includes traffic sign information and road surface area information. Traffic sign information includes, for example, roadside signs such as traffic lights, parking signs (white P signs with a blue background), and speed limit signs; road surface area information includes, for example, drivable areas. Traffic sign information and road surface area information can also be obtained by performing target detection on image data using a visual recognition algorithm or a neural network algorithm. Unlike point cloud detection results, image detection results can include more detailed road surface area information, such as drivable areas and non-drivable areas (sidewalks, non-motorized vehicle lanes, etc.). When the road surface marking lines are clear, various road surface marking lines such as lane lines and parking lines can also be detected.
[0055] According to the detection results of the historical data of the above two modes, a road environment model with high accuracy can be constructed, such as Figure 3 As shown, the road environment model can reflect the actual road status of the road area.
[0056] In addition, historical driving information can be obtained by performing vehicle detection on historical point cloud data and historical image data. The historical driving information of this embodiment may include full information on vehicles in the road area, and the full information includes basic vehicle information and reprocessed vehicle information. The basic vehicle information refers to the parking time, parking position, etc. obtained by performing vehicle stationary detection on historical driving information; the reprocessed vehicle information refers to the parking frequency, number of parking, range of parking positions, etc. corresponding to the parking positions obtained by performing statistical calculations on the basic vehicle information.
[0057] In some possible implementation schemes of this embodiment, dividing the road surface area information into regions according to the traffic sign information, the road surface area information and the historical driving information to obtain the road surface sub-area information specifically includes:
[0058] Acquire target traffic sign information having parking semantic features according to the traffic sign information, and determine a historical parking area according to the road surface area information and the historical driving information;
[0059] Dividing the road surface area information into regions according to the historical parking area and the target traffic sign information;
[0060] The road surface sub-area range and the type of the road surface sub-area are obtained according to the area division result.
[0061] The traffic sign information in this embodiment may include roadside sign information, and the roadside sign information includes parking signs. When the road area is a standardized road area or a road area with clear lane markings, the traffic sign information also includes road surface marking information, and the road surface marking information includes various road surface marking lines such as lane lines and parking lines.
[0062] The standardized road area refers to standardized roads that comply with relevant laws and regulations on road traffic and have specific markings, signs and other information. In standard road scenarios, the autonomous driving system can obtain accurate road information through standardized road markings, signs and other information.
[0063] In this embodiment, determining the historical parking area according to the road area information and the historical driving information specifically includes:
[0064] Acquire candidate historical parking areas where vehicles have parked according to the historical driving information;
[0065] Acquire preset parking feature information of the candidate historical parking area according to the historical driving information, wherein the preset parking feature information includes at least one of parking time, parking quantity, and parking frequency;
[0066] The historical parking area is determined according to preset parking feature information of the candidate historical parking area.
[0067] In practical applications, all parking locations can be clustered by clustering algorithms, and candidate historical parking areas can be obtained based on the clustering results. Generally, multiple candidate historical parking areas will be obtained. The parking locations here refer to locations where parking has been done for a certain period of time. For example, parking locations where parking has been done for more than 3 minutes are clustered to distinguish temporary parking under traffic light control from temporary parking on the roadside. Then, for each candidate historical parking area, the parking time, number of parking, and parking frequency corresponding to the candidate historical parking area are used to determine whether the candidate historical parking area is a historical parking area.
[0068] Specifically, it is determined whether the candidate historical parking area has the preset parking feature information. If the candidate historical parking area has the preset parking feature information, the candidate historical parking area is determined to be a historical parking area, and the range information of the historical parking area is obtained according to the preset area detection algorithm. The preset area detection algorithm here can be to obtain the length of the historical parking area based on the longitudinal distance between the vehicles at both ends of the area, and to obtain the width of the historical parking area based on the width of the vehicle; of course, if the road marking lines are clear, the range of each historical parking area can also be calculated based on the road marking lines.
[0069] In this implementation, the road surface area information is divided into regions according to the historical parking area and the target traffic sign information, specifically including:
[0070] Obtaining distance indication information corresponding to the target traffic sign information;
[0071] Determine the type of each historical parking area according to the distance indication information corresponding to the target traffic sign information, wherein the types of the historical parking areas include specified parking areas and suspected parking areas;
[0072] The non-historical parking area in the road surface area information is determined as a no-parking area.
[0073] In actual application scenarios, some parking signs (such as a blue background with white letters P sign) indicate that parking is allowed near the current location, and some parking signs indicate how far away from the current location there is a parking area. In this way, this embodiment can classify historical parking areas according to the distance indication information corresponding to the target traffic sign information, for example, determine the historical parking area near the blue background with white letters P sign as a specified parking area, and determine other historical parking areas that do not correspond to any parking signs as suspected parking areas.
[0074] In some embodiments of the present application, the current point cloud data includes a first frame point cloud and a second frame point cloud obtained by scanning the vehicle by a lidar, and a first frame image and a second frame image obtained by scanning the vehicle by a camera and aligned in time and space with the two frames of point clouds.
[0075] Correspondingly, in the above step S130, obtaining the target vehicle information according to the point cloud data and the image data specifically includes:
[0076] Perform vehicle detection on the first frame point cloud and the second frame point cloud respectively, track and match the detection frame of the first frame point cloud with the target frame of the second frame point cloud, and obtain the center position difference and target frame speed of the target frame;
[0077] Performing vehicle detection on the first frame image and the second frame image respectively, tracking and matching the detection frame of the first frame image with the target frame of the second frame image, and obtaining the center position difference of the target frame and the target frame speed;
[0078] According to the joint calibration relationship between the LiDAR and the camera, the target frame is fused to obtain the center position and speed of the fused target frame;
[0079] The vehicle stationary detection is performed according to the center position and speed of the target frame, and whether the vehicle corresponding to the target frame is a target vehicle in a parked state is determined according to the vehicle stationary detection result.
[0080] It should be noted that the present embodiment shows a solution for target detection based on a target tracking algorithm. In other embodiments of the present application, vehicle detection may also be performed using other target detection algorithms.
[0081] In some possible implementation schemes of this embodiment, vehicle stationary detection is performed according to the center position and speed of the target frame, specifically including:
[0082] If the speed and position changes of the target frame within the set time meet the stationary detection conditions, the vehicle corresponding to the target frame is determined to be a target vehicle in a parked state;
[0083] Otherwise, it is determined that the vehicle corresponding to the target frame is a non-target vehicle in a driving state.
[0084] The stationary detection condition in this embodiment specifies a speed threshold and a position change threshold, both of which are values close to 0. When the speed of the target frame is zero and the position change is 0 within a set time (e.g., 5 seconds), it can be determined that the vehicle corresponding to the target frame is a target vehicle in a parked state. The stationary detection condition improves the detection accuracy of the target vehicle and avoids misjudging a slowly moving vehicle as a target vehicle in a parked state.
[0085] In some embodiments of the present application, the above step S150 acquires the parking behavior specification information of the target vehicle according to the road surface sub-area of the target vehicle, specifically including:
[0086] When the road sub-area of the target vehicle is a non-parking area, generating illegal parking warning information;
[0087] When the road sub-area of the target vehicle is a prescribed parking area, generating normal parking information;
[0088] When the road sub-area of the target vehicle is a suspected parking area, a secondary detection alarm message is generated. The secondary detection can be based on historical records to determine whether the vehicle in the suspected parking area has received a parking violation notice.
[0089] The roadside parking behavior detection method in the above embodiment of the present application has at least the following advantages:
[0090] First, the integration of multimodal information can overcome the traditional method's reliance on a single sensor and achieve more comprehensive and accurate vehicle stationary detection through lidar, cameras and environmental context information.
[0091] Second, adapt to complex traffic environments and improve the roadside equipment's understanding of complex urban traffic environments. Through environmental modeling and context analysis, the accuracy of vehicle stationary detection can be further improved.
[0092] Third, it adapts to non-standard roads and does not rely on road marking information such as lane lines to complete vehicle behavior detection, which enhances the adaptability of roadside equipment to non-standard roads and reduces dependence on road markings.
[0093] Fourth, it provides a more accurate and reliable solution for roadside parking behavior detection in intelligent transportation systems, especially for severe weather and low-light conditions, making roadside equipment more applicable and real-time.
[0094] The present application also provides a vehicle behavior detection device 400, such as Figure 4 As shown, a schematic diagram of the structure of a vehicle behavior detection device in an embodiment of the present application is provided, wherein the device 400 includes: a data acquisition unit 410, a road surface division unit 420, a vehicle detection unit 430, a vehicle position determination unit 440 and a parking behavior detection unit 450, wherein:
[0095] A data acquisition unit 410 is used to acquire multimodal information corresponding to the road area, wherein the multimodal information includes point cloud data, image data, and environmental context data;
[0096] A road surface division unit 420, configured to obtain road surface area division information of the road area according to the environmental context data, wherein the road surface area division information includes different types of road surface sub-areas;
[0097] A vehicle detection unit 430, configured to obtain target vehicle information according to the point cloud data and the image data, wherein the target vehicle information includes vehicle position information;
[0098] A vehicle position determination unit 440, configured to determine a road surface sub-area of the target vehicle according to the vehicle position information of the target vehicle and the road surface area division information;
[0099] The parking behavior detection unit 450 is used to obtain parking behavior specification information of the target vehicle according to the road surface sub-area of the target vehicle.
[0100] In some embodiments of the present application, the road surface division unit 420 includes a processing module and a division module;
[0101] A processing module, used for acquiring environmental semantic information and historical driving information of the road area through the environmental context data, wherein the environmental semantic information includes traffic sign information and road surface area information;
[0102] The division module is used to divide the road surface area information into regions according to the traffic sign information, the road surface area information and the historical driving information, and obtain the road surface sub-region information.
[0103] In some embodiments of the present application, the division module is specifically used to obtain target traffic sign information with parking semantic features based on the traffic sign information, and to determine the historical parking area based on the road surface area information and the historical driving information; to divide the road surface area information into regions based on the historical parking area and the target traffic sign information; and to obtain the road surface sub-area range and the type of the road surface sub-area based on the region division result.
[0104] In some embodiments of the present application, the division module is specifically used to obtain candidate historical parking areas where vehicles are parked based on the historical driving information; obtain preset parking feature information of the candidate historical parking areas based on the historical driving information; and determine the historical parking area based on the preset parking feature information of the candidate historical parking areas.
[0105] In some embodiments of the present application, the preset parking feature information includes at least one of parking time, parking quantity, and parking frequency. The division module is also used to determine whether the candidate historical parking area has the preset parking feature information; if the candidate historical parking area has the preset parking feature information, the candidate historical parking area is determined to be a historical parking area; and the range information of the historical parking area is obtained according to a preset area detection algorithm.
[0106] In some embodiments of the present application, the division module is specifically used to obtain distance indication information corresponding to the target traffic sign information; determine the type of each historical parking area according to the distance indication information corresponding to the target traffic sign information, the type of historical parking area including specified parking areas and suspected parking areas; and determine the non-historical parking areas in the road surface area information as non-parking areas.
[0107] In some embodiments of the present application, the road surface sub-area includes a no-parking area, a prescribed parking area, and a suspected parking area. The parking behavior detection unit 450 is specifically used to generate illegal parking alarm information when the road surface sub-area of the target vehicle is a no-parking area; generate normal parking information when the road surface sub-area of the target vehicle is a prescribed parking area; and generate secondary detection alarm information when the road surface sub-area of the target vehicle is a suspected parking area.
[0108] It can be understood that the above-mentioned vehicle behavior detection device can implement each step of the vehicle behavior detection method provided in the aforementioned embodiment, and the relevant explanations about the vehicle behavior detection method are applicable to the vehicle behavior detection device and will not be repeated here.
[0109] Figure 5 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Figure 5At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. The memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. Of course, the electronic device may also include hardware required for other services.
[0110] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0111] The memory is used to store the program. Specifically, the program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides instructions and data to the processor.
[0112] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming a vehicle behavior detection device at the logical level. The processor executes the program stored in the memory and is specifically used to perform the following operations:
[0113] Acquire multimodal information corresponding to the road area, wherein the multimodal information includes point cloud data, image data, and environmental context data;
[0114] Acquire road surface area division information of the road area according to the environmental context data, wherein the road surface area division information includes different types of road surface sub-areas;
[0115] Acquire target vehicle information according to the point cloud data and the image data, wherein the target vehicle information includes vehicle position information;
[0116] Determining a road surface sub-area of the target vehicle according to the vehicle position information of the target vehicle and the road surface area division information;
[0117] Parking behavior specification information of the target vehicle is obtained according to the road surface sub-area of the target vehicle.
[0118] The above application Figure 1 The method performed by the vehicle behavior detection device disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or instructions in the form of software. The above processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the vehicle behavior detection method in combination with its hardware.
[0119] The electronic device may also perform Figure 1 The method executed by the vehicle behavior detection device in Figure 1 The functions of the illustrated embodiment will not be described in detail in the embodiments of the present application.
[0120] The present application also provides a computer-readable storage medium, which stores one or more programs, wherein the one or more programs include instructions, which, when executed by an electronic device including multiple application programs, enable the electronic device to execute Figure 1 The method performed by the vehicle behavior detection device in the illustrated embodiment is specifically used to perform:
[0121] Acquire multimodal information corresponding to the road area, wherein the multimodal information includes point cloud data, image data, and environmental context data;
[0122] Acquire road surface area division information of the road area according to the environmental context data, wherein the road surface area division information includes different types of road surface sub-areas;
[0123] Acquire target vehicle information according to the point cloud data and the image data, wherein the target vehicle information includes vehicle position information;
[0124] Determining a road surface sub-area of the target vehicle according to the vehicle position information of the target vehicle and the road surface area division information;
[0125] Parking behavior specification information of the target vehicle is obtained according to the road surface sub-area of the target vehicle.
[0126] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0127] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0128] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0129] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0130] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0131] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.
[0132] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0133] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
[0134] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.
[0135] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims
1. A vehicle behavior detection method, characterized in that: The method comprises: Acquire multimodal information corresponding to the road area, wherein the multimodal information includes point cloud data, image data, and environmental context data; Acquire road surface area division information of the road area according to the environmental context data, wherein the road surface area division information includes different types of road surface sub-areas; Acquire target vehicle information according to the point cloud data and the image data, wherein the target vehicle information includes vehicle position information; Determining a road surface sub-area of the target vehicle according to the vehicle position information of the target vehicle and the road surface area division information; Parking behavior specification information of the target vehicle is obtained according to the road surface sub-area of the target vehicle.
2. The vehicle behavior detection method according to claim 1, characterized in that: The obtaining the road surface area division information of the road area according to the environmental context data includes: Acquire environmental semantic information and historical driving information of the road area through the environmental context data, wherein the environmental semantic information includes traffic sign information and road surface area information; The road surface area information is divided into areas according to the traffic sign information, the road surface area information and the historical driving information to obtain the road surface sub-area information.
3. The vehicle behavior detection method according to claim 2, characterized in that: The step of dividing the road surface area information into regions according to the traffic sign information, the road surface area information and the historical driving information to obtain the road surface sub-region information includes: Acquire target traffic sign information having parking semantic features according to the traffic sign information, and determine a historical parking area according to the road surface area information and the historical driving information; Dividing the road surface area information into regions according to the historical parking area and the target traffic sign information; The road surface sub-area range and the type of the road surface sub-area are obtained according to the area division result.
4. The vehicle behavior detection method according to claim 3, characterized in that: The determining of the historical parking area according to the road area information and the historical driving information includes: Acquire candidate historical parking areas where vehicles have parked according to the historical driving information; Acquire preset parking feature information of a candidate historical parking area according to the historical driving information; The historical parking area is determined according to preset parking feature information of the candidate historical parking area.
5. The vehicle behavior detection method according to claim 4, characterized in that: The preset parking characteristic information includes at least one of parking time, parking quantity, and parking frequency. The determining the historical parking area according to the preset parking characteristic information of the candidate historical parking area includes: determining whether the candidate historical parking area has the preset parking feature information; If the candidate historical parking area has the preset parking feature information, determining that the candidate historical parking area is a historical parking area; The range information of the historical parking area is obtained according to a preset area detection algorithm.
6. The vehicle behavior detection method according to claim 3, characterized in that: The dividing the road surface area information into regions according to the historical parking area and the target traffic sign information includes: Obtaining distance indication information corresponding to the target traffic sign information; Determine the type of each historical parking area according to the distance indication information corresponding to the target traffic sign information, wherein the types of the historical parking areas include specified parking areas and suspected parking areas; The non-historical parking area in the road surface area information is determined as a no-parking area.
7. The vehicle behavior detection method according to claim 1, characterized in that: The road surface sub-area includes a non-parking area, a specified parking area, and a suspected parking area. The acquiring the parking behavior specification information of the target vehicle according to the road surface sub-area of the target vehicle includes: When the road sub-area of the target vehicle is a non-parking area, generating illegal parking warning information; When the road sub-area of the target vehicle is a prescribed parking area, generating normal parking information; When the road sub-area of the target vehicle is a suspected parking area, a secondary detection warning message is generated.
8. A vehicle behavior detection device, characterized in that: The device comprises: A data acquisition unit, used to acquire multimodal information corresponding to the road area, wherein the multimodal information includes point cloud data, image data and environmental context data; A road surface division unit, configured to obtain road surface area division information of the road area according to the environmental context data, wherein the road surface area division information includes different types of road surface sub-areas; A vehicle detection unit, configured to obtain target vehicle information according to the point cloud data and the image data, wherein the target vehicle information includes vehicle position information; A vehicle position determination unit, configured to determine a road surface sub-area of the target vehicle according to the parking position information of the target vehicle and the road surface area division information; The parking behavior detection unit is used to obtain parking behavior specification information of the target vehicle according to the road surface sub-area of the target vehicle.
9. An electronic device, comprising: processor; as well as A memory arranged to store computer executable instructions, wherein when the executable instructions are executed, the processor executes the vehicle behavior detection method according to any one of claims 1 to 7.
10. A computer-readable storage medium storing one or more programs, which, when executed by an electronic device including a plurality of application programs, enables the electronic device to execute the vehicle behavior detection method according to any one of claims 1 to 7.
