Violation of detection methods, devices and vehicles

Abstract

An intrusion detection method, apparatus, and vehicle are disclosed. The method includes: acquiring image information from a first camera device (S401); acquiring, based on the image information, the area where a target object is located near the vehicle and the vehicle's negative distance intrusion zone (S402), wherein the negative distance intrusion zone is related to the vehicle's projection area on the ground; determining whether the target object has entered the negative distance intrusion zone based on the relative position of the area where the target object is located and the negative distance intrusion zone (S403); and issuing a warning when the target object enters the negative distance intrusion zone (S404). This method can detect whether a target object has entered an area related to the vehicle's projection area on the ground, improving the accuracy of vehicle intrusion detection.

Description

Technical Field

[0001] This application relates to the field of intelligent vehicles, and more specifically, to an infringement detection method, apparatus, and vehicle. Background Technology

[0002] Vehicle perimeter intrusion detection is a common safety warning scenario in the automotive industry, frequently used in applications such as collision warnings, door opening warnings, automatic parking, and sentry mode. By detecting objects around the vehicle and issuing timely warnings, vehicle perimeter intrusion detection can protect the driver's personal safety and property security. Currently, vehicle perimeter intrusion detection is gradually being applied to various vehicle models.

[0003] Currently, intrusion warning scenarios typically employ visual analysis methods, using sensors such as radar to acquire image data of the area around the vehicle. Data analysis is then used to determine the presence of objects and issue warnings. For example, ultrasonic radar is primarily used to identify objects at greater distances; short-range ultrasonic radar is generally used to identify objects at distances of 15–250 centimeters (cm), while long-range ultrasonic radar is generally used to identify objects at distances of 30–500 centimeters. For objects closer to the vehicle, due to the radar's low field of view (FOV), especially the small vertical FOV, there are certain blind spots in the radar's perception of objects at close range around the vehicle. Summary of the Invention

[0004] This application provides a method for detecting vehicle intrusion, which can detect whether a target object has infringed upon a vehicle, thereby improving the accuracy of vehicle intrusion detection.

[0005] In a first aspect, a method for detecting vehicle intrusion is provided, comprising: acquiring image information from a first camera device; acquiring, based on the image information, the area where a target object is located near the vehicle and the negative distance intrusion area of ​​the vehicle, wherein the negative distance intrusion area is related to the projection area of ​​the vehicle on the ground; determining whether the target object has entered the negative distance intrusion area based on the relative position of the area where the target object is located and the negative distance intrusion area; and issuing an early warning when the target object enters the negative distance intrusion area.

[0006] The vehicle intrusion detection scheme provided in this application determines the negative distance intrusion area based on the projection area of ​​the vehicle and the ground, thereby enabling the detection of whether a target object has entered a range relatively close to the vehicle, thus improving the accuracy of vehicle intrusion detection.

[0007] Alternatively, the negative distance invasion zone can be the area enclosed by the side of the vehicle.

[0008] Thus, when a target enters the negative distance invasion zone, it can be understood that the target may come into contact with the vehicle and may actually infringe upon the vehicle.

[0009] In conjunction with the first aspect, in some implementations of the first aspect, before acquiring image information from the first camera device, the method further includes: acquiring the orientation information of the target object and the distance information between the target object and the vehicle; activating at least one camera device based on the orientation information and the distance information, wherein the at least one camera device includes the first camera device.

[0010] The vehicle intrusion detection scheme provided in this application can activate one or more camera devices based on the location information of the target object and the distance information between the target object and the vehicle. In other words, the scheme of this application can activate the camera device corresponding to the location and distance information of the target object, rather than all camera devices, thus saving energy.

[0011] In conjunction with the first aspect, in some implementations of the first aspect, the orientation information and distance information are determined based on information collected by the detection device, and the orientation corresponding to the detection range of the detection device partially or completely overlaps with the orientation corresponding to the detection range of the first camera device; after activating at least one camera device based on the orientation information and distance information, the method further includes: deactivating the detection device.

[0012] The vehicle intrusion detection scheme provided in this application can activate the camera device and then deactivate it based on the location and distance information of the target object determined by the information collected by the detection device. This saves energy and avoids unnecessary energy waste.

[0013] In conjunction with the first aspect, in some implementations of the first aspect, the orientation information and distance information are determined based on information collected by the second camera device, and the distance between the second camera device and the first camera device is less than or equal to a preset distance threshold; after turning on at least one camera device based on the orientation information and distance information, the method further includes: turning off the second camera device when the time during which there is no object within the detection range of the second camera device is greater than or equal to a first time threshold.

[0014] The vehicle intrusion detection scheme provided in this application can activate the first camera device based on the location and distance information determined by information collected by another camera device (the second camera device) other than the first camera device. Furthermore, if no object appears within the detection range of the second camera device for a certain period of time, the second camera device can be deactivated. This saves energy.

[0015] On the other hand, the decision to turn off the second camera device can be made based on the time when there are no objects within its detection range, so as to ensure comprehensive detection of objects near the vehicle and minimize the lifespan loss caused by frequent turning the camera device on or off.

[0016] In conjunction with the first aspect, in some implementations of the first aspect, when a target object enters a negative distance intrusion area, an early warning is issued, including: when the target object enters a negative distance intrusion area, determining first feature information based on image information, the first feature information including the duration of the target object's movement in the negative distance intrusion area and / or the trajectory of the target object's movement in the negative distance intrusion area; and issuing an early warning based on the first feature information.

[0017] The vehicle intrusion detection scheme provided in this application can issue an early warning based on the movement trajectory and duration of the target object in the negative distance intrusion area when the target object enters the negative distance intrusion area, thereby improving the user experience.

[0018] In conjunction with the first aspect, in some implementations of the first aspect, the negative distance invasion area is the projection area of ​​a portion of the vehicle on the ground within the detection range of the first camera device.

[0019] In conjunction with the first aspect, in some implementations of the first aspect, the outer edge of the negative distance invasion area is determined based on the outline of the projection area of ​​the vehicle on the ground.

[0020] In this way, whether a target object has entered the negative distance invasion zone can be determined based on the relative position of the outer edge of the negative distance invasion zone and the area where the target object is located. This eliminates the need to determine the entire coordinate set of the negative distance invasion zone when determining whether the target object has entered, thus saving data overhead during invasion detection.

[0021] In conjunction with the first aspect, in some implementations of the first aspect, the inner edge line of the negative distance invasion area is determined based on the outline of the projection area of ​​the top curve of the vehicle onto the ground.

[0022] Secondly, a device for detecting vehicle intrusion is provided, comprising: an acquisition unit for acquiring image information from a first camera device; a processing unit for acquiring, based on the image information, the area where a target object is located near the vehicle and the negative distance intrusion area of ​​the vehicle, wherein the negative distance intrusion area is related to the projection area of ​​the vehicle on the ground; the processing unit is further configured to determine whether the target object has entered the negative distance intrusion area based on the relative position of the area where the target object is located and the negative distance intrusion area; and the processing unit is further configured to issue an early warning when the target object enters the negative distance intrusion area.

[0023] Alternatively, the negative distance invasion zone can be the area enclosed by the side of the vehicle.

[0024] At this point, when the target enters the negative distance invasion zone, it can be understood that the target may come into contact with the vehicle and may actually infringe upon the vehicle.

[0025] In conjunction with the second aspect, in some implementations of the second aspect, the acquisition unit is further configured to acquire the orientation information of the target object and the distance information between the target object and the vehicle; the processing unit is further configured to activate at least one camera device based on the orientation information and the distance information, wherein the at least one camera device includes a first camera device.

[0026] In conjunction with the second aspect, in some implementations of the second aspect, the azimuth information and distance information are determined based on the information collected by the detection device, and the azimuth corresponding to the detection range of the detection device and the azimuth corresponding to the detection range of the first camera device partially or completely overlap; the processing unit is also used to shut down the detection device.

[0027] In conjunction with the second aspect, in some implementations of the second aspect, the orientation information and distance information are determined based on the information collected by the second camera device, and the distance between the second camera device and the first camera device is less than or equal to a preset distance threshold; the processing unit is also used to: turn off the second camera device when the time during which there is no object within the detection range of the second camera device is greater than or equal to a first time threshold.

[0028] In conjunction with the second aspect, in some implementations of the second aspect, the processing unit is further configured to: when the target object enters the negative distance invasion area, determine first feature information based on image information, the first feature information including the movement duration of the target object in the negative distance invasion area and / or the movement trajectory of the target object in the negative distance invasion area; and issue an early warning based on the first feature information.

[0029] In conjunction with the second aspect, in some implementations of the second aspect, the negative distance invasion area is the projection area of ​​a portion of the vehicle on the ground within the detection range of the first camera device.

[0030] In conjunction with the second aspect, in some implementations of the second aspect, the outer edge of the negative distance invasion area is determined based on the outline of the projection area of ​​the vehicle on the ground.

[0031] In conjunction with the second aspect, in some implementations of the second aspect, the inner edge line of the negative distance invasion area is determined based on the outline of the projection area of ​​the vehicle's top curve on the ground.

[0032] Thirdly, a computer-readable medium is provided, which stores program code that, when run on a computer, causes the computer to perform the method described in any one of the first aspects.

[0033] Fourthly, a chip system is provided, comprising: a processor and a data interface, wherein the processor reads instructions stored in a memory through the data interface to execute the method described in any one of the first aspects.

[0034] Fifthly, a vehicle intrusion detection device is provided, comprising: at least one processor and a memory, the at least one processor being coupled to the memory for reading and executing instructions in the memory to perform the method described in any one of the first aspects above.

[0035] In a sixth aspect, a computer program product is provided, the computer product comprising: a computer program that, when the computer program is run, causes a computer to perform the method described in any one of the first aspects.

[0036] A seventh aspect is to provide a means of transport comprising the device described in any one of the first aspects above.

[0037] The infringement detection method provided in this application can determine whether a target object has entered an area close to a vehicle (negative distance infringement zone) based on image information from a first camera device and issue an early warning, thus improving the accuracy of infringement detection. One or more camera devices are activated based on the target object's orientation information and the distance information between the target object and the vehicle. That is, only the camera device corresponding to the target object's orientation and distance information is activated, rather than all camera devices, to save energy. The first camera device is activated based on the target object's orientation and distance information determined by the information collected by the detection device, and the detection device is turned off after the first camera device is activated. This saves energy and avoids unnecessary energy waste. The first camera device is activated based on the orientation and distance information determined by the information collected by the second camera device, and the second camera device is turned off when there is no object within its detection range for a certain period of time. This ensures comprehensive detection of target objects near the vehicle as much as possible, saves power consumption, and minimizes the lifespan loss caused by frequent activation and deactivation of the camera devices. Warnings are issued based on the movement trajectory and duration of a target object within the negative-distance intrusion zone to alert vehicle owners or drivers, ensuring their personal and property safety. The relative position of the target object's location to the outer edge of the negative-distance intrusion zone determines whether the target object has entered the zone. This eliminates the need to determine the entire coordinate set of the negative-distance intrusion zone when identifying its entry, saving data overhead during intrusion detection. Attached Figure Description

[0038] Figure 1 This is a functional block diagram of the vehicle 100 to which this application embodiment applies.

[0039] Figure 2 This is a schematic diagram of the architecture of the infringement detection system provided in the embodiments of this application.

[0040] Figure 3 This is a schematic structural diagram of the camera device for a vehicle provided in the embodiments of this application.

[0041] Figure 4 This is a flowchart illustrating the infringement detection method provided in the embodiments of this application.

[0042] Figure 5 This is a schematic diagram of the negative distance invasion area provided in the embodiments of this application.

[0043] Figure 6 This is a flowchart illustrating the infringement detection method provided in the embodiments of this application.

[0044] Figure 7This is a schematic diagram of the negative distance intrusion warning process provided in the embodiments of this application.

[0045] Figure 8 This is a schematic diagram showing the relative positions of the area where the object is located and the infringed area, as provided in the embodiments of this application.

[0046] Figure 9 This is a schematic diagram showing the relative positions of the area where the object is located and the infringed area, as provided in the embodiments of this application.

[0047] Figure 10 This is a schematic diagram of the human-computer interaction interface for the infringement warning provided in this application.

[0048] Figure 11 This is a schematic block diagram of an infringement detection device provided in an embodiment of this application.

[0049] Figure 12 This is a schematic block diagram of an infringement detection device provided in an embodiment of this application. Detailed Implementation

[0050] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0051] Figure 1 This is a functional block diagram of the vehicle 100 to which this application embodiment applies.

[0052] The vehicle 100 may include various subsystems, such as a driving system 110, a sensing system 120, a display device 130, and a computing platform 140.

[0053] Optionally, vehicle 100 may include more or fewer subsystems, and each subsystem may include multiple components. Furthermore, each subsystem and component of vehicle 100 may be interconnected via wired or wireless means.

[0054] Exemplarily, the mobility system 110 may include components for providing powered motion to the vehicle 100. In one embodiment, the mobility system 110 may include an engine 111, a transmission 112, an energy source 113, and wheels 114 / tires. The engine 111 may be an internal combustion engine, an electric motor, an air compressor engine, or other combinations of engines; for example, a hybrid engine consisting of a gasoline engine and an electric motor, or a hybrid engine consisting of an internal combustion engine and an air compressor engine. The engine 111 can convert the energy source 113 into mechanical energy.

[0055] For example, energy source 113 may include gasoline, diesel, other petroleum-based fuels, propane, other compressed gas-based fuels, ethanol, solar panels, batteries, and other sources of electricity. Energy source 113 may also provide energy to other systems of vehicle 100.

[0056] For example, the transmission 112 may include a gearbox, a differential, and a drive shaft; wherein the transmission 112 can transmit mechanical power from the engine 111 to the wheels 114.

[0057] In one embodiment, the transmission 112 may also include other components, such as a clutch. The drive shaft may include one or more shafts that can be coupled to one or more wheels 114.

[0058] For example, the sensing system 120 may include several sensors for sensing information about the environment surrounding the vehicle 100.

[0059] For example, the sensing system 120 may include a positioning system 121 (e.g., a global positioning system, BeiDou system, or other positioning system), an inertial measurement unit (IMU) 122, a lidar 123, a millimeter-wave radar 124, an ultrasonic radar 125, and a camera device 126. The sensing system 120 may also include sensors from the internal systems of the monitored vehicle 100 (e.g., an in-vehicle air quality monitor, fuel gauge, oil temperature gauge, etc.). Sensor data from one or more of these sensors can be used to detect objects and their corresponding characteristics (position, shape, orientation, speed, etc.). This detection and identification is a key function for the safe operation of the autonomous vehicle 100.

[0060] The positioning system 121 can be used to estimate the geographical location of the vehicle 100.

[0061] The inertial measurement unit 122 is used to sense changes in the position and orientation of the vehicle 100 based on inertial acceleration. In some embodiments, the inertial measurement unit 122 may be a combination of an accelerometer and a gyroscope.

[0062] The lidar 123 can use lasers to sense objects in the environment in which the vehicle 100 is located. In some embodiments, the lidar 123 may include one or more laser sources, a laser scanner, and one or more detectors, as well as other system components.

[0063] The millimeter-wave radar 124 can use radio signals to sense objects in the surrounding environment of the vehicle 100. In some embodiments, in addition to sensing objects, the radar 126 can also be used to sense the speed and / or direction of travel of the objects.

[0064] The ultrasonic radar 125 can use ultrasonic signals to sense objects around the vehicle 100.

[0065] The camera device 126 can be used to capture image information of the surrounding environment of the vehicle 100. The camera device 126 may include a monocular camera, a binocular camera, a structured light camera, and a panoramic camera, etc. The image information acquired by the camera device 126 may include still images or video stream information.

[0066] like Figure 1 As shown, the vehicle 100 can interact with the user through the display device 130.

[0067] For example, information can be provided to the user of vehicle 100 through display device 130.

[0068] Some or all of the functions of vehicle 100 can be controlled by computing platform 140. Computing platform 140 may include processors 141 to 14n (n being a positive integer). A processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction read and execute capabilities, such as a central processing unit (CPU), microprocessor, graphics processing unit (GPU) (which can be understood as a type of microprocessor), or digital signal processor (DSP). In another implementation, the processor can implement certain functions through the logical relationships of hardware circuits. These logical relationships are fixed or reconfigurable. For example, the processor may be a hardware circuit implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), such as a field-programmable gate array (FPGA). In reconfigurable hardware circuits, the process of the processor loading a configuration document and configuring the hardware circuit can be understood as the process of the processor loading instructions to implement some or all of the functions of the aforementioned units. In addition, it can also be hardware circuitry designed for artificial intelligence, which can be understood as an ASIC, such as a neural network processing unit (NPU), tensor processing unit (TPU), deep learning processing unit (DPU), etc. Furthermore, the computing platform 140 may also include a memory for storing instructions. Some or all of the processors 141 to 14n can call the instructions in the memory to execute them and achieve the corresponding functions.

[0069] The computing platform 140 can control the functions of the vehicle 100 based on inputs received from various subsystems (e.g., the sensing system 120). In some embodiments, the computing platform 140 is operable to provide control over many aspects of the vehicle 100 and its subsystems.

[0070] Optionally, the components described above are merely examples. In actual applications, components in each of the above modules may be added or removed as needed. Figure 1 This should not be construed as a limitation on the embodiments of this application.

[0071] Autonomous vehicles traveling on roads, such as vehicle 100 above, can identify objects in their surrounding environment to determine adjustments to their current speed. These objects can be other vehicles, traffic control equipment, or other types of objects. In some examples, each identified object can be considered independently, and based on the object's individual characteristics, such as its current speed, acceleration, and distance from the vehicle, the speed adjustment to be made by the autonomous vehicle can be determined.

[0072] Optionally, vehicle 100 or its associated perception and computing devices (e.g., computing platform 140) can predict the behavior of the identified objects based on the characteristics of the identified objects and the state of the surrounding environment (e.g., traffic, rain, ice on the road, etc.). Optionally, each identified object depends on the behavior of the others, so all identified objects can also be considered together to predict the behavior of a single identified object. Vehicle 100 can adjust its speed based on the predicted behavior of the identified objects. In other words, the autonomous vehicle can determine what steady state the vehicle will need to adjust to (e.g., accelerate, decelerate, or stop) based on the predicted behavior of the objects. In this process, other factors can also be considered in determining the speed of vehicle 100, such as the lateral position of vehicle 100 in the road, the curvature of the road, the proximity of static and dynamic objects, etc.

[0073] In addition to providing instructions to adjust the speed of the autonomous vehicle, the computing device can also provide instructions to modify the steering angle of the vehicle 100 so that the autonomous vehicle follows a given trajectory and / or maintains a safe lateral and longitudinal distance from objects near the autonomous vehicle (e.g., cars in adjacent lanes on the road).

[0074] The aforementioned vehicle 100 can be a car, truck, motorcycle, public vehicle, ship, airplane, helicopter, lawnmower, recreational vehicle, amusement park vehicle, construction equipment, tram, golf cart, train, etc., and this application embodiment does not impose any special limitations.

[0075] Violation detection around a vehicle is a common safety warning scenario considered in the automotive industry, such as scratch warning, door opening warning, and sentry mode. These can protect the driver's personal safety and property while the vehicle is in motion or parked.

[0076] Currently, the most common vehicle perimeter intrusion warning systems are based on sensors such as radar. For example, ultrasonic radar is mainly used to identify objects at a relatively long distance. Short-range ultrasonic radar is generally used to identify objects at a distance of 15–250 cm, while long-range ultrasonic radar is generally used to identify objects at a distance of 30–500 cm. For objects that are relatively close to the vehicle, due to the low field of view (FOV) of radar, especially the small vertical FOV, there is a certain blind spot in radar's perception at close range around the vehicle, making it impossible to detect whether an object is intruding on the vehicle body.

[0077] Especially when most of the object is far from the vehicle (e.g., more than 20cm), and only a part of it touches the vehicle or a part is very close to the vehicle (e.g., within 20cm), it is impossible to accurately detect whether the object has infringed on the vehicle. For example, when a person is standing more than 20cm away from the vehicle, and only their arm is outstretched and about to touch or has already touched the vehicle, the vehicle cannot effectively recognize the intrusion and issue a warning, thus failing to guarantee vehicle safety.

[0078] To address the aforementioned issues, this application proposes a method for detecting vehicle intrusions, which can detect whether an object has infringed upon a vehicle and issue an early warning, thereby improving the accuracy of vehicle intrusion detection.

[0079] Figure 2 A schematic diagram of the architecture of the infringement detection system provided in an embodiment of this application is shown. Figure 2 The vehicle detection system can be applied to Figure 1 Of the 100 vehicles.

[0080] The intrusion detection system may include an object segmentation module 210, an intrusion detection module 220, a camera adaptive scheduling module 230, and a graded early warning module 240. The object segmentation module 210, intrusion detection module 220, camera adaptive scheduling module 230, and graded early warning module 240 may each be implemented by one or more modules, or they may be implemented by the same module.

[0081] The object segmentation module 210 may include a video stream acquisition module 211 and an object segmentation module 212. The object segmentation module 212 can determine the precise object segmentation region based on the data acquired by the video stream acquisition module 211. During object segmentation, ground segmentation can be added to remove issues such as shadows, ghosting, and other artifacts. A multi-source data fusion step can also be added to compensate for errors caused by long distances and insufficient information.

[0082] The intrusion detection module 220 may include an intrusion area mapping module 221, a close-range intrusion detection module 222, a negative-range intrusion detection module 223, and a vehicle body detection module 224. The intrusion area may include a close-range intrusion area and a negative-range intrusion area.

[0083] The invasion region mapping module 221 can map the invasion region in the world coordinate system onto the image. The vehicle body detection module 224 can obtain the vehicle body segmentation region based on the image and determine the outer edge line of the negative distance invasion region. The close-range invasion detection module 222 and the negative-range invasion detection module 223 can determine whether the object has entered the negative-range invasion region based on the relative position of the object's location in the image coordinate system and the invasion region.

[0084] The camera adaptive scheduling module 230 may include a multi-source data fusion module 231, a path following module 232, and a camera adaptive scheduling module 233. The multi-source data fusion module 231 can fuse one or more data sources, such as radar signal data, video image data, infrared data, vibration sensor data, and vehicle speed data. The path following module 232 can determine the movement path of an object and predict its future direction of movement, and send this information to the camera adaptive scheduling module 233. The camera adaptive scheduling module 233 can then activate or deactivate the corresponding camera based on this information.

[0085] For example, if no object appears within the camera's detection range for a certain period of time, the camera adaptive scheduling module 233 can control the camera to turn off. After the camera is turned off, the sensor corresponding to the camera will be activated. When an object appears, the detection device at the corresponding location (e.g., a radar sensor) can detect the object's position and report this information to the path following module 233. When the distance between the object and the vehicle is less than or equal to a certain distance, the camera adaptive scheduling module 233 will activate the camera corresponding to the location within the sensor's detection range.

[0086] The graded warning module 240 can present warning information in different forms on the vehicle-mounted system and user terminal side. The warning information can include different levels, such as low-level, medium-level, and high-level warnings. The graded warning module can trigger alarms of different levels on the vehicle-mounted system and / or user terminal side, and present the warning information of different levels, such as the movement trajectory of the intruding target, its distance from the vehicle, and its relative position to the vehicle, to the user through a human-machine interface in different ways. The graded warning module can also store warning information according to different warning levels.

[0087] The modules of an intrusion detection system can be implemented as processor-invoked software. For example, the intrusion detection system includes a processor connected to a memory containing instructions. The processor invokes these instructions to implement any of the above methods or to achieve the functions of each unit of the device. The processor can be, for example, a general-purpose processor, such as a CPU or microprocessor, and the memory can be internal or external to the device. Alternatively, the modules of the intrusion detection system can be implemented as hardware circuits. The functions of some or all units can be implemented through the design of the hardware circuit, which can be understood as one or more processors. For example, in one implementation, the hardware circuit is an ASIC, and the functions of some or all units are implemented through the design of the logical relationships between the components within the circuit. In another implementation, the hardware circuit can be implemented using a PLD (Plug-in Display Device), such as an FPGA, which can include a large number of logic gates. The connection relationships between the logic gates are configured through a configuration file, thereby achieving the functions of some or all units. All modules of the above intrusion detection system can be implemented entirely through processor-invoked software, entirely through hardware circuits, or partially through processor-invoked software with the remaining parts implemented through hardware circuits.

[0088] Figure 3 This is a schematic structural diagram of the camera device for a vehicle provided in an embodiment of this application. Figure 3 The camera device can detect intrusions by vehicles. Figure 3 The camera device can be applied to Figure 1 Of the 100 vehicles.

[0089] Figure 3 The document describes the number and installation location of the camera devices. The number and location of the camera devices in this application are merely examples and are not intended to limit the scope of the application.

[0090] For example, a vehicle may include multiple camera devices. For instance, it may include a front camera, a left-side camera, a right-side camera, a rear camera, a left-front camera, and a right-front camera.

[0091] In this embodiment, the multiple camera devices in the vehicle may include a first camera device and a second camera device, wherein the distance between the second camera device and the first camera device is less than or equal to a preset distance threshold. The vehicle can determine the location information of the target object and the distance information between the target object and the vehicle based on the second camera device, and then activate the first camera device based on the location information and distance information. The vehicle can determine whether there is a target object near the vehicle and the positional relationship between the area where the target object is located near the vehicle and the negative distance intrusion area based on the image information from the first camera device, and determine whether the target object has entered the negative distance intrusion area.

[0092] Alternatively, the camera device can be a linear camera. Or, it can be a non-linear camera, such as a fisheye camera.

[0093] Alternatively, the camera device can be a color camera. Or, it can be a depth camera.

[0094] Optionally, the field of view (FOV) of the camera device can be greater than or equal to 90°. The FOV of the camera device can also be greater than or equal to other thresholds, such as 120°, 110°, 100°, 80°, etc.

[0095] This application does not limit the type of camera device. By adjusting the installation position, installation angle, and number of camera devices, the camera devices can detect the negative distance intrusion area around the vehicle as comprehensively as possible.

[0096] Figure 4 This is a schematic flowchart of the vehicle intrusion detection method provided in the embodiments of this application. Figure 4 The vehicle intrusion detection method can be executed by a vehicle, a computing platform, or a system consisting of a computing platform and a camera device. In one embodiment, when the method is executed by a computing platform, the computing platform can be located in the vehicle, for example, Figure 1 The computing platform 140 shown. Alternatively, the computing platform may also be located in a cloud server. The vehicle acquires image information via a camera and sends the image information to the cloud server. The cloud server can determine whether a target object has entered the negative distance intrusion zone based on the image information sent by the vehicle. When the cloud server determines that the target object has entered the negative distance intrusion zone, it can send indication information to the vehicle, which instructs the target object to enter the negative distance intrusion zone. The method includes:

[0097] S401, acquire image information from the first camera device.

[0098] Figure 3 The multiple camera devices in the image may include a first camera device.

[0099] S402, Based on the image information, obtain the area where the target object is located near the vehicle and the negative distance invasion area of ​​the vehicle.

[0100] It should be understood that the region involved in the embodiments of this application can be a three-dimensional space. For ease of description, the embodiments of this application do not distinguish between region and space, and use region (e.g., negative distance invasion region, region where the target object is located) for description. For example, space can be represented in the form of a projected region.

[0101] In some implementations, the area where the target object is located can be the set of coordinates of the target object and the projected area of ​​the ground in the image coordinate system.

[0102] The target object can be moving or stationary. A vehicle can be moving or stationary. This application does not limit this.

[0103] In some implementations, image segmentation techniques can be used to obtain the region where the target object is located, such as frame lookup algorithms and neural network models.

[0104] In some implementations, a multi-source data fusion module can be used to analyze data from different perspectives. For example, one or more data sources such as radar signal data, video image data, infrared data, vibration sensor data, and vehicle speed data can be fused and analyzed to obtain the area where the target object is located.

[0105] In some implementations, the negative distance invasion area is related to the area of ​​the vehicle's projection on the ground. Figure 5 This is a schematic diagram of the negative distance invasion area provided in the embodiments of this application. The following is in conjunction with... Figure 5 A brief introduction to the negative distance invasion zone.

[0106] In some implementations, the negative distance invasion area can be the projected area of ​​the vehicle and the ground. For example... Figure 5 As shown in (a), the projection area of ​​the vehicle and the ground can be understood as the area enclosed by the outline of the projection area of ​​the vehicle and the ground.

[0107] In some implementations, the negative distance invasion area may be within the projection area of ​​the vehicle and the ground.

[0108] For example, such as Figure 5 As shown in (b), the annular area enclosed by the outline of the vehicle's projection area on the ground and the outline of the roofline's projection on the ground can be considered as the negative distance invasion area.

[0109] For example, the negative distance intrusion zone can also be the annular area enclosed by the outline of the vehicle's projection area on the ground and the outline of the roofline's projection on the ground.

[0110] For example, the negative distance invasion area can be the projection area of ​​the vehicle body portion onto the ground within the detection range of the first camera device. Figure 5 (c) is an image captured by a camera on the left rearview mirror of the vehicle, showing the negative distance intrusion area within the detection range of the camera, i.e. Figure 5 The negative distance invasion area in (d) (top view).

[0111] When the negative distance violation area is within the projection area of ​​the vehicle and the ground, the negative distance violation area can be determined based on its inner and outer edge lines. For example, the outer edge line of the negative distance violation area is determined based on the outline of the projection area of ​​the vehicle and the ground. The inner edge line of the negative distance violation area can be determined based on the outline of the projection area of ​​the roofline and the ground. Within the detection range of the first camera device, the outline of the projection of the outer side of the vehicle body portion within the detection range of the first camera device onto the ground can be used as the outer edge line of the negative distance violation area; the outline of the projection area of ​​the roofline of this portion of the vehicle body within the detection range of the first camera device onto the ground can be used as the inner edge line of the negative distance violation area.

[0112] In some implementations, the negative distance intrusion area may be related to the area enclosed by the vehicle's body around its perimeter. Alternatively, it can be understood as being related to the area enclosed by the vehicle's sides (front, rear, left, and right).

[0113] In some implementations, the negative distance intrusion zone can be the area enclosed by the vehicle's body on all four sides. Alternatively, it can be understood as the area enclosed by the sides (front, rear, left, and right) of the vehicle. In this case, when a target object enters the negative distance intrusion zone, it can be considered that the target object may come into contact with the vehicle and may actually intrude upon it.

[0114] The negative distance intrusion region can be a set of coordinates in a world coordinate system preset based on the vehicle model. The vehicle body detection module can obtain the vehicle body segmentation region based on the image information from the first camera device, and then map the set of coordinates of the negative distance intrusion region in the world coordinate system to the image coordinate system based on the vehicle body segmentation region, thereby obtaining the negative distance intrusion region in the image coordinate system.

[0115] In the image coordinate system, the shape of the negative distance invasion area can be determined according to different parameters such as the installation position and angle of the camera device and the vehicle model. This application is only an example and does not limit it.

[0116] S403, determine whether the target object has entered the negative distance invasion zone based on the relative position of the area where the target object is located and the negative distance invasion zone.

[0117] In some implementations, whether a target object has entered a negative distance invasion region can be determined based on the intersection ratio between the region where the target object is located and the negative distance invasion region. For example, whether a target object has entered a negative distance invasion region can be determined according to formula (1). When the intersection ratio is greater than or equal to a preset threshold, the target object has entered the negative distance invasion region; when the intersection ratio is less than the preset threshold, the target object has not entered the negative distance invasion region.

[0118] J L1=F((∑L1∩L2) / ∑L2,I1) Formula (1)

[0119] J L1 The output determines whether the target object has entered the negative distance invasion region. F is a comparison function, L1 is the set of coordinates of the region where the target object is located in the image coordinate system, L2 is the set of coordinates of the negative distance invasion region in the image coordinate system, I1 is the first preset threshold, / is the division symbol, ∑ is the summation symbol, and ∩ is the intersection symbol. Where L1∩L2 is the intersection of the coordinates of the region where the target object is located and the coordinates of the negative distance invasion region, ∑L1∩L2 is the number of coordinate points in L1∩L2, and ∑L2 is the number of coordinate points in L2.

[0120] In some implementations, whether a target object has entered a negative distance violation zone can be determined based on the relative position of the area where the target object is located and the outer edge line of the negative distance violation zone. Thus, by determining the coordinate set of the outer edge line of the negative distance violation zone and the coordinate set of the area where the target object is located through the vehicle body segmentation area, it can be determined whether the target object has entered the negative distance violation zone. For example, the intersection ratio between the area where the target object is located and the outer edge line of the negative distance violation zone can be used to determine whether the target object has entered the negative distance violation zone, as shown in formula (2). When the intersection ratio is greater than or equal to a preset threshold, the target object has entered the negative distance violation zone; when the intersection ratio is less than the preset threshold, the target object has not entered the negative distance violation zone.

[0121] J S =F((∑L1∩S) / ∑S,I2) Formula (2)

[0122] J S The output determines whether the target object has entered the negative distance invasion region. F is a comparison function, L1 is the set of coordinates of the region containing the target object in the image coordinate system, S is the set of coordinates of the outer edge of the negative distance invasion region in the image coordinate system, I2 is the second preset threshold, / is the division symbol, ∑ is the summation symbol, and ∩ is the intersection symbol. Specifically, L1∩S is the intersection of the set of coordinates of the region containing the target object and the set of coordinates of the outer edge of the negative distance invasion region, ∑L1∩S is the number of coordinate points in L1∩S, and ∑S is the number of coordinate points in S.

[0123] S404 issues an early warning when a target enters the negative-range invasion zone.

[0124] In some implementations, a negative distance intrusion warning is issued when a target object enters the negative distance intrusion zone.

[0125] In some implementations, when a target object enters a negative-range intrusion zone, first feature information can be determined based on image information. This first feature information may include the target object's movement trajectory and / or movement duration within the negative-range intrusion zone. A warning can then be issued based on the target object's movement trajectory and / or movement duration within the negative-range intrusion zone. For example, warnings can be categorized into low-level, medium-level, and high-level warnings; the longer the target object's movement trajectory and / or the longer its movement duration within the negative-range intrusion zone, the higher the warning level.

[0126] Figure 6 This is a schematic flowchart of the vehicle intrusion detection method provided in the embodiments of this application.

[0127] Figure 6 The vehicle intrusion detection method can be executed by a vehicle, a computing platform, or a system consisting of a computing platform and a camera device. In one embodiment, when the method is executed by a computing platform, the computing platform can be located in the vehicle, for example, Figure 1 The computing platform 140 shown. Alternatively, the computing platform can also be located in a cloud server. The vehicle collects information via a camera (second camera device) or a detection device and sends it to the cloud server. The cloud server can determine the location and distance information of the target object based on the information sent by the vehicle, and activate the camera (including the first camera device) corresponding to the location of the target object based on the location and distance information. The vehicle collects image information via the camera (first camera device) and sends the image information to the cloud server. The cloud server can determine whether the target object has entered the negative distance intrusion zone based on the image information sent by the vehicle. When the cloud server determines that the target object has entered the negative distance intrusion zone, it can send an instruction message to the vehicle, which is used to instruct the target object to enter the negative distance intrusion zone. The method includes:

[0128] S601, obtain the location information of the target object and the distance information between the target object and the vehicle.

[0129] In some implementations, the location information of the target object and the distance information between the target object and the vehicle can be obtained based on information collected by a detection device (e.g., radar). For example, when the target object appears at a certain location relatively far from the vehicle (e.g., more than 60 cm away), the detection device at that location can detect the target object first. The location information and distance information of the target object can be obtained based on the information collected by the detection device.

[0130] In some implementations, the location information of the target object and the distance information between the target object and the vehicle can be obtained using the second camera device. When the target object is in a position relatively close to the vehicle (e.g., within 60cm), the location information and distance information of the target object can be obtained through the information collected by the second camera device corresponding to that position.

[0131] S602, based on the orientation information and distance information, activate at least one camera device.

[0132] In some implementations, when a target object appears at a location relatively far from the vehicle, the detection device corresponding to that location can detect the target object first. When the distance between the target object and the vehicle is less than or equal to the target distance, at least one camera device can be activated, wherein the at least one camera device includes a first camera device, and the location corresponding to the detection range of the detection device partially or completely overlaps with the location corresponding to the detection range of the first camera device. For example, the location information of the target object may correspond to the detection range of only one camera device, and when the target object appears at that location, then one camera device can be activated, and that camera device can be the first camera device. The location information of the target object may correspond to the detection range of multiple camera devices, and when the target object appears at that location, multiple camera devices can be activated, and the multiple camera devices may include the first camera device.

[0133] When at least one camera device is turned on, the detection device corresponding to the location within the detection range of these cameras can be turned off to save energy.

[0134] In some implementations, when the target object is relatively close to the vehicle and is moving from the location corresponding to the detection range of the second camera to the location corresponding to the detection range of the first camera, the information collected by the second camera can be used to determine whether the target object is about to enter or has already entered the location corresponding to the first camera, and then the first camera can be activated. After the first camera is activated, if no object (including the target object and other objects) appears within the detection range of the second camera for a certain period of time, the second camera can be deactivated.

[0135] Within a preset time after the second camera device is turned off, the detection device corresponding to the location within the detection range of the second camera device can be turned on.

[0136] In some embodiments, the distance between the second camera device and the first camera device is less than or equal to a preset distance threshold. For example, the cameras around the vehicle are sequentially camera device 1, camera device 2, camera device 3, camera device 4, camera device 5, and camera device 6. For example, the first camera device can be camera device 1, and the second camera device can be camera device 2 or camera device 6. The distance between the first camera device and the second camera device is less than or equal to the preset distance threshold.

[0137] When the first camera device is in the on state, it is not necessary to turn the first camera device on again.

[0138] S603, acquire image information from the first camera device.

[0139] S604: Based on image information, obtain the area where the target object is located near the vehicle and the area infringed by the vehicle.

[0140] In some implementations, the invasion area may include a near-field invasion area and a negative-field invasion area.

[0141] In some implementations, the close-range intrusion zone can be determined based on an area outside the projected area of ​​the vehicle and the ground. For example, the inner edge of the close-range intrusion zone can be the outer edge of the negative-range intrusion zone, and the distance between the outer edge of the close-range intrusion zone and the inner edge of the close-range intrusion zone can be a certain distance (e.g., 25 cm).

[0142] In some implementations, both the near-range invasion region and the negative-range invasion region can be sets of coordinates in a preset world coordinate system. These coordinate sets are then mapped to the image coordinate system based on image information. The coordinates of the invasion region in the world coordinate system can be transformed into the coordinates of the invasion region in the image coordinate system based on one or more parameters such as the camera device's correction coefficients, rotation matrix, translation matrix, and distortion coefficients.

[0143] For example, formula (3) can be used to obtain the coordinate set L2 of the negative distance invasion region in the image coordinate system.

[0144]

[0145] in, R is the set of coordinates of the negative distance invasion region in the world coordinate system, R is the camera correction coefficient rotation matrix, and T is the translation matrix.

[0146] For example, formula (4) can be used to obtain the coordinate set L3 of the near-field invasion area in the image coordinate system.

[0147]

[0148] in, R is the set of coordinates of the close-range invasion area in the world coordinate system, R is the camera correction coefficient rotation matrix, and T is the translation matrix.

[0149] In some implementations, the vehicle body segmentation region L4(X4,Y4) can be obtained from the image information by the vehicle body detection module, and the coordinate set S(Xs,Ys) of the outer edge line of the negative distance invasion region can be determined based on the vehicle body segmentation region. For example, in the left-side scene, the coordinate set S(Xs,Ys) of the outer edge line of the negative distance invasion region can be obtained using formula (5).

[0150] X S =max{X,(X,Ys)∈L4} Formula (5)

[0151] Where L4 is the set of coordinates of the vehicle body segmentation region, (X,Ys) is the set of coordinates of the vehicle body segmentation region when Y equals Ys, and Ys is the ordinate of any point on L4. max is the maximum value symbol, and ∈ is the categorization symbol.

[0152] S605, determine whether the target object has entered the infringed area based on the relative position of the area where the target object is located and the infringed area.

[0153] In some implementations, whether a target object has entered a negative distance invasion zone can also be determined by the relative position of the area where the target object is located and the negative distance invasion zone. The determination method can be found in the relevant description in S403, and will not be repeated here.

[0154] In some implementations, whether a target has entered a close-range invasion zone can be determined by the relative position of the area where the target is located and the close-range invasion zone.

[0155] For example, the intersection ratio can be used to determine whether a target object has entered the close-range invasion zone. When the intersection ratio is greater than or equal to a preset threshold, the target object has entered the close-range invasion zone; when the intersection ratio is less than the preset threshold, the target object has not entered the close-range invasion zone. Formula (6) can be used to determine whether a target object has entered the close-range invasion zone.

[0156] J L3 =F((∑L1∩L3) / ∑L3,I3) Formula (6)

[0157] Among them, J L3This outputs whether the target object has entered the close-range invasion zone. F is the comparison function, I3 is the third preset threshold, L1 is the set of coordinates of the region where the target object is located in the image coordinate system, and L3 is the set of coordinates of the close-range invasion zone in the image coordinate system. / represents division, ∑ represents summation, and ∩ represents intersection. Where L1∩L3 is the intersection of the coordinates of the region where the target object is located and the coordinates of the close-range invasion zone, ∑L1∩L3 is the number of coordinate points in L1∩L3, and L3 is the number of coordinate points in L3.

[0158] S606 issues an early warning when a target enters the intrusion area.

[0159] In some implementations, a close-range intrusion warning is issued when a target object enters the close-range intrusion area.

[0160] In some implementations, when a target object enters the close-range intrusion zone, information such as the target object's movement trajectory and / or movement duration within the zone can be obtained from image information. Then, a close-range intrusion warning can be issued based on this information. For example, the warning may include low-level, medium-level, and high-level warnings. The longer the target object's movement trajectory and / or the longer its movement duration within the close-range intrusion zone, the higher the level of the close-range intrusion warning.

[0161] In some implementations, a negative distance intrusion warning is issued when a target object enters the negative distance intrusion zone.

[0162] In some implementations, when a target object enters the negative distance intrusion zone, information such as the target object's movement trajectory and / or movement duration within the negative distance intrusion zone can be obtained from image information. Then, a negative distance intrusion warning can be issued based on this information. For example, the warning may include low-level, medium-level, and high-level warnings. The longer the target object's movement trajectory and / or the longer its movement duration within the negative distance intrusion zone, the higher the level of the negative distance intrusion warning.

[0163] like Figure 7 The diagram shown is a flowchart of a negative distance intrusion warning system provided in an embodiment of this application. The following will combine... Figure 7 This application describes a negative distance intrusion warning method based on embodiments of the present application. The method for negative distance intrusion warning includes steps S701 to S704, which are described below.

[0164] S701, Obtain the feature information of the target object.

[0165] In some implementations, first feature information can be obtained based on image information. The first feature information includes the movement trajectory of the target object in the negative distance invasion area and / or the movement duration of the target object in the negative distance invasion area.

[0166] In some implementations, the trajectory of a target object within the negative distance violation zone can be represented by length. For example, within the negative distance violation zone, if the target object moves clockwise from a first position on the left side of the vehicle to a second position behind the vehicle, the trajectory is D1; ​​if the target object moves clockwise from the first position on the left side of the vehicle to a third position on the right side of the vehicle, the trajectory is D2. Then, the length of trajectory D2 is greater than the length of trajectory D1. As another example, if the target object moves clockwise from the first position on the left side of the vehicle to a second position behind the vehicle and back once, the trajectory is D3; if the target object moves clockwise from the first position on the left side of the vehicle to the second position behind the vehicle and back twice, the trajectory is D4. Then, the length of trajectory D4 is greater than the length of trajectory D3.

[0167] In some implementations, the duration of the target object's stay in the negative distance invasion zone may include a static stay time and a motion stay time, where the motion stay time represents the duration of the target object's movement in the negative distance invasion zone, and the static stay time represents the duration of the target object's stillness in the negative distance invasion zone.

[0168] In some implementations, the first feature information may also include one or more of the following: the duration of the target object's stationary position in the negative distance invasion area, the number of times the target object invades the negative distance invasion area, the distance between the target object and the vehicle, the target object's running speed in the negative distance invasion area, the number of target objects in the negative distance invasion area, the type of target object, and whether the target object is a registered person of the vehicle, etc.

[0169] In some implementations, second feature information may also be obtained, which may include one or more of the following: the operating speed of the vehicle, the complexity of the environment in which the vehicle is located, etc.

[0170] The categories and specific descriptions of the first and second feature information are shown in Table 1.

[0171] Table 1

[0172]

[0173] S702, determine the warning coefficient based on the characteristic information of the target object.

[0174] In some implementations, one or more of the first feature information and / or the second feature information can be fused to obtain a warning coefficient.

[0175] For example, a2, a3, a6, and b2 are continuous feature information. One or more of these continuous feature information can be normalized and weighted to obtain the continuous infringement feature value P. C As shown in formula (7).

[0176] P C =∑[r C,n f C,n (C n )] Formula (7)

[0177] Among them, f C,n (*) is a vector mapping function, C n For the numerical values ​​of continuous feature information, r C,n ∑ represents the weights of continuous feature information, and ∑ is the summation symbol.

[0178] For example, a1, a4, a5, a7, a8, a9, and b1 are discrete feature information. One or more of these discrete feature information can be weighted to obtain a discontinuous infringement feature value P. D As shown in formula (8).

[0179] P D =∑[r D,n g D,n (D n )] Formula (8)

[0180] Among them, g D,n (*) is the normalization function, D n r represents the numerical value of discontinuous feature information. D,n The weights of continuous feature information, where ∑ is the summation symbol.

[0181] For example, the continuous invasion feature value P can be used. C Non-continuous invasion characteristic value P D The warning coefficient P is obtained by summing the results according to certain weights.

[0182] S703, the warning level is determined based on the relationship between the warning coefficient and the warning threshold.

[0183] In some implementations, the warning levels may include low-level warning, medium-level warning, and high-level warning.

[0184] For example, two warning thresholds Y1 and Y2 (Y1 < Y2) can be set. A low-level warning is issued when P ≤ Y1; a medium-level warning is issued when Y1 < P < Y2; and a high-level warning is issued when P > Y2. The longer the target object's trajectory within the negative distance invasion zone, the higher the warning coefficient and the higher the warning level. The longer the target object's movement duration within the negative distance invasion zone, the higher the warning coefficient and the higher the warning level.

[0185] S704, issue warnings based on the warning level.

[0186] Different levels of early warning information can be delivered to users in different forms, so that users know the warning level and the infringement information of the target object.

[0187] For example, a low-level warning can alert the driver to the presence of a target object intrusion in the form of a normal notification ringtone (such as a short ringtone) on the vehicle-mounted system and / or user terminal side. It can also remind the driver of the warning level, the target object's movement trajectory, the area of ​​infringement, and its relative position to the vehicle body in the form of pictures and text on the human-machine interface. At the same time, it can also store the data, pictures and / or videos of the target object intruding on the vehicle according to the level.

[0188] For example, a medium-level warning can alert the driver to the presence of an intrusion target by using an important notification ringtone (such as a long ringtone) and vibration on the vehicle-mounted system and / or the user terminal. It can also inform the driver of the warning level, the target's movement trajectory, the area of ​​infringement, and its relative position to the vehicle body through pictures and text on the human-machine interface. At the same time, it can also store the data, pictures and / or videos of the target intrusion into the vehicle according to the level.

[0189] For example, advanced warning systems can alert the driver to intrusion via audible and vibration signals on the vehicle's infotainment system and / or the user terminal. The warning information, including the alert level, the target's trajectory, the area of ​​intrusion, and its relative position to the vehicle, can be displayed on the human-machine interface using prominent colors such as red, along with images and text. The system can also store data, images, and / or video recordings of the intrusion. Additionally, warnings can be issued via interior and exterior vehicle lights.

[0190] Figure 8 This diagram illustrates the relative positions of the target object's location and the vehicle encroachment area provided in this embodiment of the application.

[0191] Taking the image information obtained by the camera device on the left side mirror of the vehicle as an example, based on the relative position of the area where the target object is located and the area of ​​infringement around the vehicle, it is determined whether the target object has entered the close-range infringement area ABCD and the negative-range infringement area ABGF of the vehicle.

[0192] For example, if the preset threshold I1 can be set to 0, then it can be determined whether the target object has entered the negative distance invasion zone based on whether there is an intersection between the area where the target object is located and the negative distance invasion zone. If there is an intersection between the area where the target object is located and the negative distance invasion zone, the target object has entered the negative distance invasion zone; if there is no intersection between the area where the target object is located and the negative distance invasion zone, the target object has not entered the negative distance invasion zone.

[0193] For example, if the preset threshold I3 can be set to 0, then it can be determined whether the target object has entered the close-range invasion zone based on whether the area where the target object is located intersects with the close-range invasion zone. If the area where the target object is located intersects with the close-range invasion zone, the target object has entered the close-range invasion zone; if the area where the target object is located does not intersect with the close-range invasion zone, the target object has not entered the close-range invasion zone.

[0194] like Figure 8 As shown in (a), at time T1, when the target object is far from the vehicle (e.g., greater than 60 cm), the detection device, such as radar, detects the presence of the target object. When the distance between the target object and the vehicle is less than or equal to the target distance, the left-side camera can be activated. At time T1, the area where the target object is located does not intersect with the close-range intrusion zone ABCD, and the target object has not entered the close-range intrusion zone. At time T1, the area where the target object is located does not intersect with the negative-range intrusion zone ABGF, and it can be considered that the target object has not entered the negative-range intrusion zone.

[0195] like Figure 8 As shown in (b), at time T2, the target object gradually approaches the vehicle. At time T2, the area where the target object is located intersects with the close-range invasion area ABCD, so it can be considered that the target object has entered the close-range invasion area. At time T2, the area where the target object is located does not intersect with the negative-range invasion area ABGF, so it can be considered that the target object has not entered the negative-range invasion area.

[0196] like Figure 8 As shown in (c), at time T3, the target object continues to approach the vehicle, and part of the target object's area enters the negative range intrusion zone. At time T3, the area where the target object is located intersects with the close range intrusion zone ABCD, so the target object can be considered to have entered the close range intrusion zone. At time T3, the target object's area intersects with the negative range intrusion zone ABGF, so the target object can be considered to have entered the negative range intrusion zone.

[0197] Figure 9 This diagram illustrates the relative positions of the area where the target object is located and the area invaded by the vehicle, as provided in an embodiment of this application.

[0198] Taking the image information from the camera device at the rear of the vehicle as an example, based on the relative position of the target object's location and the invasion area, it is determined whether the target object has entered the vehicle's close-range invasion area MNJK and negative-range invasion area MNOQ.

[0199] The target object moves away from the left side of the vehicle and gradually approaches the rear side. As the target object approaches the rear side, and the distance between the target object and the vehicle remains less than the target distance, the left-side camera can detect whether the target object is about to enter or has already entered the detection range of the rear-side camera. When the target object is about to enter or has already entered the detection range of the rear-side camera, the rear-side camera is activated.

[0200] For example, the first preset threshold I1 can be set to 0. Then, it can be determined whether the target object has entered the negative distance invasion zone based on whether there is an intersection between the area where the target object is located and the negative distance invasion zone. When there is an intersection between the area where the target object is located and the negative distance invasion zone, the target object has entered the negative distance invasion zone; when there is no intersection between the area where the target object is located and the negative distance invasion zone, the target object has not entered the negative distance invasion zone.

[0201] For example, the third preset threshold I3 can be set to 0. Then, it can be determined whether the target object has entered the close-range invasion zone based on whether there is an intersection between the area where the target object is located and the close-range invasion zone. If there is an intersection between the area where the target object is located and the close-range invasion zone, the target object has entered the close-range invasion zone; if there is no intersection, the target object has not entered the close-range invasion zone.

[0202] like Figure 9 As shown in (a), at time T4, the vehicle appears behind the vehicle. At time T4, the area where the target is located does not intersect with the close-range invasion area MNJK, and the target has not entered the close-range invasion area. At time T4, the area where the target is located does not intersect with the negative-range invasion area MNOQ, and the target has not entered the negative-range invasion area.

[0203] like Figure 9 As shown in (b), at time T5, the target continues to approach the vehicle. At time T5, the area where the target is located intersects with the close-range invasion zone MNJK, and the target enters the close-range invasion zone. At time T5, the area where the target is located does not intersect with the negative-range invasion zone MNOQ, and the target does not enter the negative-range invasion zone.

[0204] like Figure 9As shown in (c), at time T6, the target continues to approach the vehicle. At time T6, the area where the target is located intersects with the close-range invasion zone MNJK, and the target enters the close-range invasion zone. At time T6, the area where the target is located intersects with the negative-range invasion zone MNOQ, and the target enters the close-range invasion zone.

[0205] Figure 10 A schematic diagram of the human-computer interaction interface for the infringement warning provided in this application.

[0206] The human-computer interaction interface can display the current status warning information and also view historical warning information.

[0207] For example, when a target object enters the intrusion area, the human-computer interaction interface can display keyframe photos of the target object's intrusion, as well as the area and location of the intrusion, and indicate the level of the intrusion. For instance, in the case of a high-level negative-distance intrusion warning, the human-computer interaction interface can display keyframe photos of the target object's intrusion, and can also display the warning level, the target object's movement trajectory, the target object's area and location, and other information in a conspicuous color such as red.

[0208] For example, historical violations can also be queried through a human-computer interaction interface. For instance, historical violations can be queried by warning level, violation type, or time.

[0209] This application also provides an apparatus for implementing any of the above methods. For example, an apparatus is provided that includes a unit (or means) for implementing the steps performed by a vehicle in any of the above methods.

[0210] Figure 11 This is a schematic block diagram of a vehicle violation detection device according to an embodiment of this application. Figure 11 The apparatus 4000 shown includes an acquisition unit 4010 and a processing unit 4020.

[0211] The acquisition unit 4010 and the processing unit 4020 can be used to execute the infringement detection method of the embodiments of this application.

[0212] The acquisition unit 4010 is used to acquire image information from the first camera device.

[0213] The processing unit 4020 is used to obtain, based on image information, the area where the target object is located near the vehicle and the negative distance invasion area of ​​the vehicle, the negative distance invasion area being related to the projection area of ​​the vehicle on the ground.

[0214] The processing unit 4020 is used to determine whether the target object has entered the negative distance invasion zone based on the relative position of the area where the target object is located and the negative distance invasion zone.

[0215] The processing unit 4020 is also used to issue an early warning when a target object enters the negative distance invasion zone.

[0216] Optionally, as an embodiment, the acquisition unit 4010 is further configured to acquire the orientation information of the target object and the distance information between the target object and the vehicle; the processing unit 4020 is configured to activate at least one camera device based on the orientation information and the distance information, wherein the at least one camera device includes a first camera device.

[0217] Optionally, as an embodiment, the orientation information and distance information are determined based on the information collected by the detection device, and the orientation corresponding to the detection range of the detection device partially or completely overlaps with the orientation corresponding to the detection range of the first camera device; after turning on at least one camera device based on the orientation information and distance information, the processing unit 4020 is also used to turn off the detection device.

[0218] Optionally, as an embodiment, the orientation information and distance information are determined based on the information collected by the second camera device, and the distance between the second camera device and the first camera device is less than or equal to a preset distance threshold; after turning on at least one camera device based on the orientation information and distance information, the processing unit is further configured to: turn off the second camera device when the time during which there is no object within the detection range of the second camera device is greater than or equal to a first time threshold.

[0219] Optionally, as an embodiment, the processing unit 4020 is further configured to: determine first feature information based on image information when the target object enters the negative distance invasion area, the first feature information including the movement trajectory of the target object in the negative distance invasion area and / or the movement duration of the target object in the negative distance invasion area; and issue a warning based on the first feature information.

[0220] Alternatively, as an embodiment, the outer edge line of the negative distance invasion area is determined based on the outline of the projection area of ​​the vehicle on the ground.

[0221] Optionally, as an embodiment, the inner edge line of the negative distance invasion area is determined based on the contour line of the projection area of ​​the vehicle's top curve on the ground.

[0222] Optionally, as an embodiment, the negative distance invasion area is the projection area of ​​a portion of the vehicle on the ground within the detection range of the first camera device.

[0223] It should be understood that the division of units in the aforementioned device 4000 is only a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, the units in the device can be implemented by a processor calling software; for example, the device includes a processor connected to memory, which stores instructions. The processor calls the instructions stored in memory to implement any of the above methods or to implement the functions of each unit in the device. The processor can be, for example, a general-purpose processor, such as a CPU or microprocessor, and the memory can be internal or external to the device. Alternatively, the units in the device can be implemented as hardware circuits. The functions of some or all units can be implemented through the design of the hardware circuits, which can be understood as one or more processors. For example, in one implementation, the hardware circuit is an ASIC, and the functions of some or all units are implemented through the design of the logical relationships between the components within the circuit. In another implementation, the hardware circuit can be implemented using a PLD, such as an FPGA, which can include a large number of logic gates. The connection relationships between the logic gates are configured through a configuration file, thereby implementing the functions of some or all units. All units of the above devices can be implemented entirely through processor calling software, or entirely through hardware circuits, or partially through processor calling software with the remaining parts implemented through hardware circuits.

[0224] In this application's embodiments, a processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction reading and execution capabilities, such as a CPU, microprocessor, GPU (which can be understood as a type of microprocessor), or DSP. In another implementation, the processor can implement certain functions through the logical relationships of hardware circuits. These logical relationships are fixed or reconfigurable. For example, the processor is a hardware circuit implemented as an Application-Specific Integrated Circuit (ASIC) or PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document and configuring the hardware circuit can be understood as the processor loading instructions to implement the functions of some or all of the above units. Furthermore, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as an NPU, TPU, or DPU.

[0225] As can be seen, each unit in the above device can be one or more processors (or processing circuits) configured to implement the above methods, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.

[0226] Furthermore, the units in the above devices can be integrated in whole or in part, or they can be implemented independently. In one implementation, these units are integrated together as a system-on-a-chip (SOC). The SOC may include at least one processor for implementing any of the above methods or implementing the functions of the units in the device. The at least one processor may be of different types, such as CPU and FPGA, CPU and artificial intelligence processor, CPU and GPU, etc.

[0227] Figure 12 This is a schematic diagram of the hardware structure of the vehicle intrusion detection device provided in the embodiments of this application. Figure 12 The vehicle intrusion detection device 5000 shown includes a memory 5001, a processor 5002, a communication interface 5003, and a bus 5004. The memory 5001, processor 5002, and communication interface 5003 are interconnected via the bus 5004.

[0228] The memory 5001 may be a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 5001 may store a program, and when the program stored in the memory 5001 is executed by the processor 5002, the processor 5002 is used to perform the various steps of the vehicle intrusion detection method of the present application embodiment.

[0229] The processor 5002 may be a general-purpose CPU, microprocessor, ASIC, GPU, or one or more integrated circuits, used to execute relevant programs to implement the vehicle intrusion detection method of the present application method embodiments.

[0230] The processor 5002 can also be an integrated circuit chip with signal processing capabilities. In implementation, each step of the vehicle intrusion detection method of this application can be completed through integrated logic circuits in the processor 5002 or through software instructions.

[0231] The processor 5002 described above can also be a general-purpose processor, DSP, ASIC, FPGA, or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory 5001, and the processor 5002 reads the information in memory 5001 and combines it with its hardware to complete the task. Figure 12 The apparatus shown includes units that are required to perform functions, or to perform the vehicle intrusion detection method of the method embodiments of this application.

[0232] The communication interface 5003 uses transceiver devices, such as, but not limited to, transceivers, to enable communication between the device 5000 and other devices or communication networks. For example, image information can be acquired through the communication interface 5003.

[0233] Bus 5004 may include a pathway for transmitting information between various components of device 5000 (e.g., memory 5001, processor 5002, communication interface 5003).

[0234] This application also provides a computer-readable medium storing program code for execution by a device, the program code including a method for performing vehicle intrusion detection according to embodiments of this application.

[0235] This application also provides a computer program product containing instructions that, when run on a computer, cause the computer to execute the vehicle intrusion detection method described in this application.

[0236] This application also provides a chip, which includes a processor and a data interface. The processor reads instructions stored in a memory through the data interface and executes the vehicle intrusion detection method of this application.

[0237] Optionally, as one implementation, the chip may further include a memory storing instructions, and the processor is used to execute the instructions stored in the memory. When the instructions are executed, the processor is used to perform the vehicle intrusion detection method in the embodiments of this application.

[0238] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0239] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0240] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0241] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0242] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0243] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0244] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0245] In this application, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or multiple items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0246] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0247] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for detecting vehicle intrusion, characterized in that, include: Acquire image information from the first camera device; Based on the image information, the area where the target object is located near the vehicle and the negative distance invasion area of ​​the vehicle are obtained. The outer edge line of the negative distance invasion area is determined based on the outline of the projection area of ​​the vehicle on the ground, and the inner edge line of the negative distance invasion area is determined based on the outline of the projection area of ​​the top curve of the vehicle on the ground. Whether the target object has entered the negative distance invasion zone is determined based on the relative position of the area where the target object is located and the negative distance invasion zone. An early warning is issued when the target object enters the negative distance invasion area.

2. The method as described in claim 1, characterized in that, Before acquiring image information from the first camera device, the method further includes: Obtain the location information of the target object, and the distance information between the target object and the vehicle; At least one camera device is activated based on the orientation information and the distance information, wherein the at least one camera device includes the first camera device.

3. The method as described in claim 2, characterized in that, The orientation information and the distance information are determined based on the information collected by the detection device. The orientation corresponding to the detection range of the detection device partially or completely overlaps with the orientation corresponding to the detection range of the first camera device. After activating at least one camera device based on the orientation information and the distance information, the method further includes: The detection device was shut down.

4. The method as described in claim 2, characterized in that, The orientation information and the distance information are determined based on the information collected by the second camera device, and the distance between the second camera device and the first camera device is less than or equal to a preset distance threshold. After activating at least one camera device based on the orientation information and the distance information, the method further includes: When the time during which there is no object within the detection range of the second camera device is greater than or equal to a first time threshold, the second camera device is turned off.

5. The method according to any one of claims 1 to 4, characterized in that, The provision of an early warning when the target object enters the negative distance invasion zone includes: When the target object enters the negative distance invasion area, first feature information is determined based on the image information. The first feature information includes the duration of the target object's movement in the negative distance invasion area and / or the trajectory of the target object's movement in the negative distance invasion area. A warning will be issued based on the first feature information.

6. The method according to any one of claims 1 to 5, characterized in that, The negative distance invasion area is the projection area of ​​a portion of the vehicle on the ground within the detection range of the first camera device.

7. A device for detecting vehicle intrusion, characterized in that, include: The acquisition unit acquires image information from the first camera device; The processing unit is configured to obtain, based on the image information, the area where the target object near the vehicle is located and the negative distance invasion area of ​​the vehicle, wherein the outer edge line of the negative distance invasion area is determined based on the outline of the projection area of ​​the vehicle on the ground, and the inner edge line of the negative distance invasion area is determined based on the outline of the projection area of ​​the top curve of the vehicle on the ground. The processing unit is configured to determine whether the target object has entered the negative distance invasion area based on the relative position of the area where the target object is located and the negative distance invasion area. The processing unit is also used to issue an early warning when the target object enters the negative distance invasion area.

8. The apparatus as claimed in claim 7, characterized in that, include: The acquisition unit is also used to acquire the location information of the target object and the distance information between the target object and the vehicle; The processing unit is further configured to activate at least one camera device based on the orientation information and the distance information, wherein the at least one camera device includes the first camera device.

9. The apparatus as claimed in claim 8, characterized in that, The orientation information and the distance information are determined based on the information collected by the detection device. The orientation corresponding to the detection range of the detection device partially or completely overlaps with the orientation corresponding to the detection range of the first camera device. The processing unit is also used to: shut down the detection device.

10. The apparatus as claimed in claim 8, characterized in that, The orientation information and the distance information are determined based on the information collected by the second camera device, and the distance between the second camera device and the first camera device is less than or equal to a preset distance threshold. The processing unit is further configured to: shut down the second camera device when the time during which there is no object within the detection range of the second camera device is greater than or equal to a first time threshold.

11. The apparatus according to any one of claims 7 to 10, characterized in that, The processing unit is also used for: When the target object enters the negative distance invasion area, first feature information is determined based on the image information. The first feature information includes the duration of the target object's movement in the negative distance invasion area and / or the trajectory of the target object's movement in the negative distance invasion area. A warning will be issued based on the first feature information.

12. The apparatus according to any one of claims 7 to 11, characterized in that, The negative distance invasion area is the projection area of ​​a portion of the vehicle on the ground within the detection range of the first camera device.

13. A computer-readable medium, characterized in that, A computer-readable medium stores program code that, when run on a computer, causes the computer to perform the method as described in any one of claims 1 to 6.

14. A chip system, characterized in that, include: At least one processor and a memory, the at least one processor being coupled to the memory for reading and executing instructions in the memory to perform the method as claimed in any one of claims 1 to 6.

15. A vehicle intrusion detection device, characterized in that, include: At least one processor and a memory, the at least one processor being coupled to the memory for reading and executing instructions in the memory to perform the method as claimed in any one of claims 1 to 6.

16. A computer program product, characterized in that, The computer product includes: a computer program that, when run, causes the computer to perform the method as described in any one of claims 1 to 6.

17. A means of transportation, characterized in that, Includes the apparatus as described in any one of claims 7 to 12, 15.

Images