Vehicle control method, electronic equipment and vehicle

Abstract

The invention provides a vehicle control method, electronic equipment and a vehicle, and is applied to the technical field of vehicle intelligent control, and the method comprises the steps: obtaining the distortion information of a monitoring image, determining a plurality of target regions with different distortion degrees in the monitoring image based on the distortion information, and carrying out the target detection of the monitoring image; in response to the detected target, determining at least one target area where the target is located; and based on the target area where the target is located, determining the relative position of the target and the vehicle so as to judge whether the target invades the warning area or not. According to the invention, errors caused by monitoring image distortion are reduced, and the accuracy and reliability of target intrusion detection are improved.

Description

Technical Field

[0001] The present application relates to the field of vehicle intelligent control technology, and in particular to a vehicle control method, electronic equipment and a vehicle. Background Art

[0002] In recent years, more and more car brands have begun to install intrusion detection functions. Through on-board cameras and other sensors, abnormal conditions around the vehicle are monitored in real time after the vehicle is parked and turned off, and abnormal videos are recorded. However, traditional intrusion detection functions mainly rely on images taken by fisheye cameras for monitoring and judgment, but the images taken by fisheye cameras are highly distorted, and there are large errors in target intrusion judgment. Summary of the invention

[0003] In view of this, the purpose of the present application is to propose a vehicle control method, an electronic device and a vehicle to avoid large errors in target intrusion judgment caused by distortion of monitoring images.

[0004] Based on the above objectives, the present application provides a vehicle control method, comprising:

[0005] Obtaining distortion information of the monitoring image, determining a plurality of target areas with different distortion degrees in the monitoring image based on the distortion information, and performing target detection on the monitoring image;

[0006] In response to detecting the target, determining at least one target area where the target is located;

[0007] Based on the target area where the target is located, the relative position of the target and the vehicle is determined to determine whether the target invades the warning area.

[0008] Furthermore, the method further comprises:

[0009] In response to detecting the object, determining an area of ​​a bounding box of the object;

[0010] In response to determining that the area is less than or equal to a preset area, a relative distance between the target and the vehicle is determined, and in response to determining that the relative distance is less than or equal to a preset distance, it is determined that the target has invaded the warning area.

[0011] By calculating the area of ​​the target bounding box, you can preliminarily determine the distance of the target. A larger bounding box area usually means that the target is closer to the camera, while a smaller bounding box area means that the target is farther away. Using the area of ​​the target bounding box as an auxiliary judgment basis improves the accuracy and reliability of target position judgment. Select the appropriate detection method based on the area size to quickly determine whether the target is intrusive. This significantly improves the accuracy and reliability of detection, reduces false positives and false negatives, and enhances real-time response capabilities and user experience.

[0012] Based on the same inventive concept, the present application also provides a vehicle control device, including:

[0013] an acquisition module configured to acquire distortion information of the monitoring image, determine a plurality of target areas with different distortion degrees in the monitoring image based on the distortion information, and perform target detection on the monitoring image;

[0014] a determination module, configured to determine at least one target area where the target is located in response to detecting the target;

[0015] The detection module is configured to determine the relative position of the target and the vehicle based on the target area where the target is located, so as to determine whether the target invades the warning area.

[0016] Based on the same inventive concept, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein the processor implements the method as described above when executing the computer program.

[0017] Based on the same inventive concept, the present application also provides a vehicle, which includes the above-mentioned electronic device.

[0018] As can be seen from the above, the vehicle control method, electronic device and vehicle provided by the present application, wherein the method includes: obtaining the distortion information of the monitoring image, determining multiple target areas with different distortion degrees in the monitoring image based on the distortion information, and performing target detection on the monitoring image; in response to detecting the target, determining at least one target area where the target is located; based on the target area where the target is located, determining the relative position of the target and the vehicle to determine whether the target has invaded the warning area. Due to the characteristics of the fisheye camera, the image distortion is more serious at the edge of the monitoring image, resulting in a large error in position judgment. Based on the distortion information of the monitoring image, multiple target areas with different distortion degrees are determined in the monitoring image, and different methods for judging whether the target has invaded the warning area are selected based on the target area where the target is located, thereby reducing the error caused by the distortion of the fisheye camera image, so as to accurately detect whether the target has invaded, and can prevent common security threats such as vehicle scratches, door opening accidents, property theft, etc., and detect other potential abnormal situations, such as suspicious persons wandering around the vehicle, etc., to improve the accuracy and reliability of target intrusion detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the present application or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 This is a flow chart of a vehicle control method according to an embodiment of the present application;

[0021] Figure 2 This is monitoring image 1 in the vehicle control method of an embodiment of the present application;

[0022] Figure 3 This is monitoring image 2 in the vehicle control method of the embodiment of the present application;

[0023] Figure 4 This is monitoring image 3 in the vehicle control method of the embodiment of the present application;

[0024] Figure 5 This is monitoring image 4 in the vehicle control method of an embodiment of the present application;

[0025] Figure 6 This is monitoring image 5 in the vehicle control method of the embodiment of the present application;

[0026] Figure 7 This is monitoring image 6 in the vehicle control method of the embodiment of the present application;

[0027] Figure 8 This is monitoring image seven in the vehicle control method of the embodiment of the present application;

[0028] Fig. 9 This is monitoring image eight in the vehicle control method of the embodiment of the present application;

[0029] Fig.10 This is monitoring image nine in the vehicle control method of an embodiment of the present application;

[0030] Fig.11 A schematic diagram of a vehicle control device according to an embodiment of the present application;

[0031] Fig.12 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0032] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0033] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should be the usual meanings understood by people with ordinary skills in the field to which the present application belongs. The "first", "second" and similar words used in the embodiments of the present application do not represent any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing in front of the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0034] In related technologies, with the rapid development of smart car technology, vehicle safety monitoring systems have become an important part of improving driving safety and user experience. In recent years, more and more car brands have begun to carry intrusion detection functions. Through the on-board 360-degree fisheye camera and other sensors, after the vehicle is parked and turned off, abnormal conditions around the vehicle are monitored in real time and abnormal videos are recorded. This function can not only effectively prevent vehicle scratches, door opening accidents, property theft and other incidents, but also provide important evidence support. However, the traditional intrusion detection function mainly relies on the image of the 360-degree fisheye camera for monitoring and judgment, but due to the large image distortion of the fisheye camera, there is a large error in relying solely on the image for target detection and warning area intrusion judgment. This error may lead to missed recognition and misidentification of abnormal conditions, affecting the accuracy and reliability of the intrusion detection function.

[0035] Based on the above problems, the applicant found that: the distortion information of the monitoring image is obtained, multiple target areas with different distortion degrees are determined in the monitoring image based on the distortion information, and the monitoring image is subjected to target detection; in response to the detection of the target, at least one target area where the target is located is determined; based on the target area where the target is located, the relative position of the target and the vehicle is determined to determine whether the target has invaded the warning area. Due to the characteristics of the fisheye camera, the image distortion is more serious at the edge of the monitoring image, resulting in a large error in position judgment. Based on the distortion information of the monitoring image, multiple target areas with different distortion degrees are determined in the monitoring image, and different methods for judging whether the target has invaded the warning area are selected based on the target area where the target is located, thereby reducing the error caused by the distortion of the fisheye camera image, so as to accurately detect whether the target has invaded, prevent common security threats such as vehicle scratches, door opening accidents, property theft, etc., and detect other potential abnormal situations, such as suspicious persons wandering around the vehicle, etc., and improve the accuracy and reliability of target intrusion detection.

[0036] The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0037] The present application provides a vehicle control method, such as Figure 1 As shown, in some embodiments, the method is executed by a vehicle controller or a data processor independently provided by the vehicle controller, and the subsequent embodiments are all illustrated by taking the vehicle controller as an example; a camera is provided on the rearview mirror of the vehicle to collect monitoring images outside the vehicle; the monitoring images include a warning area; the method includes:

[0038] S101, obtaining distortion information of the monitoring image, determining a plurality of target areas with different distortion degrees in the monitoring image based on the distortion information, and performing target detection on the monitoring image;

[0039] In specific implementation, a camera is installed on the rearview mirror of the vehicle to collect monitoring images around the vehicle in real time. The type of camera usually equipped on the vehicle is a 360-degree fisheye camera, which can provide a full range of viewing angles. According to the distortion information, the monitoring image can be divided into multiple target areas with different distortion degrees. For example, the central area of ​​the image may be less distorted, while the edge area is more distorted. The division can be based on a quantitative indicator of the degree of distortion, such as the numerical range of the distortion parameter, or by the proportion of the distorted image area in the monitoring image. For example, when a region is a region in which the proportion of the distorted image area in the monitoring image is less than or equal to a preset proportion (exemplarily, the preset proportion can be set to 0.6), the region is determined as a target area. The monitoring image may be affected by factors such as lighting changes, weather conditions (such as rainy and snowy weather), and low-light environment at night. Therefore, before performing target detection, the image may need to be preprocessed. The preprocessing steps may include image denoising, brightness adjustment, contrast enhancement, etc. to improve image quality and facilitate subsequent target detection. Object detection refers to identifying and locating specific objects (such as pedestrians, vehicles, bicycles, etc.) in an image; deep learning-based object detection algorithms are usually used, such as R-CNN (Region-based Convolutional Neural Networks), YOLO (You Only Look Once), SSD (Single Shot MultiBox Detector), etc. These algorithms analyze the input image through a trained model, identify the objects in the image, and output the object bounding boxes and their categories (such as pedestrians, cars, bicycles, etc.) of these objects.

[0040] S102, in response to detecting a target, determining at least one target area where the target is located;

[0041] In specific implementation, after acquiring and preprocessing the monitoring image, the target detection algorithm is used to identify the target in the image. The target detection algorithm outputs the target bounding box of each target (such as Figure 7 In the detection, the vehicle is selected by the target bounding box, which is used to describe the position and size of the target in the monitoring image. The position of the target can be determined by determining the position of the center point of the target bounding box. The center point of the target bounding box is the geometric center point of the target bounding box. The position of the center point provides a simplified reference point, making position determination in complex images more intuitive and efficient, so as to accurately determine at least one target area where the target is located.

[0042] S103: Based on the target area where the target is located, determine the relative position of the target and the vehicle to determine whether the target invades the warning area.

[0043] In specific implementation, the monitoring image is usually divided into multiple target areas to facilitate a more precise judgment of the target position. Specifically, due to the characteristics of the fisheye camera, the edge of the monitoring image is severely distorted, resulting in a large error in position judgment. If the target area is determined to be an area in the monitoring image where the distorted image area accounts for less than or equal to the preset proportion (for example, the target area is Figure 2 When it is determined that the center point of the target detection frame is in the target area of ​​the monitoring image, the target is detected to have invaded the warning area based on the relative position of the target boundary box and the warning area of ​​the vehicle; because the distorted image area of ​​the target area accounts for a small proportion, the accuracy of detecting whether the target has invaded the target area by the position of the target boundary box in the target area is higher. If it is determined that the target area is an area in the monitoring image where the distorted image area accounts for more than a preset proportion (for example, the target area is 1 / 5 of the left edge and 1 / 5 of the right edge of the monitoring image, such as Figure 2S1A and S1B in the figure). When it is determined that the center point of the target detection frame is in the target area of ​​the monitoring image, since the distorted image area of ​​the target area accounts for a large proportion, there may be a large error in directly relying on the monitoring image to judge the target position. Therefore, the relative distance between the target and the vehicle is determined, and the target is detected to be intruded according to the relative distance; the vehicle's radar sensor can provide high-precision distance measurement data, thereby improving the accuracy of target position judgment and more accurately judging whether the target is intruded. When it is determined that the target has indeed invaded the warning area, it indicates that there are potential security threats around the vehicle, such as vehicle scratches, property theft, etc. At this time, a target intrusion alarm is issued, and an alarm sound can be sounded through a speaker inside or outside the car to remind the owner or surrounding personnel to pay attention; the alarm information is displayed through the dashboard or central control screen inside the car to prompt the owner to pay attention; the vehicle system is connected to the owner's mobile phone, and an alarm notification is sent to the owner's mobile phone to remind the owner to check for abnormal conditions; the camera recording function can also be automatically started to record the entire process of target intrusion and save it as a video file for subsequent viewing and evidence retention. The main purpose of issuing an alarm is to remind the owner or people around the vehicle to pay attention to potential security threats around the vehicle and take necessary preventive measures in time. The owner can understand the abnormal situation around the vehicle at the first time, prevent possible losses and safety accidents, and ensure the safety of the vehicle and the property inside the vehicle.

[0044] In this embodiment, by dividing the target area according to the degree of image distortion in the area and selecting different detection methods, the error caused by the fisheye camera image distortion is reduced, and the target is accurately detected to see if it is intruding. This can not only prevent common security threats such as vehicle scratches, door opening accidents, and property theft, but also detect other potential abnormal situations, such as suspicious persons wandering around the vehicle, etc. This improves the accuracy and reliability of detection, and enhances user experience and vehicle safety.

[0045] Based on the following embodiments, determining whether a target has invaded the warning area is described in detail to ensure that the vehicle surrounding environment can be effectively monitored in different states. In response to determining that the target area is an area in which the proportion of the distorted image area in the monitoring image is less than or equal to a preset proportion, the relative position of the target and the vehicle is determined based on the target area where the target is located to determine whether the target has invaded the warning area, including:

[0046] Determine whether the target invades the warning area according to the position of the lower left / right corner of the target bounding box of the target relative to the warning area of ​​the vehicle;

[0047] In a specific implementation, whether the target is intruded is determined according to the position of the left / right lower corner of the target boundary box or the position of the lower boundary of the target boundary box; first, the position of the center point of the target boundary box is determined; when the center point of the target boundary box is in the left preset area of ​​the monitoring image (when the rearview mirror is in the folded state, the left preset area of ​​the monitoring image can be set to the area of ​​the left half of the monitoring image, such as Figure 5 (S3 in the figure), determine whether the lower right corner of the target boundary box is in the warning area of ​​the vehicle; when it is determined that the lower right corner of the target boundary box is in the warning area of ​​the vehicle, it indicates that the target is too close to the vehicle, and the target is determined to be intruding; when it is determined that the center point of the target boundary box is in the right preset area of ​​the monitoring image (when the rearview mirror is in the folded state, the right preset area of ​​the monitoring image can be set to the area of ​​half of the right side of the monitoring image, such as Figure 6 In S4 of FIG. 1 , it is determined whether the lower left corner of the target bounding box is in the warning area of ​​the vehicle; when it is determined that the lower left corner of the target bounding box is in the warning area of ​​the vehicle (i.e., the target detection algorithm detects the target vehicle and selects the vehicle through the target detection box, and the lower left corner of the target bounding box is in the warning area of ​​the vehicle), it indicates that the target is too close to the vehicle, and the target is determined to be intruding. When it is determined that the lower left / right corner of the target bounding box is not in the warning area of ​​the vehicle.

[0048] Determining whether the target invades the warning area according to the positions of the left / right lower corners of the target bounding box of the target and the lower boundary of the target bounding box relative to the warning area of ​​the vehicle;

[0049] In a specific implementation, first determine whether the target is intruding according to the position of the left / right lower corner of the target boundary box. When it is determined that the left / right lower corner of the target boundary box is not in the warning area of ​​the vehicle, determine whether the target is intruding according to the position of the lower boundary of the target boundary box, and judge whether the lower boundary of the target boundary box intersects with the upper boundary and the lower boundary of the warning area at the same time; if it is determined that the lower boundary of the target boundary box intersects with the upper boundary and the lower boundary of the warning area at the same time (such as Figure 7 As shown), it indicates that the target is too close to the vehicle, and the target is determined to be intruding. This ensures safe monitoring of the vehicle's surroundings.

[0050] Whether the target invades the warning area is determined according to the position of the lower boundary of the target bounding box of the target relative to the warning area of ​​the vehicle.

[0051] In specific implementation, whether the target is invaded can be directly determined according to the position of the lower boundary of the target boundary box, that is, whether the lower boundary of the target boundary box intersects with the upper boundary and the lower boundary of the warning area at the same time; if it is determined that the lower boundary of the target boundary box intersects with the upper boundary and the lower boundary of the warning area at the same time (such as Figure 7 As shown), it indicates that the target is too close to the vehicle, and the target is determined to be intruding.

[0052] In this embodiment, the detection adaptability and accuracy are improved by judging the position of the lower left / right corner or the lower boundary of the target boundary box. When it is impossible to judge whether the target is intruded by the position of the lower left / right corner of the target boundary box, the position of the lower boundary of the target boundary box is used to judge, thereby ensuring the utilization rate and detection accuracy of the monitoring image, improving the accuracy and comprehensiveness of the detection, and enhancing the user experience and vehicle safety.

[0053] Based on the following embodiments, how to determine whether the target has invaded the warning area according to the position of the left / right lower corner of the target bounding box and the lower boundary of the target bounding box relative to the warning area of ​​the vehicle is described in detail to ensure that all potential intrusion threats can be detected in time. Determining whether the target has invaded the warning area according to the position of the left / right lower corner of the target bounding box and the lower boundary of the target bounding box relative to the warning area of ​​the vehicle includes:

[0054] In response to determining that the lower left / right corner of the target bounding box does not invade the warning area of ​​the vehicle, determining whether the lower boundary of the target bounding box intersects with the upper boundary and the lower boundary of the warning area at the same time;

[0055] In specific implementation, when it is determined that the target intrusion is not detected based on the position of the lower left / right corner of the target bounding box, that is, when it is determined that the lower left corner or the lower right corner of the target bounding box does not enter the warning area, it is necessary to further determine whether the lower boundary of the target bounding box intersects with the upper boundary and the lower boundary of the warning area at the same time; even if the corner point of the target does not enter the warning area, its lower boundary may still intersect with the warning area, thus posing a potential intrusion threat. The lower boundary of the target bounding box refers to the bottom edge of the target bounding box. If the lower boundary of the target bounding box intersects with the upper boundary and the lower boundary of the warning area at the same time, the target is determined to be invaded. If the lower boundary does not intersect with the upper boundary and the lower boundary of the warning area at the same time, the target is not invaded.

[0056] In response to determining that a lower boundary of the target bounding box intersects both an upper boundary and a lower boundary of the warning area, the target intrusion warning area is determined.

[0057] In specific implementation, when it is determined that the lower boundary of the target boundary box intersects with the upper boundary and the lower boundary of the warning area at the same time, it indicates that the target is too close to the vehicle, the target is determined to be intruded, and an alarm is triggered in time to remind the owner of the potential security threat. The accuracy and reliability of target position judgment are ensured, the accuracy of intrusion detection is improved, and the safety monitoring of the vehicle's surrounding environment is ensured.

[0058] In this embodiment, by determining whether the lower boundary of the target bounding box intersects with both the upper boundary and the lower boundary of the warning area, it is possible to detect targets whose corner points of the bounding box have not entered the warning area but may still pose a threat. This ensures that even if the corner points of the target bounding box have not entered the warning area, its lower boundary may still intersect with the warning area, thus posing a potential intrusion threat, and ensures that all potential intrusion threats can be detected in a timely manner.

[0059] Based on the following embodiments, how to determine whether the target has invaded the warning area according to the position of the lower boundary of the target bounding box relative to the warning area of ​​the vehicle is described in detail, thereby improving the comprehensiveness and accuracy of detection and reducing false positives and false negatives. The monitoring image includes an intermediate preset area; determining whether the target has invaded the warning area according to the position of the lower boundary of the target bounding box relative to the warning area of ​​the vehicle includes:

[0060] In response to determining that the center point of the target bounding box is in a middle preset area of ​​the monitoring image, determining a folding state of the vehicle rearview mirror when collecting the monitoring image outside the vehicle;

[0061] In specific implementation, the middle preset area is usually set in the middle third of the monitoring image. When it is determined that the center point of the target boundary box is in the middle preset area of ​​the monitoring image, the folding state of the vehicle rearview mirror when collecting the monitoring image outside the vehicle is determined. The rearview mirror is usually installed on both sides of the vehicle and equipped with a fisheye camera for collecting the monitoring image outside the vehicle. The folding state of the vehicle rearview mirror will directly affect the collection angle of the fisheye camera; when the rearview mirror is in a non-folded state, the collected monitoring image is as follows: Figure 3 As shown in the figure, the upper boundary of the warning area is basically parallel to the lower edge of the monitoring image, and the upper boundary of the warning area is on the side away from the vehicle, and the lower boundary of the warning area is on the side close to the vehicle; when the rearview mirror is in the folded state, the collected monitoring image is as follows Figure 4As shown in the figure, the upper boundary of the warning area is at a certain angle to the lower edge of the monitoring image, and the upper boundary of the warning area is on the side away from the vehicle, and the lower boundary of the warning area is on the side close to the vehicle. When the center point of the target boundary box is in the middle preset area of ​​the monitoring image, it is usually impossible to accurately determine whether the target is intruded based on the position of the lower left / right corner of the target boundary box. It is necessary to determine the folding state of the vehicle rearview mirror when collecting the monitoring image outside the vehicle, and determine the strategy for determining whether the target is intruded based on the folding state of the vehicle rearview mirror.

[0062] In response to determining that the rearview mirror is in a non-folded state, determining whether a central segment of a lower boundary of the target bounding box is in a warning area of ​​the vehicle;

[0063] In specific implementation, when the rearview mirror is in the unfolded state, it is possible to more accurately determine whether the target has invaded by determining whether the lower boundary center segment of the target boundary box is in the warning area of ​​the vehicle; first, the coordinate values ​​of all points in the lower boundary center segment of the target boundary box are extracted; if the coordinate value of any point in the lower boundary center segment is within the range of the warning area, it is determined that the lower boundary center segment enters the warning area.

[0064] In response to determining that a lower boundary center segment of the target bounding box is within a warning area of ​​the vehicle, it is determined that the target is intruding.

[0065] In specific implementation, it is determined whether the lower boundary center section of the target boundary box is in the warning area of ​​the vehicle; when it is determined that the lower boundary center section of the target boundary box is in the warning area of ​​the vehicle, it indicates that the target is too close to the vehicle, and the target is determined to be intruding. The comprehensiveness and accuracy of intrusion detection are improved, ensuring the safe monitoring of the vehicle's surrounding environment.

[0066] In this embodiment, by setting a middle preset area in the monitoring image, the center point of the target boundary box is in the middle preset area of ​​the monitoring image, and the folding state of the vehicle rearview mirror when collecting the monitoring image outside the vehicle, if the rearview mirror is in a non-folding state, by judging whether the center segment of the lower boundary of the target boundary box is in the warning area of ​​the vehicle, it is determined whether the target is intruding, ensuring that even if the target is in the middle area of ​​the monitoring image, it can be accurately judged whether it constitutes an intrusion threat, and ensuring that all potential intrusion threats can be detected in time. The comprehensiveness and accuracy of detection are significantly improved, false alarms and missed alarms are reduced, and real-time response capabilities and user experience are enhanced.

[0067] Based on the following embodiments, determining whether the target has invaded according to the position of the left / right lower corner of the target boundary box is described in detail to more accurately determine the position relationship of the target in the monitoring image and improve the accuracy and reliability of the target position determination. The monitoring image includes a left preset area and a right preset area; determining whether the target has invaded the warning area according to the position of the left / right lower corner of the target boundary box relative to the warning area of ​​the vehicle includes:

[0068] In response to determining that the center point of the target bounding box is in a preset area on the left side of the monitoring image, determining whether a lower right corner of the target bounding box is in a warning area of ​​the vehicle;

[0069] In specific implementation, the warning area is used to determine whether the target poses a threat to the vehicle. The size and position of the warning area can be configured according to actual needs. When it is determined that the center point of the target boundary box is in the preset area on the left side of the monitoring image, the coordinate value (xmax, ymin) of the lower right corner of the target boundary box can be extracted. Compare the coordinate value of the lower right corner with the boundary coordinate value of the warning area. If the xmax and ymax coordinate values ​​of the lower right corner are both within the range of the warning area, it is determined that the lower right corner enters the warning area.

[0070] In response to determining that the lower right corner of the target bounding box is in the warning area of ​​the vehicle, determining that the target is intruding;

[0071] In specific implementation, if it is determined that the lower right corner of the target boundary box is in the warning area of ​​the vehicle, it indicates that the target is too close to the vehicle and the target is determined to be intruding; then a target intrusion alarm is issued to remind the owner of the potential security threat. This ensures the accuracy and reliability of target position judgment, improves the accuracy of intrusion detection, and ensures safe monitoring of the vehicle's surrounding environment.

[0072] In response to determining that the center point of the target bounding box is in a preset area on the right side of the monitoring image, determining whether the lower left corner of the target bounding box is in a warning area of ​​the vehicle;

[0073] In specific implementation, when it is determined that the center point of the target boundary box is in the preset area on the right side of the monitoring image, the coordinate value (xmin, ymin) of the lower left corner of the target boundary box can be extracted. The coordinate value of the lower left corner is compared with the boundary coordinate value of the warning area. If the xmin and ymin coordinate values ​​of the lower left corner are both within the range of the warning area, it is determined that the lower left corner enters the warning area.

[0074] In response to determining that a lower left corner of the target bounding box is in a warning area of ​​the vehicle, it is determined that the target is intruding.

[0075] In specific implementation, if it is determined that the lower left corner of the target boundary box is in the warning area of ​​the vehicle, it indicates that the target is too close to the vehicle and the target is determined to be intruding; then a target intrusion alarm is issued to remind the owner of the potential security threat. This ensures the accuracy and reliability of target position judgment, improves the accuracy of intrusion detection, and ensures safe monitoring of the vehicle's surrounding environment.

[0076] In this embodiment, by determining the position of the center point of the target bounding box, the position relationship of the target in the monitoring image can be judged more accurately, which helps to improve the accuracy and reliability of target position judgment. In different target areas, judgment is made based on the position of the lower left / right corner of the target bounding box, which significantly improves the accuracy and reliability of detection, reduces false positives and false negatives, and enhances user experience and vehicle safety.

[0077] In some embodiments, in response to determining that the target area is an area in the monitoring image where the proportion of the distorted image area is greater than a preset proportion, determining the relative position of the target and the vehicle based on the target area where the target is located to determine whether the target has invaded the warning area includes:

[0078] Determine the relative distance between the target and the vehicle;

[0079] In response to determining that the relative distance is less than or equal to a preset distance, it is determined that the target has invaded the warning area.

[0080] In a specific implementation, if it is determined that the target area is an area in the monitoring image where the proportion of the distorted image area is greater than a preset proportion (such as Figure 8 As shown in the figure, the center point of the target boundary box is in the target area of ​​the monitoring image), and the relative distance between the target and the vehicle is determined; the vehicle's radar sensor can provide high-precision distance measurement data, thereby improving the accuracy of target position judgment and more accurately judging whether the target is intruding. When it is determined that the relative distance is less than or equal to the preset distance (exemplarily, the preset distance can be set to 20cm), it indicates that the target is too close to the vehicle, and the target is determined to have invaded the warning area.

[0081] In response to determining that the relative distance is greater than a preset distance, it is determined that the target has not invaded.

[0082] In specific implementation, if the relative distance between the target and the vehicle is greater than the preset distance, indicating that the target is far away from the vehicle, the target is judged not to have invaded. To ensure the accuracy and reliability of intrusion detection, the distance judgment provides a quantitative standard to confirm whether the target poses a threat to the vehicle, ensuring the safe monitoring of the vehicle's surrounding environment.

[0083] In this embodiment, relative distance judgment is used to confirm whether the target is an intrusion, which reduces false positives and false negatives caused by judgment errors due to image distortion. Relative distance judgment provides a quantitative standard to confirm whether the target poses a threat to the vehicle, ensures the objectivity and consistency of the judgment, and improves the reliability of intrusion detection; ensures that any potential intrusion threats can be discovered in a timely manner and provide necessary evidence support.

[0084] In some embodiments, the method further comprises:

[0085] In response to detecting the object, determining an area of ​​a bounding box of the object;

[0086] In specific implementation, the target is identified in the monitoring image through the target detection algorithm. The target detection algorithm generates a target bounding box of the target, which is used to describe the position and range of the target in the image. After the target is detected and the target bounding box is generated, the area of ​​the target bounding box is calculated. The size of the target bounding box area can reflect the size of the target in the image. A larger target bounding box area usually means that the target is closer to the fisheye camera, while a smaller target bounding box area means that the target is farther away from the fisheye camera.

[0087] In response to determining that the area is less than or equal to a preset area, a relative distance between the target and the vehicle is determined, and in response to determining that the relative distance is less than or equal to a preset distance, it is determined that the target has invaded the warning area.

[0088] In specific implementation, when it is determined that the area is less than or equal to a preset area (the preset area is a threshold set according to the specific application scenario and vehicle safety requirements, used to distinguish between closer and farther targets. For example, the preset area can be set to one-third of the monitoring image area), it means that the target is far away from the fisheye camera, and it is possible that the target is at the rear end of the vehicle. At this time, the accuracy of detecting whether the target is intruding based on the position of the target boundary box is poor, so it is necessary to determine the relative distance between the target and the vehicle, and detect whether the target is intruding based on the relative distance. When it is determined that the relative distance is less than or equal to the preset distance (for example, the preset distance can be set to 20cm), it indicates that the target is too close to the vehicle, and the target intrusion warning area is determined.

[0089] In this embodiment, by calculating the area of ​​the target bounding box, the distance of the target can be preliminarily determined. A larger bounding box area usually indicates that the target is closer to the camera, while a smaller bounding box area indicates that the target is farther away. Using the area of ​​the target bounding box as an auxiliary judgment basis improves the accuracy and reliability of the target position judgment. According to the size of the area, a suitable detection method is selected to quickly determine whether the target is intruded. The accuracy and reliability of the detection are significantly improved, false alarms and missed alarms are reduced, and the real-time response capability and user experience are enhanced.

[0090] The vehicle further includes a fisheye camera disposed at the front of the vehicle and a fisheye camera disposed at the rear of the vehicle; the method further includes:

[0091] Obtaining distortion information of the monitoring image, determining a plurality of target areas with different distortion degrees in the monitoring image based on the distortion information, and performing target detection on the monitoring image;

[0092] In response to detecting the target, determining at least one target area where the target is located;

[0093] Based on the target area where the target is located, the relative position of the target and the vehicle is determined to determine whether the target invades the warning area.

[0094] In a specific implementation, the monitoring images (such as Fig. 9 The monitoring image captured by the fisheye camera at the front of the vehicle shown in FIG. Fig.10 The monitoring image captured by the fisheye camera at the rear of the vehicle shown in the figure is similar to the monitoring image captured by the fisheye camera on the rearview mirror in the unfolded state of the rearview mirror, and the position of the warning area in the monitoring image is similar. Therefore, when it is determined that the center point of the target detection frame of the target is in the target area of ​​the monitoring image (the target area is an area in the monitoring image where the distorted image area accounts for less than or equal to the preset proportion), it is directly determined whether the target is intruded according to the position of the lower left / right corner of the target boundary box or the lower boundary center segment of the target boundary box relative to the warning area. When the center point of the target boundary box is in the preset area on the left side of the monitoring image, it is determined whether the lower right corner of the target boundary box is in the warning area of ​​the vehicle; when it is determined that the lower right corner of the target boundary box is in the warning area of ​​the vehicle, it indicates that the target is too close to the vehicle, and the target is determined to be intruded; when it is determined that the center point of the target boundary box is in the preset area on the right side of the monitoring image, it is determined whether the lower left corner of the target boundary box is in the warning area of ​​the vehicle; when it is determined that the lower left corner of the target boundary box is in the warning area of ​​the vehicle, it indicates that the target is too close to the vehicle, and the target is determined to be intruded. The monitoring image also includes a preset middle area; when it is determined that the center point of the target boundary box is in the preset middle area of ​​the monitoring image, it is determined whether the lower boundary center section of the target boundary box is in the warning area of ​​the vehicle; when it is determined that the lower boundary center section of the target boundary box is in the warning area of ​​the vehicle, it indicates that the target is too close to the vehicle, and the target is determined to be intruding. The comprehensiveness and accuracy of intrusion detection are improved to ensure safe monitoring of the vehicle's surrounding environment.

[0095] In this embodiment, by installing fisheye cameras at the front and rear of the vehicle, a full range of views can be provided to cover all key areas in front and behind the vehicle. This all-round monitoring ensures that there are no blind spots around the vehicle, and any potential intrusion threats can be discovered in time, which significantly improves the comprehensiveness and accuracy of detection, reduces false positives and missed positives, and enhances real-time response capabilities and user experience. It ensures that any potential intrusion threats can be discovered in time.

[0096] It should be noted that the method of the embodiment of the present application can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present application, and the multiple devices will interact with each other to complete the described method.

[0097] It should be noted that the above describes some embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0098] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a vehicle control device.

[0099] refer to Fig.11 , the vehicle control device comprises:

[0100] An acquisition module 701 is configured to acquire distortion information of the monitoring image, determine a plurality of target areas with different distortion degrees in the monitoring image based on the distortion information, and perform target detection on the monitoring image;

[0101] A determination module 702, configured to determine at least one target area where the target is located in response to detecting the target;

[0102] The detection module 703 is configured to determine the relative position between the target and the vehicle based on the target area where the target is located, so as to determine whether the target invades the warning area.

[0103] Furthermore, the detection module 703 is specifically used for:

[0104] Determine whether the target has invaded the warning area according to the position of the lower left / right corner of the target bounding box of the target relative to the warning area of ​​the vehicle; or,

[0105] Determine whether the target invades the warning area according to the positions of the lower left / right corners of the target bounding box and the lower boundary of the target bounding box relative to the warning area of ​​the vehicle; or,

[0106] Whether the target invades the warning area is determined according to the position of the lower boundary of the target bounding box of the target relative to the warning area of ​​the vehicle.

[0107] Further, the determination module 702 is specifically used for:

[0108] In response to determining that the lower left / right corner of the target bounding box does not invade the warning area of ​​the vehicle, determining whether the lower boundary of the target bounding box intersects with the upper boundary and the lower boundary of the warning area at the same time;

[0109] In response to determining that a lower boundary of the target bounding box intersects both an upper boundary and a lower boundary of the warning area, the target intrusion warning area is determined.

[0110] Furthermore, the detection module 703 is further specifically used for:

[0111] In response to determining that the center point of the target bounding box is in a middle preset area of ​​the monitoring image, determining a folding state of the vehicle rearview mirror when collecting the monitoring image outside the vehicle;

[0112] In response to determining that the rearview mirror is in a non-folded state, determining whether a central segment of a lower boundary of the target bounding box is in a warning area of ​​the vehicle;

[0113] In response to determining that a lower boundary center segment of the target bounding box is within the warning area of ​​the vehicle, the target intrusion warning area is determined.

[0114] Furthermore, the detection module 703 is further specifically used for:

[0115] In response to determining that the center point of the target bounding box is in a preset area on the left side of the monitoring image, determining whether a lower right corner of the target bounding box is in a warning area of ​​the vehicle;

[0116] In response to determining that the lower right corner of the target bounding box is in the warning area of ​​the vehicle, the target intrusion warning area is determined.

[0117] Furthermore, the detection module 703 is further specifically used for:

[0118] In response to determining that the center point of the target bounding box is in a preset area on the right side of the monitoring image, determining whether the lower left corner of the target bounding box is in a warning area of ​​the vehicle;

[0119] In response to determining that the lower left corner of the target bounding box is in the warning area of ​​the vehicle, the target intrusion warning area is determined.

[0120] Furthermore, the detection module 703 is further specifically used for:

[0121] Determine the relative distance between the target and the vehicle;

[0122] In response to determining that the relative distance is less than or equal to a preset distance, it is determined that the target has invaded the warning area.

[0123] Furthermore, the detection module 703 is further specifically used for:

[0124] In response to detecting the object, determining an area of ​​a bounding box of the object;

[0125] In response to determining that the area is less than or equal to a preset area, a relative distance between the target and the vehicle is determined, and in response to determining that the relative distance is less than or equal to a preset distance, it is determined that the target has invaded the warning area.

[0126] For the convenience of description, the above device is described in terms of functions divided into various modules. Of course, when implementing the present application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0127] The device of the above embodiment is used to implement the corresponding vehicle control method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0128] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the vehicle control method described in any of the above embodiments is implemented.

[0129] Fig.12 A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown, and the device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 in the device.

[0130] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0131] The memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 may store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1020 and are called and executed by the processor 1010.

[0132] The input / output interface 1030 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0133] The communication interface 1040 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0134] The bus 1050 includes a path that transmits information between the various components of the device (eg, the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040).

[0135] It should be noted that, although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0136] The electronic device of the above embodiment is used to implement the corresponding vehicle control method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0137] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the vehicle control method described in any of the above embodiments.

[0138] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0139] The computer instructions stored in the storage medium of the above embodiments are used to enable the computer to execute the vehicle control method described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0140] Based on the same concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a computer program product, including computer program instructions. When the computer program instructions are run on a computer, the computer executes the method described in any of the above embodiments, which has the beneficial effects of the corresponding method embodiments and will not be repeated here.

[0141] It is understandable that before using the technical solutions of each embodiment of the present disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved will be informed to the user in an appropriate manner, and the user's authorization will be obtained.

[0142] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly remind the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can independently choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0143] As an optional but non-limiting implementation, in response to receiving the user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0144] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0145] A person skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application is limited to these examples. In line with the concept of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0146] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present application difficult to understand, the known power supply / ground connection with the integrated circuit (IC) chip and other components may or may not be shown in the provided drawings. In addition, the device can be shown in the form of a block diagram to avoid making the embodiments of the present application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform to be implemented in the embodiments of the present application (that is, these details should be fully within the scope of understanding of those skilled in the art). In the case of elaborating specific details (e.g., circuits) to describe exemplary embodiments of the present application, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details or when these specific details are changed. Therefore, these descriptions should be considered to be illustrative rather than restrictive.

[0147] Although the present application has been described in conjunction with specific embodiments of the present application, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0148] The embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the present application. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the protection scope of the present application.

Claims

1. A vehicle control method, characterized in that: The rearview mirror of the vehicle is provided with a camera for collecting monitoring images outside the vehicle, wherein the monitoring images include a warning area; the method comprises: Obtaining distortion information of the monitoring image, determining a plurality of target areas with different distortion degrees in the monitoring image based on the distortion information, and performing target detection on the monitoring image; In response to detecting the target, determining at least one target area where the target is located; Based on the target area where the target is located, the relative position of the target and the vehicle is determined to determine whether the target invades the warning area.

2. The vehicle control method according to claim 1, characterized in that: In response to determining that the target area is an area in the monitoring image where the proportion of the distorted image area is less than or equal to a preset proportion, determining the relative position of the target and the vehicle based on the target area where the target is located to determine whether the target invades the warning area includes: Determine whether the target has invaded the warning area according to the position of the lower left / right corner of the target bounding box of the target relative to the warning area of ​​the vehicle; or, Determine whether the target invades the warning area according to the positions of the lower left / right corners of the target bounding box and the lower boundary of the target bounding box relative to the warning area of ​​the vehicle; or, Whether the target invades the warning area is determined according to the position of the lower boundary of the target bounding box of the target relative to the warning area of ​​the vehicle.

3. The vehicle control method according to claim 2, characterized in that: The determining whether the target invades the warning area according to the position of the left / right lower corner of the target bounding box of the target and the lower boundary of the target bounding box relative to the warning area of ​​the vehicle comprises: In response to determining that the lower left / right corner of the target bounding box does not invade the warning area of ​​the vehicle, determining whether the lower boundary of the target bounding box intersects with the upper boundary and the lower boundary of the warning area at the same time; In response to determining that a lower boundary of the target bounding box intersects both an upper boundary and a lower boundary of the warning area, the target intrusion warning area is determined.

4. The vehicle control method according to claim 2, characterized in that: The monitoring image includes a middle preset area; and determining whether the target invades the warning area according to the position of the lower boundary of the target boundary box of the target relative to the warning area of ​​the vehicle includes: In response to determining that the center point of the target bounding box is in a middle preset area of ​​the monitoring image, determining a folding state of the vehicle rearview mirror when collecting the monitoring image outside the vehicle; In response to determining that the rearview mirror is in a non-folded state, determining whether a central segment of a lower boundary of the target bounding box is in a warning area of ​​the vehicle; In response to determining that a lower boundary center segment of the target bounding box is within the warning area of ​​the vehicle, the target intrusion warning area is determined.

5. The vehicle control method according to claim 2, characterized in that: The monitoring image includes a left preset area and a right preset area; and determining whether the target invades the warning area according to the position of the left / right lower corner of the target boundary box of the target relative to the warning area of ​​the vehicle includes: In response to determining that the center point of the target bounding box is in a preset area on the left side of the monitoring image, determining whether a lower right corner of the target bounding box is in a warning area of ​​the vehicle; In response to determining that the lower right corner of the target bounding box is in the warning area of ​​the vehicle, the target intrusion warning area is determined.

6. The vehicle control method according to claim 2, characterized in that: The monitoring image includes a left preset area and a right preset area; and determining whether the target invades the warning area according to the position of the left / right lower corner of the target boundary box of the target relative to the warning area of ​​the vehicle includes: In response to determining that the center point of the target bounding box is in a preset area on the right side of the monitoring image, determining whether the lower left corner of the target bounding box is in a warning area of ​​the vehicle; In response to determining that the lower left corner of the target bounding box is in the warning area of ​​the vehicle, the target intrusion warning area is determined.

7. The vehicle control method according to claim 2, characterized in that: In response to determining that the target area is an area in the monitoring image where the proportion of the distorted image area is greater than a preset proportion, determining the relative position of the target and the vehicle based on the target area where the target is located to determine whether the target invades the warning area includes: Determine the relative distance between the target and the vehicle; In response to determining that the relative distance is less than or equal to a preset distance, it is determined that the target has invaded the warning area.

8. The vehicle control method according to claim 1, characterized in that: The method further comprises: In response to detecting the object, determining an area of ​​a bounding box of the object; In response to determining that the area is less than or equal to a preset area, a relative distance between the target and the vehicle is determined, and in response to determining that the relative distance is less than or equal to a preset distance, it is determined that the target has invaded the warning area.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 8 is implemented.

10. A vehicle, characterized in that: The vehicle comprises the electronic device as claimed in claim 9.

Images