Vehicle-based abnormal behavior processing method and device, electronic equipment and medium

Abstract

The embodiment of the invention provides a vehicle-based abnormal behavior processing method and device, electronic equipment and a medium. The method comprises the following steps: acquiring first image data acquired by a vehicle-mounted camera; controlling a vehicle-mounted camera to collect second image data for the person object when it is detected that the first image data contains image features indicating that the person object has an abnormal behavior; and when it is detected that the second image data contains the image features indicating that the figure object has the same abnormal behavior, carrying out abnormal behavior alarm. Through the embodiment of the invention, the problem that a party cannot give an alarm due to various reasons in an emergency can be avoided, the concealment, timeliness and convenience of the alarm are improved, and the travel safety of a driver or a pedestrian is further guaranteed.

Description

Technical Field

[0001] The present invention relates to the field of vehicles, and in particular to a method, device, electronic equipment and medium for processing abnormal behaviors based on vehicles. Background Art

[0002] In today's society, with the rapid development of the economy and the continuous improvement of people's living standards, cars have become an important tool for people's daily travel. The acceleration of urbanization has brought about the densification of transportation networks and the surge in traffic volume, which has not only improved people's travel efficiency, but also brought many traffic safety challenges. Vehicle safety has become the focus of social attention, which is related to the safety of life and property of every road user, as well as the harmony and stability of the entire society.

[0003] With the densification of transportation networks and the surge in traffic volume, the potential risks faced by drivers and pedestrians in their daily travel have also increased. In the event of an emergency, such as a traffic accident, a health crisis, or a criminal threat, it is often necessary for the parties involved to be able to send out a distress signal quickly and effectively.

[0004] However, traditional alarm methods, such as manually dialing a phone or shouting loudly, may be difficult to implement in an emergency due to various reasons. Moreover, traditional alarm methods also have certain limitations and cannot convey emergency information covertly, accurately and intuitively. Summary of the invention

[0005] In view of the above problems, a method, device, electronic device and medium for handling abnormal behavior based on a vehicle are proposed to overcome the above problems or at least partially solve the above problems, including:

[0006] A vehicle-based abnormal behavior processing method, the method comprising:

[0007] Acquire first image data collected by the vehicle-mounted camera;

[0008] When it is detected that the first image data contains an image feature indicating that the person object has abnormal behavior, controlling the vehicle-mounted camera to collect second image data for the person object;

[0009] When it is detected that the second image data contains image features indicating that the human object exhibits the same abnormal behavior, an abnormal behavior alarm is issued.

[0010] Optionally, when it is detected that the first image data contains image features indicating that a human object has exhibited abnormal behavior, the on-board camera is controlled to collect second image data for the human object, including: when it is detected that the first image data contains image features indicating that a human object has exhibited abnormal behavior, the angle of the on-board camera is adjusted, and the on-board camera is controlled to focus on the human object to collect the second image data for the human object.

[0011] Optionally, adjust the angle of the vehicle camera, including:

[0012] Determining feature points for characterizing abnormal behavior of a human object from the first image data by using an image feature detection algorithm;

[0013] Determine the pixel coordinate information of the feature point by using the image feature detection algorithm, convert the pixel coordinate information of the feature point into world coordinate information according to a preset pixel coordinate to world coordinate conversion relationship, and output the pixel coordinate information and the world coordinate information;

[0014] Use the gradient direction histogram to calculate and output the direction information of the feature points;

[0015] Calculate the calibration parameters of the vehicle camera according to the pixel coordinate information, world coordinate information and direction information of the feature points, where the calibration parameters include distortion coefficient, focal length, principal point and rotation matrix;

[0016] Adjust the angle of the vehicle camera according to the rotation matrix. The rotation matrix is ​​used to describe the rotation angle of the vehicle camera coordinate system relative to the world coordinate system.

[0017] Optionally, determining feature points for characterizing abnormal behavior of a person object from the first image data by using an image feature detection algorithm includes:

[0018] The brightness change value of each pixel in the continuous frame image data is detected by an image feature detection algorithm; when the brightness change value of the pixel in the continuous frame image data is greater than a preset value, the pixel whose brightness change value is greater than the preset value is determined as a feature point.

[0019] Optionally, when it is detected that the second image data contains an image feature indicating that the person object has the same abnormal behavior, before issuing an abnormal behavior alarm, the method further includes:

[0020] Using focal length and principal point, construct the internal parameter matrix;

[0021] Calculating a correction map according to the distortion coefficient and the intrinsic parameter matrix, the correction map is used to convert the pixel coordinates of the pixel points in the second image data into corrected pixel coordinates;

[0022] The second image data is corrected using the correction map.

[0023] Optionally, in the case where it is detected that the second image data contains an image feature indicating that the human object has the same abnormal behavior, an abnormal behavior alarm is issued, including:

[0024] In the case where it is detected that the second image data contains an image feature indicating that the human object exhibits the same abnormal behavior, determining a behavior type of the abnormal behavior;

[0025] An alarm is issued according to the preset alarm method corresponding to the behavior type.

[0026] Optionally, the abnormal behavior includes abnormal gesture behavior, and the abnormal gesture behavior includes multiple abnormal gesture behavior types, and each abnormal gesture behavior type corresponds to a preset alarm method.

[0027] Optionally, the vehicle-mounted camera includes any one or more of the following: an electronic rearview camera, a driver monitoring camera, and an intelligent driving system camera.

[0028] A vehicle-based abnormal behavior processing device, the device comprising:

[0029] A first image data acquisition module, used to acquire first image data collected by the vehicle-mounted camera;

[0030] The vehicle camera control module is used to control the vehicle camera to collect second image data for the person object when it is detected that the first image data contains image features indicating that the person object has abnormal behavior;

[0031] The alarm module is used to issue an abnormal behavior alarm when it is detected that the second image data contains image features indicating that the human object has the same abnormal behavior.

[0032] An electronic device includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor. When the computer program is executed by the processor, the steps of the above-mentioned method for abnormal behavior processing based on a vehicle are implemented.

[0033] A computer-readable storage medium stores a computer program, which implements the steps of the above-mentioned method for abnormal behavior processing based on a vehicle when the computer program is executed by a processor.

[0034] In the embodiment of the present invention, by acquiring the first image data collected by the vehicle camera; in the case where it is detected that the first image data contains image features indicating that the person object has abnormal behavior, the vehicle camera is controlled to collect second image data for the person object; in the case where it is detected that the second image data contains image features indicating that the person object has the same abnormal behavior, an abnormal behavior alarm is issued. This can avoid the problem that the parties are unable to report an emergency due to various reasons, improve the concealment, timeliness and convenience of the alarm, and thus ensure the travel safety of the driver or pedestrian. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solution of the present invention, the accompanying drawings required for use in the description of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0036] Figure 1 is a flowchart of a method for handling abnormal behavior of a vehicle provided by an embodiment of the present invention;

[0037] Figure 2 is a schematic diagram of abnormal behavior processing based on a vehicle provided by an embodiment of the present invention;

[0038] Figure 3 It is a schematic diagram of the structure of a device for handling abnormal behavior based on a vehicle provided in one embodiment of the present invention. DETAILED DESCRIPTION

[0039] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] Reference Figure 1 , shows a flowchart of a method for handling abnormal behavior based on a vehicle provided by an embodiment of the present invention, which may specifically include the following steps:

[0041] Step 101: Acquire first image data collected by a vehicle-mounted camera.

[0042] In some embodiments of the present invention, the vehicle-mounted camera includes any one or more of the following: an electronic rearview camera, a driver monitoring camera, and an intelligent driving system camera.

[0043] The electronic rear view camera may include: a left outer electronic rear view camera and a right outer electronic rear view camera. The intelligent driving system camera may include: a front view camera, a rear view camera and a 360-degree surround view camera.

[0044] In practical applications, the installation location of the above-mentioned vehicle-mounted camera needs to meet at least the following conditions:

[0045] 1) For driver monitoring cameras: It is necessary to ensure that the installation position of the driver monitoring camera does not affect the driver monitoring camera's capture of various behaviors made by the driver, such as making a fist with one hand, covering the mouth and nose with one hand, etc.;

[0046] 2) For electronic rearview cameras: electronic rearview cameras are installed in the left and right electronic rearview mirrors outside the vehicle;

[0047] 3) For the intelligent driving system camera: It is necessary to ensure that the installation positions of the intelligent driving system camera and the electronic rearview camera should be coordinated with each other to ensure that they can cooperate with the electronic rearview camera to take all-round pictures of the environment within a certain range around the vehicle.

[0048] In some embodiments of the present invention, when the first image data is collected through a driver monitoring camera, the first image data may be vehicle interior image data; when the first image data is collected through an electronic rearview camera and an intelligent driving system camera, the first image data may be vehicle exterior image data.

[0049] In some embodiments of the present invention, in addition to using a camera to collect image data, a sound sensor may be installed in the left and right electronic rearview mirrors of the vehicle to collect sound signals.

[0050] Step 102 : When it is detected that the first image data contains image features indicating that the person object has abnormal behavior, the vehicle-mounted camera is controlled to collect second image data for the person object.

[0051] In some embodiments of the present invention, the image data may also be processed using a vehicle-mounted camera.

[0052] Specifically, processing the image data using the vehicle-mounted camera may include:

[0053] Sub-step 11, after acquiring the image data collected by the vehicle-mounted camera, pre-processing the acquired image data.

[0054] In practical applications, the image data collected by the vehicle-mounted camera is often disturbed by various noises, such as noise caused by the device itself or environmental factors. At the same time, uneven lighting conditions may also cause the image to be partially too dark or too bright, and these factors may have an adverse effect on the analysis results of the subsequent image data. Therefore, in order to ensure the accuracy and reliability of the analysis, after obtaining the image data collected by the vehicle-mounted camera, the embodiment of the present invention can pre-process the collected image data.

[0055] In some embodiments of the present invention, preprocessing the collected image data may include at least one of the following: removing noise, adjusting brightness and contrast. Of course, the above-mentioned preprocessing method for the collected first image data is only exemplary, and the embodiment of the present invention does not limit this.

[0056] Sub-step 12, extracting target feature data from the preprocessed image data, and sorting the target feature data in a specific order to form a feature sequence.

[0057] In some embodiments of the present invention, the extracted target feature data may be sorted according to the behavior logic order to form a feature sequence. For example, the extracted target feature data may be sorted according to the behavior logic order of "normal walking - shaky gait - fainting". Other orders may also be used for sorting, such as time order, which is not limited in the embodiments of the present invention.

[0058] Sub-step 13, mapping the feature sequence to the probability model of the behavior, and establishing the behavior model.

[0059] In some embodiments of the present invention, a suitable probability model may be selected according to actual needs to establish a behavior model, such as a hidden Markov model, a Bayesian network, a conditional random field, etc., which is not limited in the embodiments of the present invention.

[0060] Sub-step 14, using the vehicle-mounted camera to collect first image data, and extracting target behavior features from the first image data.

[0061] In some embodiments of the present invention, a vehicle-mounted camera can be used to perform real-time video tracking of a person object and collect first image data. The collected first image data is preprocessed to reduce noise interference and improve image quality. Then, according to actual needs, a suitable feature extraction method is used to extract target behavior features of the person object from the first image data. These target behavior features can be used to analyze the current behavior of the person object and can also be used to iteratively optimize the above-mentioned behavior model. Among them, the target behavior features may include: gait shaking, fist clenching, covering the mouth and nose and other behavioral features, which are not limited in the embodiments of the present invention.

[0062] In some embodiments of the present invention, statistical methods may be used to analyze the extracted target behavior features.

[0063] Sub-step 15, inputting the target behavior characteristics into the behavior model to determine the target behavior type.

[0064] In some embodiments of the present invention, the behavior model can be used to determine the type of the target behavior, for example, it can be determined whether the current state of the person object is standing, walking, or other. Further, it can also include: comparing the determined target behavior type with the preset abnormal behavior type to determine whether the target behavior is abnormal, that is, determining whether the first image data contains image features indicating that the person object has abnormal behavior.

[0065] In some embodiments of the present invention, a sound sensor may be used to analyze the sound signal collected by the sound sensor.

[0066] Specifically, the following steps may be included:

[0067] Sub-step 21, using a sound sensor to collect sound signals from the surrounding environment and converting them into digital signals.

[0068] In actual applications, the driving environment of vehicles is changeable, and there may be extreme situations where all on-board cameras cannot capture images. For example, there is a cry for help behind an obstacle, but the on-board camera cannot capture images through the obstacle. At this time, the sound sensor can collect sound signals from the surrounding environment.

[0069] Sub-step 22, performing Fourier transform on the digital signal using a Fourier transform algorithm.

[0070] During the transformation process, the time domain signal can be converted into a frequency domain signal, and the presence of an abnormality can be determined by analyzing the characteristics of the spectrum distribution. Time domain analysis, such as waveform analysis, autocorrelation function, etc., can also be used to determine whether an abnormality exists.

[0071] In some embodiments of the present invention, when it is detected that the first image data contains an image feature indicating that the person object has abnormal behavior, controlling the vehicle-mounted camera to collect second image data for the person object includes:

[0072] When it is detected that the first image data contains image features indicating that the human object has abnormal behavior, the angle of the vehicle-mounted camera is adjusted, and the vehicle-mounted camera is controlled to focus on the human object to collect second image data for the human object.

[0073] In practical applications, when it is detected that the first image data contains image features indicating abnormal behavior of a person object, the vehicle can be controlled to slow down until the vehicle stops completely to ensure that the person object can remain within the shooting range of the vehicle camera for a long time.

[0074] In some embodiments of the present invention, adjusting the angle of the vehicle-mounted camera includes:

[0075] Sub-step 31, determining feature points for characterizing abnormal behavior of a human object from the first image data by using an image feature detection algorithm.

[0076] In practical applications, before extracting feature points from the first image data, the first image data needs to be preprocessed to ensure the accuracy of subsequent calculation of calibration parameters. Specific preprocessing methods may include gray value adjustment, filtering, edge detection, and the like.

[0077] The above-mentioned image feature detection algorithms may include corner detection algorithms, spot detection algorithms, edge detection algorithms, and the like.

[0078] In some embodiments of the present invention, a corner detection algorithm can be used to extract feature points from the first image data. It can be understood that the corner detection algorithm has high computational efficiency. In practical applications, the corner detection algorithm is used to quickly extract feature points that can characterize abnormal behavior of human objects, thereby avoiding the subsequent untimely warning of abnormal behavior.

[0079] In addition, a spot detection algorithm or an edge detection algorithm may be used to extract feature points from the first image data. The above algorithm may be freely selected according to actual conditions, and the embodiment of the present invention does not limit this.

[0080] The embodiment of the present invention may include the following steps of extracting feature points from the first image data using an image feature detection algorithm:

[0081] Detect the brightness change value of each pixel in the continuous frame image data by using an image feature detection algorithm;

[0082] When the brightness change value of a pixel point in the continuous frame image data is greater than a preset value, the pixel point whose brightness change value is greater than the preset value is determined as a feature point.

[0083] It can be understood that if the brightness change of a pixel in consecutive frames of an image is greater than a preset value, it means that an abnormality may have occurred in the image area near the pixel, that is, the person object has abnormal behavior.

[0084] Sub-step 32, using an image feature detection algorithm to determine the pixel coordinate information of the feature point, converting the pixel coordinate information of the feature point into world coordinate information according to a preset pixel coordinate to world coordinate conversion relationship, and outputting the pixel coordinate information and the world coordinate information.

[0085] In the embodiment of the present invention, the coordinates of the feature point include the pixel position of the feature point in the image, ie, the pixel coordinates, and the real position of the feature point in the world coordinate system, ie, the world coordinates.

[0086] The above pixel coordinates can be directly obtained from the detection results output by the detection algorithm. For example, when extracting feature points from the first image data using a corner detection algorithm, a coordinate system can be first established based on any pixel in the image, and then the pixel coordinates of each feature point can be directly output using a corner detector.

[0087] The above world coordinates can be directly converted from pixel coordinates to obtain world coordinates through a preset two-dimensional to three-dimensional conversion relationship.

[0088] The above two-dimensional to three-dimensional conversion relationship may be determined using conventional planar target methods, and this embodiment does not impose any limitation on this.

[0089] Sub-step 33, using the gradient direction histogram to calculate and output the direction information of the feature point.

[0090] As an embodiment of the present invention, the direction information of the feature point can be obtained through a gradient direction histogram, or can be obtained by other methods according to actual needs, which is not limited in the embodiment of the present invention.

[0091] Exemplarily, the embodiment of the present invention may include the following steps to obtain feature point direction information through a gradient direction histogram:

[0092] 1) For each pixel around the above feature point, use the Sobel operator to calculate the gradient value of each pixel in the horizontal and vertical directions;

[0093] 2) Use the following formula to calculate the gradient magnitude and gradient direction of each pixel:

[0094]

[0095] In the above formula, M is the gradient amplitude, D is the gradient direction, G is x is the gradient value in the horizontal direction, G y is the gradient value in the vertical direction;

[0096] 3) Construct the gradient direction histogram in the following way:

[0097] In a window area around the feature point, the gradient direction is quantized into several intervals, and the gradient amplitude of each pixel is weighted and accumulated into the corresponding direction interval according to its direction;

[0098] 4) Find the interval with the largest amplitude in the gradient direction histogram. The direction of this interval is the direction of the feature point.

[0099] Sub-step 34, calculating the calibration parameters of the vehicle-mounted camera according to the pixel coordinate information, world coordinate information and direction information of the feature point, wherein the calibration parameters include distortion coefficient, focal length, principal point and rotation matrix.

[0100] The calibration parameters of the vehicle camera can be used to describe the internal characteristics and external posture of the vehicle camera. They can be divided into two categories: internal parameters and external parameters. Among them, internal parameters can include: focal length, principal point position, distortion coefficient; external parameters can include: rotation matrix, translation vector.

[0101] As an example, existing calibration tools and libraries may be used to simplify the calibration process, such as the camera calibration function in the OpenCV library (OpenSource Computer Vision Library), which may improve the accuracy and robustness of the calibration results.

[0102] Specifically, the "cv2.calibrateCamera() function" provided in OpenCV can be used to input the above-obtained feature point pixel coordinate information and world coordinate information, and the internal parameters and external parameters of the vehicle-mounted camera can be directly calculated and output. In practical applications, other calibration algorithms can also be used to calculate the calibration parameters, and the embodiment of the present invention does not limit this.

[0103] In addition, the directional information of the feature points can be used to improve the robustness of the calibration algorithm to noise. In practical applications, the coordinate information of the feature points may be disturbed due to uncontrollable factors such as illumination changes, while the directional information can eliminate the above interference and serve as a stable reference to ensure that the calibration algorithm can still calculate the calibration parameters more accurately in the presence of noise.

[0104] Sub-step 35, adjusting the angle of the vehicle-mounted camera according to the rotation matrix, where the rotation matrix is ​​used to describe the rotation angle of the vehicle-mounted camera coordinate system relative to the world coordinate system.

[0105] In some embodiments of the present invention, the angle of the vehicle-mounted camera may be adjusted according to the calculated calibration parameters.

[0106] Specifically, the calculated external parameters can be used to adjust the angle of the vehicle-mounted camera, that is, the above rotation matrix can be used to rotate and adjust the angle of the vehicle-mounted camera to ensure that the direction of the vehicle-mounted camera can be aimed at the human object.

[0107] As other embodiments of the present invention, the calculated internal parameters can also be used to adjust the focal length and principal point position of the vehicle-mounted camera. It is understandable that by adjusting the focal length and principal point position of the vehicle-mounted camera, it is possible to ensure that the vehicle-mounted camera can focus on the human object, thereby ensuring the quality of the captured image and avoiding affecting the accuracy of subsequent abnormal behavior detection of the human object.

[0108] Furthermore, the distortion coefficient can be used to correct the distortion of the vehicle-mounted camera, so that the image collected by the vehicle-mounted camera in the subsequent collection process is close to the actual situation.

[0109] It is understandable that the embodiment of the present invention may also utilize the distortion coefficient to optimize the collected first image data to improve the accuracy of detecting whether the first image data contains image features indicating abnormal behavior of the human object.

[0110] In some embodiments of the present invention, the calibration parameters may be calculated multiple times until the vehicle-mounted camera is adjusted to the optimal shooting angle, wherein the calibration parameters may be calculated according to a predetermined period, and this embodiment does not limit the calculation period.

[0111] After the step of adjusting the angle of the vehicle-mounted camera according to the calibration parameters is completed, the vehicle-mounted camera can be controlled to collect the second image data for the person object.

[0112] Step 103: When it is detected that the second image data contains image features indicating that the human object has the same abnormal behavior, an abnormal behavior alarm is issued.

[0113] It can be understood that the present invention can further confirm whether the person object has abnormal behavior by determining whether the image features indicating abnormal behavior of the person object contained in the second image data are consistent with the image features indicating abnormal behavior of the person object contained in the first image data, thereby avoiding false alarms.

[0114] In some embodiments of the present invention, when it is detected that the second image data contains image features indicating that a human object exhibits the same abnormal behavior, before an abnormal behavior alarm is issued, it also includes: constructing an internal parameter matrix using focal length and principal point; calculating a correction map based on the distortion coefficient and the internal parameter matrix, the correction map being used to convert pixel coordinates of pixel points in the second image data into corrected pixel coordinates; and correcting the second image data using the correction map.

[0115] In a specific application, the embodiment of the present invention may also utilize the calibration parameters obtained by the above calculation to optimize the second image data collected by the vehicle-mounted camera to improve the accuracy of detecting whether the second image data contains image features indicating that a human object exhibits the same abnormal behavior.

[0116] As an embodiment of the present invention, the internal parameter matrix may be in the following form:

[0117]

[0118] Among them, Matrix is ​​the internal parameter matrix, f x and f y are the focal lengths of the on-board camera in the x and y directions, c x and c y are the principal points of the image plane.

[0119] In an embodiment of the present invention, the correction map may be calculated by the following method:

[0120] After constructing the internal parameter matrix, a new internal parameter matrix of the vehicle-mounted camera is redefined. The new internal parameter matrix can be the same as the originally constructed internal parameter matrix, or can be adjusted according to actual needs, which is not limited in the embodiment of the present invention.

[0121] The calculated distortion coefficients, the originally constructed intrinsic parameter matrix, and the redefined new intrinsic parameter matrix are input into the "cv::initUndistortRectifyMap function" provided in OpenCV to directly calculate and output the output correction map.

[0122] In some embodiments of the present invention, when it is detected that the second image data contains image features indicating that the human object has the same abnormal behavior, an abnormal behavior alarm is issued, including:

[0123] Sub-step 41 , when it is detected that the second image data contains image features indicating that the person object exhibits the same abnormal behavior, determining the behavior type of the abnormal behavior.

[0124] In some embodiments of the present invention, a method similar to sub-step 14 and sub-step 15 may be used to determine whether the second image data contains image features indicating abnormal behavior of the human object. If so, it is further determined whether the image features are consistent with the image features indicating abnormal behavior of the human object contained in the first image data, and if they are consistent, the behavior type of the abnormal behavior is determined.

[0125] Sub-step 42, issuing an alarm according to a preset alarm method corresponding to the behavior type.

[0126] In some embodiments of the present invention, each behavior type is provided with a corresponding alarm method, which can ensure that in the event of an emergency, such as fainting, robbery, etc., the most reasonable alarm method can be used in time to warn and avoid further expansion of the harm.

[0127] In some embodiments of the present invention, the abnormal behavior includes abnormal gesture behavior, and the abnormal gesture behavior includes a plurality of abnormal gesture behavior types, and each abnormal gesture behavior type corresponds to a preset alarm method.

[0128] Exemplarily, the abnormal gesture behavior in the present invention may include at least one of the following:

[0129] Abnormal gesture behavior 1: One hand clenched into a fist, indicating that the driver or passenger needs to be rushed to the hospital for emergency treatment.

[0130] Abnormal gesture behavior 2: Covering the mouth and nose with one hand indicates that the driver or passenger’s personal safety is in danger.

[0131] Abnormal gesture behavior 3: Waving a fist left and right with one hand indicates that the driver is in a dangerous situation while driving and needs to seek help from other vehicles or pedestrians.

[0132] It is understandable that, in practical applications, the embodiments of the present invention may also add abnormal gesture behaviors through OTA (Over-The-Air Technology) and other methods.

[0133] like Figure 2 As shown, when the vehicle-mounted camera includes an electronic rearview camera, a driver monitoring camera, and an intelligent driving system camera, at least one of the following methods can be used for warning.

[0134] Method 1: The electronic rearview camera collects abnormal gesture behavior images and transmits them to the electronic rearview mirror controller. After receiving the abnormal gesture behavior images, the electronic rearview mirror controller immediately performs analysis and recognition. After recognizing the abnormal gesture behavior, the abnormal gesture behavior information is immediately sent to the GW (Gateway) through 1000M Ethernet. After receiving the abnormal gesture behavior information, the GW immediately transparently transmits it to the T-BOX (Telematics Box, vehicle-mounted wireless terminal). After receiving the abnormal gesture behavior information, the T-BOX immediately makes a corresponding phone call. If it is abnormal gesture behavior 1, the T-BOX first calls the emergency contact three times, each call lasts 15 seconds. If the emergency contact is not connected, the T-BOX will directly dial 120; if it is abnormal gesture behavior 2, the T-BOX first calls the emergency contact three times, each call lasts 15 seconds. If the emergency contact is not connected, the T-BOX will directly dial 110; if it is abnormal gesture behavior 3, the T-BOX will directly initiate a call for help and call the service center.

[0135] Method 2: The driver monitoring camera collects abnormal gesture behavior images and transmits them to IVI (In-Vehicle Infotainment). After receiving the abnormal gesture behavior images, IVI immediately analyzes and identifies them. After identifying the abnormal gesture behavior, it immediately transmits the abnormal gesture behavior information to GW (Gateway) through 1000M Ethernet. After receiving the abnormal gesture behavior information, GW immediately transparently transmits it to T-BOX (Telematics Box). After receiving the abnormal gesture behavior information, T-BOX immediately makes a corresponding phone call. If it is abnormal gesture behavior 1, T-BOX first calls the emergency contact three times, each call lasts 15 seconds. If the emergency contact is not connected, T-BOX will directly call 120; if it is abnormal gesture behavior 2, T-BOX first calls the emergency contact three times, each call lasts 15 seconds. If the emergency contact is not connected, T-BOX will directly call 110; if it is abnormal gesture behavior 3, T-BOX will directly initiate a call for help and call the service center.

[0136] Method 3: The camera of the intelligent driving system collects abnormal gesture behavior images and transmits them to ADAS (Advanced Driver Assistance Systems). After receiving the abnormal gesture behavior images, ADAS immediately performs analysis and recognition. After recognizing the abnormal gesture behavior, it immediately transmits the abnormal gesture behavior information to GW (Gateway) through 1000M Ethernet. After receiving the abnormal gesture behavior information, GW immediately transparently transmits it to T-BOX (Telematics Box, in-vehicle wireless terminal). After receiving the abnormal gesture behavior information, T-BOX immediately makes a corresponding phone call. If it is abnormal gesture behavior 1, T-BOX first calls the emergency contact three times, each call lasts 15 seconds. If the emergency contact is not connected, T-BOX will directly call 120; if it is abnormal gesture behavior 2, T-BOX first calls the emergency contact three times, each call lasts 15 seconds. If the emergency contact is not connected, T-BOX will directly call 110; if it is abnormal gesture behavior 3, T-BOX will directly initiate a call for help and call the service center.

[0137] It should be noted that the abnormal behavior in the embodiment of the present invention is not limited to abnormal gesture behavior, and the alarm method is not limited to the above method. This is only an exemplary description.

[0138] In some embodiments of the present invention, the first image data collected by the vehicle camera is obtained; when it is detected that the first image data contains image features indicating that the person object has abnormal behavior, the vehicle camera is controlled to collect second image data for the person object; when it is detected that the second image data contains image features indicating that the person object has the same abnormal behavior, an abnormal behavior alarm is issued. This can avoid the problem that the parties are unable to report an emergency due to various reasons, improve the concealment, timeliness and convenience of the alarm, and thus ensure the travel safety of the driver or pedestrian.

[0139] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0140] Reference Figure 3 , shows a schematic diagram of the structure of a device for abnormal behavior processing based on a vehicle provided by an embodiment of the present invention, which may specifically include the following modules:

[0141] The first image data acquisition module 301 is used to acquire first image data collected by the vehicle-mounted camera;

[0142] The vehicle camera control module 302 is used to control the vehicle camera to collect second image data for the person object when it is detected that the first image data contains image features indicating that the person object has abnormal behavior;

[0143] The alarm module 303 is used to issue an abnormal behavior alarm when it is detected that the second image data contains image features indicating that the human object has the same abnormal behavior.

[0144] Optionally, the vehicle-mounted camera control module 302 includes:

[0145] The camera adjustment submodule is used to adjust the angle of the vehicle-mounted camera and control the vehicle-mounted camera to focus on the human object when it is detected that the first image data contains image features indicating abnormal behavior of the human object, so as to collect second image data for the human object.

[0146] Optionally, the camera adjustment submodule includes:

[0147] An angle adjustment unit is used to determine feature points used to characterize abnormal behavior of a human object from first image data through an image feature detection algorithm; determine pixel coordinate information of the feature points using the image feature detection algorithm, convert the pixel coordinate information of the feature points into world coordinate information according to a preset pixel coordinate to world coordinate conversion relationship, and output the pixel coordinate information and the world coordinate information; calculate and output the direction information of the feature points using a gradient direction histogram; calculate the calibration parameters of the vehicle-mounted camera based on the pixel coordinate information, world coordinate information and direction information of the feature points, wherein the calibration parameters include distortion coefficient, focal length, principal point and rotation matrix; adjust the angle of the vehicle-mounted camera according to the rotation matrix, and the rotation matrix is ​​used to describe the rotation angle of the vehicle-mounted camera coordinate system relative to the world coordinate system.

[0148] Optionally, the angle adjustment unit comprises:

[0149] The feature point determination subunit is used to detect the brightness change value of each pixel in the continuous frame image data through the image feature detection algorithm; when the brightness change value of the pixel in the continuous frame image data is greater than the preset value, the pixel whose brightness change value is greater than the preset value is determined as a feature point.

[0150] Optionally, before issuing an abnormal behavior alarm when it is detected that the second image data contains an image feature indicating that the person object has the same abnormal behavior, the method further includes:

[0151] The second image data correction submodule is used to construct an internal parameter matrix using focal length and principal point; calculate a correction map based on distortion coefficients and the internal parameter matrix, the correction map is used to convert pixel coordinates of pixel points in the second image data into corrected pixel coordinates; and correct the second image data using the correction map.

[0152] Optionally, the alarm module 303 includes:

[0153] The behavior type judgment submodule is used to judge the behavior type of abnormal behavior when it is detected that the second image data contains image features indicating that the human object has the same abnormal behavior; and to issue an alarm according to a preset alarm method corresponding to the behavior type.

[0154] Optionally, the abnormal behavior includes abnormal gesture behavior, and the abnormal gesture behavior includes multiple abnormal gesture behavior types, and each abnormal gesture behavior type corresponds to a preset alarm method.

[0155] Optionally, the vehicle-mounted camera includes any one or more of the following: an electronic rearview camera, a driver monitoring camera, and an intelligent driving system camera.

[0156] An embodiment of the present invention further provides an electronic device, which may include a processor, a memory, and a computer program stored in the memory and capable of running on the processor. When the computer program is executed by the processor, the steps of the above method for abnormal behavior processing based on the vehicle are implemented.

[0157] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above method for handling abnormal behavior based on a vehicle are implemented.

[0158] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0159] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0160] Those skilled in the art will appreciate that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0161] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0162] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0163] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable terminal device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0164] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0165] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0166] The above provides a detailed introduction to the vehicle-based abnormal behavior processing method, device, electronic device and medium. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for a general technician in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A vehicle-based abnormal behavior processing method, characterized in that: The method comprises: Acquire first image data collected by the vehicle-mounted camera; When it is detected that the first image data contains an image feature indicating that a person object has abnormal behavior, controlling the vehicle-mounted camera to collect second image data for the person object; When it is detected that the second image data contains image features indicating that the human object exhibits the same abnormal behavior, an abnormal behavior alarm is issued.

2. The method according to claim 1, characterized in that When it is detected that the first image data contains an image feature indicating that the person object has abnormal behavior, controlling the vehicle-mounted camera to collect second image data for the person object includes: When it is detected that the first image data contains image features indicating abnormal behavior of a human object, the angle of the vehicle-mounted camera is adjusted, and the vehicle-mounted camera is controlled to focus on the human object to collect second image data for the human object.

3. The method according to claim 2, characterized in that The adjusting the angle of the vehicle-mounted camera includes: Determining feature points for characterizing abnormal behavior of the human object from the first image data by using an image feature detection algorithm; Determine the pixel coordinate information of the feature point by using the image feature detection algorithm, convert the pixel coordinate information of the feature point into world coordinate information according to a preset pixel coordinate to world coordinate conversion relationship, and output the pixel coordinate information and the world coordinate information; Calculating and outputting the direction information of the feature point using a gradient direction histogram; Calculating calibration parameters of the vehicle-mounted camera according to the pixel coordinate information, the world coordinate information and the direction information of the feature point, wherein the calibration parameters include distortion coefficient, focal length, principal point and rotation matrix; The angle of the vehicle-mounted camera is adjusted according to the rotation matrix, where the rotation matrix is ​​used to describe the rotation angle of the vehicle-mounted camera coordinate system relative to the world coordinate system.

4. The method according to claim 3, characterized in that The determining, from the first image data, feature points used to characterize abnormal behavior of the person object by using an image feature detection algorithm includes: The image feature detection algorithm is used to detect the brightness change value of each pixel in the continuous frame image data; when the brightness change value of the pixel in the continuous frame image data is greater than a preset value, the pixel whose brightness change value is greater than the preset value is determined as a feature point.

5. The method according to claim 4, characterized in that Before issuing an abnormal behavior alarm when it is detected that the second image data contains an image feature indicating that the person object has the same abnormal behavior, the method further includes: Using the focal length and the principal point, construct an internal parameter matrix; Calculating a correction map according to the distortion coefficient and the intrinsic parameter matrix, wherein the correction map is used to convert pixel coordinates of pixel points in the second image data into corrected pixel coordinates; The second image data is corrected using the correction map.

6. The method according to any one of claims 1 to 5, characterized in that: The step of issuing an abnormal behavior alarm when it is detected that the second image data contains an image feature indicating that the person object has the same abnormal behavior includes: In the case where it is detected that the second image data contains an image feature indicating that the human object exhibits the same abnormal behavior, determining a behavior type of the abnormal behavior; An alarm is issued according to a preset alarm method corresponding to the behavior type.

7. The method according to claim 6, characterized in that The abnormal behavior includes abnormal gesture behavior, and the abnormal gesture behavior includes a plurality of abnormal gesture behavior types, and each abnormal gesture behavior type corresponds to a preset alarm mode.

8. The method according to claim 1, characterized in that The vehicle-mounted camera includes any one or more of the following: an electronic rearview camera, a driver monitoring camera, and an intelligent driving system camera.

9. A vehicle-based abnormal behavior processing device, characterized in that: The device comprises: A first image data acquisition module, used to acquire first image data collected by the vehicle-mounted camera; The vehicle-mounted camera control module is used to control the vehicle-mounted camera to collect second image data for the human object when it is detected that the first image data contains image features indicating that the human object has abnormal behavior; The alarm module is used to issue an abnormal behavior alarm when it is detected that the second image data contains image features indicating that the human object has the same abnormal behavior.

10. An electronic device, characterized in that: The invention comprises a processor, a memory and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the steps of the method for abnormal behavior processing based on a vehicle as claimed in any one of claims 1 to 8 are implemented.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for abnormal behavior processing based on a vehicle as claimed in any one of claims 1 to 8 are implemented.

Images