A method and system for identifying abnormal behavior state of a vehicle

Abstract

The present invention discloses a method and system for identifying abnormal behavior states of vehicles, which relates to the field of intelligent parking management, including: using an existing vehicle detection and recognition model in combination with a trainable vehicle state recognition model and integrating recognition result information to realize the recognition and monitoring of abnormal vehicle states, and when confirming the existence of abnormal vehicles by comparing the door opening probability and / or trunk opening probability in the vehicle state recognition vector with a preset state judgment threshold, feedback of abnormal vehicle information is provided. The difficulty of implementing the algorithm can be reduced, and because the present invention is independent of the vehicle detection and recognition model, the vehicle state recognition model can be independently deployed according to actual needs, ensuring that the entire vehicle abnormal behavior state recognition algorithm is efficient, stable and controllable; at the same time, because the multi-classification vehicle state recognition model used in the present invention does not require positioning information, it also reduces the labeling requirements of the target refinement area, greatly reducing the difficulty of data labeling.

Description

Technical Field

[0001] The present invention relates to the field of intelligent parking management, and in particular to a method and system for identifying abnormal behavior states of vehicles. Background Art

[0002] In dynamic and static traffic monitoring scenarios based on high-position video, capturing multi-dimensional information on the status of vehicles in the visible area and correlating and integrating multiple information on vehicles and surrounding related targets can provide necessary data support for abnormal vehicle behavior in open and complex monitoring scenarios. Among them, the main vehicle behavior states include the opening and closing of doors and trunks in addition to driving and parking: the opening and closing of doors and trunks are related to the passengers' boarding and disembarking behaviors, and the opening of the trunk is also highly correlated with the driver's avoidance of camera monitoring behaviors. Therefore, monitoring the status of doors and trunks plays an important role in identifying illegal behaviors such as dropping off passengers at illegal sections and deliberately avoiding the collection of electronic information by cameras.

[0003] Currently, when recognizing the status of vehicle doors and trunks, a deep learning network based on target detection is usually used. The model training data corresponding to the deep learning network not only needs to judge whether the vehicle doors and trunks are open, but also needs to pre-label the location information of the vehicle doors and trunks. As a result, too much additional labeling information will greatly increase the application cost. In addition, the opening state of the vehicle doors and trunks is represented as a continuous process with morphological similarity. Under the actual multi-view sampling state, using information in too local an area for classification and recognition will largely cause state confusion, which will lead to low recognition accuracy. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides a method and system for identifying abnormal behavior status of a vehicle, which can solve the problems of high cost and low accuracy of existing illegal parking management based on vehicle directed bounding boxes.

[0005] To achieve the above object, on the one hand, the present invention provides a method for identifying abnormal behavior of a vehicle, the method comprising:

[0006] Acquire a real-time collected monitoring area image, and acquire the vehicle detection frame position coordinates from the monitoring area image according to a preset vehicle detection and recognition model;

[0007] Acquire the clipping frame coordinates corresponding to the vehicle state recognition frame according to the vehicle detection frame position coordinates, and process the monitoring area image according to the clipping frame coordinates to obtain a monitoring area image sub-atlas;

[0008] Processing the monitoring area image sub-atlas according to a preset vehicle state recognition model to obtain a vehicle state recognition vector corresponding to the monitoring area image, wherein the preset vehicle state recognition model is used to recognize the probability of a door being opened, the probability of a trunk being opened, and the probability of a person being present around the vehicle, and the vehicle state recognition vector includes the probability of a door being opened and / or the probability of a trunk being opened;

[0009] The door opening probability and / or trunk opening probability in the vehicle state recognition vector are compared with a preset state determination threshold to confirm whether the vehicle has abnormal state behavior.

[0010] Furthermore, before the step of acquiring the real-time collected monitoring area image, the method further includes:

[0011] Obtaining a vehicle detection frame from the monitoring area image stored in the database through the preset vehicle detection and recognition model, and cropping and saving a local target area of ​​the monitoring area image data according to the preset size of the length and width of the vehicle detection frame as a training image data set;

[0012] The training image data set is labeled according to the ternary vectors corresponding to the classification type order to obtain model training data, wherein the categories of the ternary vectors corresponding to the classification type order include door status information, trunk status information, and status information of whether there are people around the vehicle;

[0013] The model training data is input into the preset classification model to obtain a three-component vector Pv=[P_do, P_to, P_per] as the preset vehicle state recognition model, where P_do is the probability of the door being opened, P_to is the probability of the trunk being opened, and P_per is the probability of there being someone near the vehicle.

[0014] Furthermore, the step of acquiring the clipping frame coordinates corresponding to the vehicle state identification frame according to the vehicle detection frame position coordinates includes:

[0015] The length and width of the vehicle detection frame are expanded by preset sizes to obtain the clipping frame coordinates corresponding to the vehicle state recognition frame.

[0016] Furthermore, the step of processing the monitoring area image sub-atlas according to the preset vehicle state recognition model to obtain the vehicle state recognition vector corresponding to the monitoring area image includes:

[0017] The vehicle door status information is obtained according to the formula P_do_new=a1*P_do+a2*P_per, where P_do_new is the door opening probability after considering the presence of people, a1=exp(P_do) / (exp(P_do)+exp(P_per)), a2=exp(P_per) / (exp(P_do)+exp(P_per));

[0018] The vehicle trunk status information is obtained according to the formula P_to_new=P_to*(1-P_per), where P_to_new is the trunk opening probability after considering the presence of a person.

[0019] Furthermore, the step of comparing the door opening probability and / or trunk opening probability in the vehicle state recognition vector with a preset state determination threshold to confirm whether the vehicle has abnormal state behavior includes:

[0020] Obtain the vehicle state indication vector state=[S_d, S_t] according to the preset state determination threshold thr=[thr_do, thr_to] and the vehicle state identification vector Pv_new=[P_do_new, P_to_new], wherein S_d=(P_do_new>thr_do), S_t=(P_to_new>thr_to);

[0021] If the value of the vehicle state indication vector is 1, it is confirmed that the vehicle has abnormal state behavior;

[0022] Output warning information carrying the vehicle status information and the vehicle target detection and recognition result.

[0023] On the other hand, the present invention provides a system for identifying abnormal behavior of a vehicle, the system comprising: an acquisition unit, for acquiring a monitoring area image collected in real time, and acquiring the position coordinates of a vehicle detection frame from the monitoring area image according to a preset vehicle detection and recognition model;

[0024] The acquisition unit is further used to acquire the clipping frame coordinates corresponding to the vehicle state recognition frame according to the vehicle detection frame position coordinates, and process the monitoring area image according to the clipping frame coordinates to obtain a monitoring area image sub-atlas;

[0025] a processing unit, configured to process the monitoring area image sub-atlas according to a preset vehicle state recognition model to obtain a vehicle state recognition vector corresponding to the monitoring area image, wherein the preset vehicle state recognition model is used to recognize a probability of a door being opened, a probability of a trunk being opened, and a probability of a person being present around the vehicle, and the vehicle state recognition vector includes a probability of a door being opened and / or a probability of a trunk being opened;

[0026] The determination unit is used to compare the door opening probability and / or trunk opening probability in the vehicle state recognition vector with a preset state determination threshold to confirm whether the vehicle has abnormal state behavior.

[0027] Furthermore, the system further comprises:

[0028] A cropping unit, used to obtain a vehicle detection frame from the monitoring area image stored in the database through the preset vehicle detection and recognition model, and to crop and save a local target area of ​​the monitoring area image data according to the preset size of the length and width of the vehicle detection frame as a training image data set;

[0029] A labeling unit, used to obtain a vehicle detection frame from the monitoring area image stored in the database through the preset vehicle detection and recognition model, and to crop and save a local target area of ​​the monitoring area image data according to the preset size of the length and width of the vehicle detection frame as a training image data set;

[0030] A generating unit is used to input the model training data into a preset classification model to obtain a three-component vector Pv=[P_do, P_to, P_per] as the preset vehicle state recognition model, where P_do is the probability of a door being opened, P_to is the probability of a trunk being opened, and P_per is the probability of someone being present near the vehicle.

[0031] Furthermore, the acquisition unit is specifically used to expand the length and width of the vehicle detection frame by preset sizes to obtain the clipping frame coordinates corresponding to the vehicle state identification frame.

[0032] Further, the processing unit is specifically used to obtain the vehicle door status information according to the formula P_do_new=a1*P_do+a2*P_per, wherein P_do_new is the probability of door opening after considering the presence of people, a1=exp(P_do) / (exp(P_do)+exp(P_per)), a2=exp(P_per) / (exp(P_do)+exp(P_per)); and obtain the vehicle trunk status information according to the formula P_to_new=P_to*(1-P_per), wherein P_to_new is the probability of trunk opening after considering the presence of people.

[0033] Furthermore, the confirmation unit is specifically used to obtain a vehicle state indication vector state = [S_d, S_t] according to a preset state judgment threshold thr = [thr_do, thr_to] and a vehicle state identification vector Pv_new = [P_do_new, P_to_new], wherein S_d = (P_do_new>thr_do), S_t = (P_to_new>thr_to); if the value of the vehicle state indication vector is 1, it is confirmed that the vehicle has abnormal state behavior; and output an alarm message carrying the vehicle state information and the vehicle target detection and identification result.

[0034] The present invention provides a method and system for identifying abnormal behavior states of vehicles. The method and system utilize an existing vehicle detection and identification model in combination with a trainable vehicle state identification model and integrates the identification result information to realize the identification and monitoring of abnormal vehicle states. When the door opening probability and / or trunk opening probability in the vehicle state identification vector are compared with a preset state determination threshold to confirm the existence of an abnormal vehicle, the abnormal vehicle information is fed back. Since the present invention is performed based on an existing vehicle detection and identification model, the difficulty of implementation is reduced. Moreover, since the present invention is independent of the vehicle detection and identification model, the vehicle state identification model can be independently deployed according to actual needs, ensuring that the entire vehicle abnormal behavior state identification algorithm is efficient, stable and controllable. At the same time, since the multi-classification vehicle state identification model used in the present invention does not require positioning information, the labeling requirements of the target refinement area are also reduced, greatly reducing the difficulty of data labeling. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a flow chart of a method for identifying abnormal behavior state of a vehicle provided by the present invention;

[0036] Figure 2 It is a structural schematic diagram of a vehicle abnormal behavior state recognition system provided by the present invention. DETAILED DESCRIPTION

[0037] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments.

[0038] like Figure 1 As shown, a method for identifying abnormal behavior of a vehicle provided by an embodiment of the present invention includes the following steps:

[0039] 101. Obtain a real-time collected monitoring area image, and obtain the position coordinates of a vehicle detection frame from the monitoring area image according to a preset vehicle detection and recognition model.

[0040] For the embodiment of the present invention, before step 101, it may also include: obtaining a vehicle detection frame from the monitoring area image stored in the database through the preset vehicle detection and recognition model, and cropping and saving the local target area of ​​the monitoring area image data according to the preset size of the vehicle detection frame and expanding the preset size according to the length and width of the vehicle detection frame as a training image data set; labeling the training image data set according to the ternary vector corresponding to the classification type order to obtain model training data, and the categories of the ternary vector corresponding to the classification type order include door status information, trunk status information, and status information of whether there are people around the vehicle; inputting the model training data into the preset classification model to obtain a three-component vector Pv=[P_do, P_to, P_per] as the preset vehicle status recognition model, P_do is the probability of the door being opened, P_to is the probability of the trunk being opened, and P_per is the probability of there being people near the vehicle.

[0041] Among them, the preset size can be 125%, 130%, 135% of the original size, etc., which is not limited in the embodiment of the present invention. The step of labeling the training image data set according to the ternary vector corresponding to the classification type order to obtain the model training data can specifically include: labeling the collected training image data set, setting the labeled data to a ternary vector corresponding to the classification type order: according to the vehicle state recognition requirements, the classification types are set to be door open, trunk open, and person present, and the corresponding label vector label = [l1, l2, l3], and then marking according to the actual image content, obtaining the label data set corresponding to the training image data set, and forming a complete model training data D, for example, if the door opening angle in the picture is approximately >45 degrees, then l1 = 1, otherwise l1 = 0; if the trunk opening angle in the picture is approximately >45 degrees, then l2 = 1, otherwise l2 = 0; if there is a person in the area near the vehicle in the picture, then l3 = 1, otherwise l3 = 0.

[0042] 102. Acquire the clipping frame coordinates corresponding to the vehicle state recognition frame according to the vehicle detection frame position coordinates, and process the monitoring area image according to the clipping frame coordinates to obtain a monitoring area image sub-atlas.

[0043] For the embodiment of the present invention, the step of obtaining the coordinates of the clipping frame corresponding to the vehicle state identification frame according to the position coordinates of the vehicle detection frame includes: expanding the length and width of the vehicle detection frame by a preset size to obtain the coordinates of the clipping frame corresponding to the vehicle state identification frame. The preset size can be 125%, 130%, 135%, etc. of the original size.

[0044] For an embodiment of the present invention, the step of processing the monitoring area image according to the cropping frame coordinates to obtain a monitoring area image sub-atlas may specifically include: inputting the real-time collected camera image data into an existing vehicle detection and recognition model, obtaining the vehicle detection frame position coordinates pos0 = (x0, y0, w, h) of the current image, expanding each detection frame area by 30%, obtaining new cropping frame coordinates pos = (x0, y0, w*1.15, h*1.15), and cropping the target area of ​​the original image to obtain the sub-atlas Ip.

[0045] 103. Process the monitoring area image sub-atlas according to a preset vehicle state recognition model to obtain a vehicle state recognition vector corresponding to the monitoring area image.

[0046] The preset vehicle state recognition model is used to identify the probability of a door being opened, the probability of a trunk being opened, and the probability of someone being present around the vehicle, and the vehicle state recognition vector includes the probability of a door being opened and / or the probability of a trunk being opened.

[0047] For the embodiment of the present invention, step 103 may specifically include: obtaining the vehicle door status information according to the formula P_do_new=a1*P_do+a2*P_per, wherein P_do_new is the probability of the door opening after considering the presence of people, a1=exp(P_do) / (exp(P_do)+exp(P_per)), a2=exp(P_per) / (exp(P_do)+exp(P_per)); obtaining the vehicle trunk status information according to the formula P_to_new=P_to*(1-P_per), wherein P_to_new is the probability of the trunk opening after considering the presence of people.

[0048] It should be noted that the sub-atlas Ip is input into the preset vehicle state recognition model to obtain the current image recognition result vector Pv = [P_do, P_to, P_per]. Since the monitoring system has different expected goals for the interactive relationship between people and door opening or trunk opening behaviors, in order to improve the final state recognition accuracy, the recognition probability of human presence has different effects on the other two items. The following information fusion evaluation method can be defined according to different situations to obtain the final vehicle state recognition vector Pv_new = [P_do_new, P_to_new]: When the goal of identifying door opening is mainly to identify whether someone gets on or off the vehicle illegally, the interaction between the person and the door is mostly a short-term behavior that is performed synchronously. Therefore, when the two states exist at the same time, it indicates that the possibility of an abnormal state is higher. Therefore, the recognition of the door opening state behavior by people is positively correlated, and the comprehensive probability of the door opening state is finally expressed as P_do_new = a1*P_do+a2*P_per, a1 = exp(P_do) / (exp(P_do)+exp(P_per)), a2=exp(P_per) / (exp(P_do)+exp(P_per)); when the goal of identifying whether the trunk is open is mainly to identify whether someone deliberately opens the trunk when parking in order to avoid camera surveillance and take pictures, evade payment, etc., the interaction between the person and the trunk is relatively weak. There is a high probability that the person has left during the monitoring period but the trunk is still open. In addition, when a person interacts with the trunk, it will affect the recognition of the trunk open state, or the interaction itself is only a reasonable behavior for a short time, such as taking items from the trunk when parking, which is a normal behavior. Therefore, the person is negatively correlated with the recognition of the trunk open state, and the comprehensive probability of the trunk open state is finally expressed as P_to_new=P_to*(1-P_per).

[0049] 104. Compare the door opening probability and / or trunk opening probability in the vehicle state recognition vector with a preset state determination threshold to confirm whether the vehicle has abnormal state behavior.

[0050] For the embodiment of the present invention, step 104 may specifically include: obtaining a vehicle state indication vector state=[S_d,S_t] according to a preset state judgment threshold thr=[thr_do,thr_to] and a vehicle state identification vector Pv_new=[P_do_new,P_to_new], wherein S_d=(P_do_new>thr_do), S_t=(P_to_new>thr_to); if the value of the vehicle state indication vector is 1, it is confirmed that the vehicle has abnormal state behavior; and outputting an alarm message carrying the vehicle state information and the vehicle target detection and identification result.

[0051] For example, a preset state determination threshold thr=[thr_do, thr_to] is set, and the vehicle state recognition vector Pv_new is further determined to obtain the final vehicle state indication vector state=[S_d, S_t], wherein S_d=(P_do_new>thr_do), S_t=(P_to_new>thr_to), if the vehicle state indication vector state=[0,0], the recognition result is ignored, and the abnormal state monitoring signal is not fed back to the system, if (state==1).any()==True, the license plate recognition information of the current target vehicle in the current frame is retrieved, and the information such as the existing vehicle state indication vector state and the vehicle target detection and recognition results is comprehensively transmitted to the background monitoring system for further subsequent abnormal processing procedures; wherein, if the license plate recognition information that can be obtained in the current frame is missing or the credibility of the recognition information is not high due to target occlusion, poor image quality, etc., the continuous collection of images in the previous and subsequent time periods is used to track the current target, and the license plate information of the corresponding vehicle is captured, thereby improving the accuracy of the collection of abnormal state vehicle information.

[0052] The present invention provides a method for identifying abnormal behavior states of vehicles. It utilizes an existing vehicle detection and identification model in combination with a trainable vehicle state identification model and integrates the identification result information to realize the identification and monitoring of abnormal vehicle states. When the door opening probability and / or trunk opening probability in the vehicle state identification vector are compared with a preset state judgment threshold to confirm the existence of an abnormal vehicle, the abnormal vehicle information is fed back. Since the present invention is performed based on an existing vehicle detection and identification model, the difficulty of implementation is reduced. Moreover, since the present invention is independent of the vehicle detection and identification model, the vehicle state identification model can be independently deployed according to actual needs, ensuring that the entire vehicle abnormal behavior state identification algorithm is efficient, stable and controllable. At the same time, since the multi-classification vehicle state identification model used in the present invention does not require positioning information, the labeling requirements of the target refinement area are also reduced, greatly reducing the difficulty of data labeling.

[0053] To implement the method provided by the embodiment of the present invention, the embodiment of the present invention provides a system for identifying abnormal behavior of a vehicle, such as Figure 2 As shown, the system includes: an acquisition unit 21, a processing unit 22, and a determination unit 23.

[0054] The acquisition unit 21 is used to acquire the monitoring area image collected in real time, and obtain the vehicle detection frame position coordinates from the monitoring area image according to a preset vehicle detection and recognition model.

[0055] The acquisition unit 21 is further configured to acquire the clipping frame coordinates corresponding to the vehicle state recognition frame according to the vehicle detection frame position coordinates, and process the monitoring area image according to the clipping frame coordinates to obtain a monitoring area image sub-atlas.

[0056] The processing unit 22 is used to process the monitoring area image sub-atlas according to a preset vehicle state recognition model to obtain a vehicle state recognition vector corresponding to the monitoring area image.

[0057] The preset vehicle state recognition model is used to identify the probability of a door being opened, the probability of a trunk being opened, and the probability of someone being present around the vehicle, and the vehicle state recognition vector includes the probability of a door being opened and / or the probability of a trunk being opened.

[0058] The determination unit 23 is used to compare the door opening probability and / or trunk opening probability in the vehicle state recognition vector with a preset state determination threshold to confirm whether the vehicle has abnormal state behavior.

[0059] Furthermore, the system also includes: a cropping unit 24, which is used to obtain a vehicle detection frame from the monitoring area image stored in the database through the preset vehicle detection and recognition model, and to crop and save a local target area of ​​the monitoring area image data according to the preset size of the length and width of the vehicle detection frame as a training image data set.

[0060] The preset size may be 125%, 130%, 135% or the like of the original size, which is not limited in the embodiment of the present invention.

[0061] Furthermore, the system also includes: a labeling unit 25, which is used to obtain a vehicle detection frame from the monitoring area image stored in the database through the preset vehicle detection and recognition model, and to expand the preset size according to the length and width of the vehicle detection frame to crop and save the local target area of ​​the monitoring area image data as a training image data set.

[0062] Furthermore, the system also includes: a generating unit 26, which is used to input the model training data into a preset classification model to obtain a three-component vector Pv=[P_do, P_to, P_per] as the preset vehicle state recognition model, where P_do is the probability of a door being opened, P_to is the probability of a trunk being opened, and P_per is the probability of someone being present near the vehicle.

[0063] Furthermore, the acquisition unit 21 is further configured to expand the length and width of the vehicle detection frame by preset sizes to obtain the clipping frame coordinates corresponding to the vehicle state recognition frame.

[0064] Further, the processing unit 22 is specifically used to obtain the vehicle door status information according to the formula P_do_new=a1*P_do+a2*P_per, wherein P_do_new is the probability of door opening after considering the presence of people, a1=exp(P_do) / (exp(P_do)+exp(P_per)), a2=exp(P_per) / (exp(P_do)+exp(P_per)); and obtain the vehicle trunk status information according to the formula P_to_new=P_to*(1-P_per), wherein P_to_new is the probability of trunk opening after considering the presence of people.

[0065] Furthermore, the confirmation unit 23 is specifically used to obtain a vehicle state indication vector state = [S_d, S_t] according to a preset state judgment threshold thr = [thr_do, thr_to] and a vehicle state identification vector Pv_new = [P_do_new, P_to_new], wherein S_d = (P_do_new>thr_do), S_t = (P_to_new>thr_to); if the value of the vehicle state indication vector is 1, it is confirmed that the vehicle has abnormal state behavior; and output an alarm message carrying the vehicle state information and the vehicle target detection and identification result.

[0066] The present invention provides a vehicle abnormal behavior state recognition system, which utilizes an existing vehicle detection and recognition model in combination with a trainable vehicle state recognition model and fusion recognition result information to realize the recognition and monitoring of vehicle abnormal states, and when the presence of an abnormal vehicle is confirmed by comparing the door opening probability and / or trunk opening probability in the vehicle state recognition vector with a preset state judgment threshold, the abnormal vehicle information is fed back. Since the present invention is carried out by an existing vehicle detection and recognition model, the difficulty of implementation is reduced, and since the present invention is independent of the vehicle detection and recognition model, the vehicle state recognition model can be independently deployed according to actual needs, ensuring that the recognition algorithm of the entire vehicle abnormal behavior state is efficient, stable and controllable; at the same time, since the multi-classification vehicle state recognition model used in the present invention does not require positioning information, the labeling requirements of the target refinement area are also reduced, greatly reducing the difficulty of data labeling.

[0067] It should be understood that the specific order or hierarchy of steps in the disclosed process is an example of an exemplary method. Based on design preferences, it should be understood that the specific order or hierarchy of steps in the process can be rearranged without departing from the scope of protection of the present disclosure. The attached method claims present the elements of the various steps in an exemplary order and are not intended to be limited to the specific order or hierarchy described.

[0068] In the above detailed description, various features are grouped together in a single embodiment to simplify the disclosure. This method of disclosure should not be interpreted as reflecting an intention that the embodiments of the claimed subject matter require more features than are clearly stated in each claim. On the contrary, as reflected in the appended claims, the invention is in a state of having less than all the features of the disclosed individual embodiments. Therefore, the appended claims are hereby expressly incorporated into the detailed description, with each claim standing on its own as a separate preferred embodiment of the invention.

[0069] The disclosed embodiments are described above to enable any person skilled in the art to implement or use the present invention. Various modifications of these embodiments are obvious to those skilled in the art, and the general principles defined herein may also be applied to other embodiments without departing from the spirit and scope of the present disclosure. Therefore, the present disclosure is not limited to the embodiments given herein, but is consistent with the broadest scope of the principles and novel features disclosed in this application.

[0070] The above description includes examples of one or more embodiments. Of course, it is impossible to describe all possible combinations of components or methods for the purpose of describing the above embodiments, but it should be recognized by those skilled in the art that the various embodiments may be further combined and arranged. Therefore, the embodiments described herein are intended to cover all such changes, modifications and variations that fall within the scope of protection of the appended claims. In addition, with respect to the term "comprising" used in the specification or claims, the word is covered in a manner similar to the term "including", just as "including," is explained as a transitional word in the claims. In addition, any term "or" used in the specification of the claims is intended to mean "non-exclusive or".

[0071] Those skilled in the art may also understand that the various illustrative logical blocks, units, and steps listed in the embodiments of the present invention may be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly demonstrate the interchangeability of hardware and software, the various illustrative components, units, and steps described above have generally described their functions. Whether such functions are implemented by hardware or software depends on the specific application and the design requirements of the entire system. Those skilled in the art may use various methods to implement the described functions for each specific application, but such implementation should not be understood as exceeding the scope of protection of the embodiments of the present invention.

[0072] The various illustrative logic blocks or units described in the embodiments of the present invention can be implemented or operated by a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field programmable gate array or other programmable logic device, a discrete gate or transistor logic, a discrete hardware component, or any combination of the above. The general-purpose processor can be a microprocessor, and optionally, the general-purpose processor can also be any conventional processor, controller, microcontroller or state machine. The processor can also be implemented by a combination of computing devices, such as a digital signal processor and a microprocessor, a plurality of microprocessors, one or more microprocessors combined with a digital signal processor core, or any other similar configuration.

[0073] The steps of the method or algorithm described in the embodiments of the present invention can be directly embedded in hardware, a software module executed by a processor, or a combination of the two. The software module can be stored in a RAM memory, a flash memory, a ROM memory, an EPROM memory, an EEPROM memory, a register, a hard disk, a removable disk, a CD-ROM, or other storage media of any form in the art. Exemplarily, the storage medium can be connected to the processor so that the processor can read information from the storage medium and can write information to the storage medium. Optionally, the storage medium can also be integrated into the processor. The processor and the storage medium can be arranged in an ASIC, and the ASIC can be arranged in a user terminal. Optionally, the processor and the storage medium can also be arranged in different components in the user terminal.

[0074] In one or more exemplary designs, the above functions described in the embodiments of the present invention can be implemented in hardware, software, firmware or any combination of the three. If implemented in software, these functions can be stored on a computer-readable medium, or transmitted in the form of one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media that facilitate the transfer of computer programs from one place to another. The storage medium can be any available medium that can be accessed by any general or special computer. For example, such computer-readable media can include but are not limited to RAM, ROM, EEPROM, CD-ROM or other optical disk storage, disk storage or other magnetic storage devices, or any other medium that can be used to carry or store program codes in the form of instructions or data structures and other forms that can be read by general or special computers, or general or special processors. In addition, any connection can be appropriately defined as a computer-readable medium, for example, if the software is transmitted from a website site, server or other remote resource through a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wirelessly, such as infrared, wireless and microwave, it is also included in the defined computer-readable medium. The disk and disc include compact disk, laser disk, optical disk, DVD, floppy disk and Blu-ray disk. Disks usually reproduce data magnetically, while discs usually reproduce data optically with lasers. Combinations of the above may also be included in computer readable media.

[0075] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for identifying abnormal behavior of a vehicle, characterized in that: The method comprises: Acquire a real-time collected monitoring area image, and acquire the vehicle detection frame position coordinates from the monitoring area image according to a preset vehicle detection and recognition model; Acquire the clipping frame coordinates corresponding to the vehicle state recognition frame according to the vehicle detection frame position coordinates, and process the monitoring area image according to the clipping frame coordinates to obtain a monitoring area image sub-atlas; Processing the monitoring area image sub-atlas according to a preset vehicle state recognition model to obtain a vehicle state recognition vector corresponding to the monitoring area image, wherein the preset vehicle state recognition model is used to recognize the probability of a door being opened, the probability of a trunk being opened, and the probability of a person being present around the vehicle, and the vehicle state recognition vector includes the probability of a door being opened and / or the probability of a trunk being opened; The door opening probability and / or trunk opening probability in the vehicle state recognition vector are compared with a preset state determination threshold to confirm whether the vehicle has abnormal state behavior.

2. The method for identifying abnormal behavior of a vehicle according to claim 1, characterized in that: Before the step of acquiring the monitored area image collected in real time, the method further includes: Obtaining a vehicle detection frame from the monitoring area image stored in the database through the preset vehicle detection and recognition model, and cropping and saving a local target area of ​​the monitoring area image data according to the preset size of the length and width of the vehicle detection frame as a training image data set; The training image data set is labeled according to the ternary vectors corresponding to the classification type order to obtain model training data, wherein the categories of the ternary vectors corresponding to the classification type order include door status information, trunk status information, and status information of whether there are people around the vehicle; The model training data is input into the preset classification model to obtain a three-component vector Pv=[P_do, P_to, P_per] as the preset vehicle state recognition model, where P_do is the probability of the door being opened, P_to is the probability of the trunk being opened, and P_per is the probability of there being someone near the vehicle.

3. A method for identifying abnormal behavior of a vehicle according to claim 1 or 2, characterized in that: The step of obtaining the coordinates of the clipping frame corresponding to the vehicle state identification frame according to the position coordinates of the vehicle detection frame comprises: The length and width of the vehicle detection frame are expanded by preset sizes to obtain the clipping frame coordinates corresponding to the vehicle state recognition frame.

4. The method for identifying abnormal behavior of a vehicle according to claim 2, characterized in that: The step of processing the monitoring area image sub-atlas according to the preset vehicle state recognition model to obtain the vehicle state recognition vector corresponding to the monitoring area image comprises: The vehicle door status information is obtained according to the formula P_do_new=a1*P_do+a2*P_per, where P_do_new is the door opening probability after considering the presence of people, a1=exp(P_do) / (exp(P_do)+exp(P_per)), a2=exp(P_per) / (exp(P_do)+exp(P_per)); The vehicle trunk status information is obtained according to the formula P_to_new=P_to*(1-P_per), where P_to_new is the probability of the trunk being opened after considering the presence of people.

5. The method for identifying abnormal behavior of a vehicle according to claim 4, characterized in that: The step of comparing the door opening probability and / or trunk opening probability in the vehicle state recognition vector with a preset state determination threshold to confirm whether the vehicle has abnormal state behavior includes: Obtain the vehicle state indication vector state=[S_d, S_t] according to the preset state determination threshold thr=[thr_do, thr_to] and the vehicle state identification vector Pv_new=[P_do_new, P_to_new], wherein S_d=(P_do_new>thr_do), S_t=(P_to_new>thr_to); If the value of the vehicle state indication vector is 1, it is confirmed that the vehicle has abnormal state behavior; Output warning information carrying the vehicle status information and the vehicle target detection and recognition result.

6. A vehicle abnormal behavior state recognition system, characterized in that: The system comprises: An acquisition unit, used to acquire a monitoring area image collected in real time, and obtain the position coordinates of a vehicle detection frame from the monitoring area image according to a preset vehicle detection and recognition model; The acquisition unit is further used to acquire the clipping frame coordinates corresponding to the vehicle state recognition frame according to the vehicle detection frame position coordinates, and process the monitoring area image according to the clipping frame coordinates to obtain a monitoring area image sub-atlas; a processing unit, configured to process the monitoring area image sub-atlas according to a preset vehicle state recognition model to obtain a vehicle state recognition vector corresponding to the monitoring area image, wherein the preset vehicle state recognition model is used to recognize a probability of a door being opened, a probability of a trunk being opened, and a probability of a person being present around the vehicle, and the vehicle state recognition vector includes a probability of a door being opened and / or a probability of a trunk being opened; The determination unit is used to compare the door opening probability and / or trunk opening probability in the vehicle state recognition vector with a preset state determination threshold to confirm whether the vehicle has abnormal state behavior.

7. A vehicle abnormal behavior state recognition system according to claim 6, characterized in that: The system further comprises: A cropping unit, used to obtain a vehicle detection frame from the monitoring area image stored in the database through the preset vehicle detection and recognition model, and to crop and save a local target area of ​​the monitoring area image data according to the preset size of the length and width of the vehicle detection frame as a training image data set; A labeling unit, used to obtain a vehicle detection frame from the monitoring area image stored in the database through the preset vehicle detection and recognition model, and to crop and save a local target area of ​​the monitoring area image data according to the preset size of the length and width of the vehicle detection frame as a training image data set; A generating unit is used to input the model training data into a preset classification model to obtain a three-component vector Pv=[P_do, P_to, P_per] as the preset vehicle state recognition model, where P_do is the probability of a door being opened, P_to is the probability of a trunk being opened, and P_per is the probability of someone being present near the vehicle.

8. A vehicle abnormal behavior state recognition system according to claim 6 or 7, characterized in that: The acquisition unit is further configured to expand the length and width of the vehicle detection frame by preset sizes to obtain the clipping frame coordinates corresponding to the vehicle state recognition frame.

9. The system for identifying abnormal behavior of a vehicle according to claim 7, characterized in that: The processing unit is specifically used to obtain the vehicle door status information according to the formula P_do_new=a1*P_do+a2*P_per, where P_do_new is the probability of door opening after considering the presence of people, a1=exp(P_do) / (exp(P_do)+exp(P_per)), a2=exp(P_per) / (exp(P_do)+exp(P_per)); and obtain the vehicle trunk status information according to the formula P_to_new=P_to*(1-P_per), where P_to_new is the probability of trunk opening after considering the presence of people.

10. The vehicle abnormal behavior state recognition system according to claim 9, characterized in that: The confirmation unit is specifically used to obtain a vehicle state indication vector state = [S_d, S_t] according to a preset state judgment threshold thr = [thr_do, thr_to] and a vehicle state identification vector Pv_new = [P_do_new, P_to_new], wherein S_d = (P_do_new>thr_do), S_t = (P_to_new>thr_to); if the value of the vehicle state indication vector is 1, it is confirmed that the vehicle has abnormal state behavior; and output an alarm message carrying the vehicle state information and the vehicle target detection and identification result.

Images