Abnormal behavior determination method and apparatus, and readable storage medium

Abstract

The application discloses a method and device for determining abnormal behavior and a readable storage medium, relates to the technical field of road safety, and is used for accurately determining abnormal behavior. The method comprises the following steps: determining the height of a target limb of a first driver according to position information of the target limb; in the case that the height of the target limb of the first driver is less than a first threshold, determining the motion trajectory of a first vehicle in a target scene according to position information of the first vehicle in a plurality of image frames; in the case that the height of the target limb of the first driver is greater than or equal to the first threshold, determining the motion trajectory of the first vehicle in the target scene according to position information of the target limb of the first driver in the plurality of image frames; and determining whether the first vehicle has abnormal behavior according to the motion trajectory of the first vehicle in the target scene.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of road safety, and in particular to a method and device for determining abnormal behavior and a readable storage medium. BACKGROUND

[0002] With the expansion of the city size and the continuous increase of the population, the traffic jam of motor vehicles is increasing, and non-motor vehicles have become the preferred traffic tool for people's short trips. However, due to the large number of non-motor vehicles and the random riding, the safety risk of traffic is large.

[0003] In order to detect the illegal behavior of non-motor vehicles, it is necessary to identify the illegal behavior of the video of the non-motor vehicle driving process. However, in the scene where non-motor vehicles are relatively dense (such as a red light intersection), there are a large number of occlusions between non-motor vehicles, resulting in low accuracy of illegal behavior identification. SUMMARY

[0004] The present application provides a method and device for determining abnormal behavior and a readable storage medium for accurately determining abnormal behavior.

[0005] To achieve the above purpose, the present application adopts the following technical scheme:

[0006] In a first aspect, a method for determining abnormal behavior is provided, comprising: obtaining video data of a target scene; the video data comprising a plurality of image frames, each image frame comprising one or more vehicles and one or more drivers; performing body position detection on each image frame to obtain position information of a target body of each driver in each image frame; determining a height of the target body of a first driver according to the position information of the target body of the first driver; the first driver being any one of the one or more drivers; in a case where the height of the target body of the first driver is less than a first threshold, performing vehicle position detection on each image frame to obtain position information of a first vehicle in the plurality of image frames, and determining a motion trajectory of the first vehicle in the target scene according to the position information of the first vehicle in the plurality of image frames; the first vehicle being a vehicle corresponding to the first driver; in a case where the height of the target body of the first driver is greater than or equal to the first threshold, determining the motion trajectory of the first vehicle in the target scene according to the position information of the target body of the first driver in the plurality of image frames; and determining whether the first vehicle has abnormal behavior according to the motion trajectory of the first vehicle in the target scene.

[0007] Based on the technical solutions provided in the embodiments of the present application, the determining apparatus can determine the height of the first driver target body by performing body position detection on each image frame. In the case that the height of the first driver target body is less than a first threshold value, vehicle position detection is performed on each image frame to obtain position information of the first vehicle in the plurality of image frames, and the motion trajectory of the first vehicle in the target scene is determined according to the position information of the first vehicle in the plurality of image frames. In the case that the height of the first driver target body is greater than or equal to the first threshold value, the motion trajectory of the first vehicle in the target scene is determined according to the position information of the first driver target body in the plurality of image frames. In this way, when the target body region of the driver is large, i.e., the driver is close to the collection device, the position information of the driver target body is used preferentially to determine the motion trajectory of the vehicle, so as to avoid the case that the target of the driver is large when the driver is close to the collection device, and the vehicle is shielded, thereby improving the accuracy of determining the abnormal behavior of the vehicle. When the head region of the person is small, the position information of the vehicle is used for tracking, and the position information of the driver target body does not need to be determined, thereby reducing the system load.

[0008] Optionally, the position information of an object includes the top pixel coordinates of the minimum circumscribed rectangle of the object; and the height of the first driver target body is determined according to the position information of the first driver target body, including: determining the maximum value of the vertical coordinates and the minimum value of the vertical coordinates in the top pixel coordinates of the minimum circumscribed rectangle of the first driver target body; and determining the height of the first driver target body according to the difference between the maximum value of the vertical coordinates and the minimum value of the vertical coordinates.

[0009] Optionally, the position information of an object includes the top pixel coordinates of the minimum circumscribed rectangle of the object; and the height of the first driver target body is determined according to the position information of the first driver target body, including: determining the maximum value of the vertical coordinates and the minimum value of the vertical coordinates in the top pixel coordinates of the minimum circumscribed rectangle of the first driver target body; and determining the height of the first driver target body according to the difference between the maximum value of the vertical coordinates and the minimum value of the vertical coordinates.

[0010] Optionally, the method further comprises: determining the distance between each vehicle and each driver in each image frame; processing the distance between each vehicle and each driver according to a preset computer algorithm to obtain a corresponding relationship between each vehicle and each driver; and determining the first driver corresponding to the first vehicle from the corresponding relationship between each vehicle and each driver.

[0011] Optionally, determining whether the first vehicle has abnormal behavior according to the motion trajectory of the first vehicle in the target scene comprises: obtaining calibration data; the calibration data comprises a specified driving direction and signal light data; and determining that the first vehicle has abnormal behavior when it is detected that the motion trajectory of the first vehicle is different from the specified driving direction, or the signal light data is the first type of signal and the length of the motion trajectory of the first vehicle is greater than a fourth threshold.

[0012] Optionally, the method further comprises: performing body category detection on each image frame to obtain category information of each driver target body in each image frame; the category information is used to indicate whether the driver wears a helmet; and determining that the driver has abnormal behavior when the category information indicates that the driver does not wear a helmet.

[0013] Optionally, the method further comprises: performing license plate frame detection on each image frame to obtain position information of the license plate frame of each vehicle in each image frame; performing perspective change processing on each license plate frame according to the position information of the license plate frame of each vehicle; and performing identification processing on each license plate frame of each vehicle subjected to the perspective change processing to obtain the license plate number of each vehicle.

[0014] In a second aspect, an abnormal behavior determining apparatus is provided, including an acquisition unit, a detection unit, and a determining unit. The acquisition unit is configured to acquire video data of a target scene. The video data includes a plurality of image frames, and each image frame includes one or more vehicles and one or more drivers. The detection unit is configured to perform body position detection on each image frame to obtain position information of a target body of each driver in each image frame. The determining unit is configured to determine a height of the target body of a first driver according to the position information of the target body of the first driver, the first driver being any one of the one or more drivers. When the height of the target body of the first driver is less than a first threshold, the determining unit is further configured to perform vehicle position detection on each image frame to obtain position information of a first vehicle in the plurality of image frames, and determine a motion trajectory of the first vehicle in the target scene according to the position information of the first vehicle in the plurality of image frames, the first vehicle being a vehicle corresponding to the first driver. When the height of the target body of the first driver is greater than or equal to the first threshold, the determining unit is further configured to determine the motion trajectory of the first vehicle in the target scene according to the position information of the target body of the first driver in the plurality of image frames. The determining unit is further configured to determine whether the first vehicle has an abnormal behavior according to the motion trajectory of the first vehicle in the target scene.

[0015] Optionally, the position information of an object includes top pixel coordinates of a minimum circumscribed rectangle of the object. The determining unit is specifically configured to determine a maximum value of a vertical coordinate and a minimum value of the vertical coordinate among the top pixel coordinates of the minimum circumscribed rectangle of the target body of the first driver, and determine the height of the target body of the first driver according to a difference between the maximum value and the minimum value of the vertical coordinate.

[0016] Optionally, the detection unit is specifically configured to perform position detection on the plurality of vehicles in the first image frame to obtain a target vehicle set in the first image frame, the target vehicle set including a plurality of target vehicles and historical position information corresponding to the plurality of target vehicles, the height of a target vehicle being greater than a third threshold. The detection unit is further configured to predict position information of each target vehicle in a second image frame according to a Kalman filter and the target vehicle set in the first image frame to obtain a first prediction set, the shooting time of the second image frame being after the first image frame. The detection unit is further configured to perform position detection on the plurality of vehicles in the second image frame to obtain a target vehicle set in the second image frame. The detection unit is further configured to determine an intersection over union of each target vehicle in the first prediction set and each target vehicle in the target vehicle set in the second image frame. The detection unit is further configured to determine the first vehicle in each image frame according to the intersection over union and a preset computer algorithm to obtain the position information of the first vehicle in the plurality of image frames.

[0017] Optionally, the determining unit is further configured to determine the distance between each vehicle and each driver in each image frame, process the distance between each vehicle and each driver according to a preset computer algorithm to obtain a corresponding relationship between each vehicle and each driver, and determine the first driver corresponding to the first vehicle from the corresponding relationship between each vehicle and each driver.

[0018] Optionally, the determining unit is specifically configured to: obtain calibration data, wherein the calibration data comprises a specified driving direction and signal lamp data; and if it is detected that the motion trajectory of the first vehicle is different from the specified driving direction or the signal lamp data is of the first type, and the length of the motion trajectory of the first vehicle is greater than a fourth threshold value, determine that the first vehicle has abnormal behavior.

[0019] Optionally, the detecting unit is further configured to: perform body category detection on each image frame to obtain category information of each driver target body in each image frame, wherein the category information is used to indicate whether the driver wears a helmet; and the determining unit is further configured to determine that the driver has abnormal behavior if the category information indicates that the driver does not wear a helmet.

[0020] Optionally, the detecting unit is further configured to: perform license plate frame detection on each image frame to obtain position information of the license plate frame of each vehicle in each image frame; perform perspective change processing on each license plate frame according to the position information of the license plate frame of each vehicle; and perform identification processing on each license plate frame of each vehicle subjected to the perspective change processing to obtain the license plate number of each vehicle.

[0021] In a third aspect, an abnormal behavior determining apparatus is provided, which can implement the functions performed by the abnormal behavior determining apparatus in the above aspects or possible designs, and the functions can be implemented by hardware, for example, in a possible design, the abnormal behavior determining apparatus can include a processor and a communication interface, and the processor can be configured to support the abnormal behavior determining apparatus to implement the functions involved in the first aspect or any possible design of the first aspect.

[0022] In another possible design, the abnormal behavior determining apparatus can further include a memory configured to store computer-executable instructions and data necessary for the abnormal behavior determining apparatus. When the abnormal behavior determining apparatus is running, the processor executes the computer-executable instructions stored in the memory, so that the abnormal behavior determining apparatus performs the abnormal behavior determining method in the first aspect or any possible design of the first aspect.

[0023] In a fourth aspect, a computer readable storage medium is provided, which can be a readable nonvolatile storage medium, and the computer readable storage medium stores computer instructions or programs, which, when executed on a computer, enable the computer to perform the method for determining abnormal behavior of any possible implementation of the first aspect or the method for determining abnormal behavior of any possible implementation of the aspects.

[0024] In a fifth aspect, a computer program product containing instructions is provided, which, when executed on a computer, enable the computer to perform the method for determining abnormal behavior of any possible implementation of the first aspect or the method for determining abnormal behavior of any possible implementation of the aspects.

[0025] In a sixth aspect, a device for determining abnormal behavior is provided, which includes one or more processors and one or more memories. The one or more memories are coupled to the one or more processors, and the one or more memories are configured to store computer program codes including computer instructions, which, when executed by the one or more processors, enable the device for determining abnormal behavior to perform the method for determining abnormal behavior of any possible implementation of the first aspect or the method for determining abnormal behavior of any possible implementation of the aspects.

[0026] In a seventh aspect, a chip system is provided, which includes a processor and a communication interface, and the chip system can be used to implement the functions performed by the device for determining abnormal behavior of any possible implementation of the first aspect or the first aspect, for example, the processor is configured to obtain the first request message from the terminal device through the communication interface. In a possible implementation, the chip system further includes a memory, which is configured to store program instructions and / or data. The chip system can be composed of a chip, or can include a chip and other discrete devices, without limitation. BRIEF DESCRIPTION OF DRAWINGS

[0027] Figure 1 A schematic diagram of an abnormal behavior determination system provided by an embodiment of the present application;

[0028] Figure 2 A schematic diagram of another abnormal behavior determination system provided by an embodiment of the present application;

[0029] Figure 3 A structural schematic diagram of a device for determining abnormal behavior provided by an embodiment of the present application;

[0030] Figure 4 A flowchart of a method for determining abnormal behavior provided by an embodiment of the present application;

[0031] Figure 5 A schematic diagram before perspective change provided by an embodiment of the present application;

[0032] Figure 6A schematic diagram of a perspective change is provided for an embodiment of the present application.

[0033] Figure 7 A flowchart of another method for determining abnormal behavior is provided for an embodiment of the present application.

[0034] Figure 8 A flowchart of another method for determining abnormal behavior is provided for an embodiment of the present application.

[0035] Figure 9 A flowchart of another method for determining abnormal behavior is provided for an embodiment of the present application.

[0036] Figure 10 A flowchart of another method for determining abnormal behavior is provided for an embodiment of the present application.

[0037] Figure 11 A structure diagram of another device for determining abnormal behavior is provided for an embodiment of the present application. DETAILED DESCRIPTION

[0038] In order for those skilled in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings.

[0039] It should be noted that the terms "first", "second", and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present disclosure described herein can be implemented in an order other than that illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Rather, they are merely examples of devices and methods consistent with some aspects of the embodiments of the present application, as detailed in the appended claims.

[0040] It should also be understood that the term "comprising" indicates the presence of the described features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, and / or components.

[0041] With the expansion of the city size and the continuous increase of the population, the traffic jam of motor vehicles is increasing, and non-motor vehicles have become the preferred means of transportation for short trips. However, due to the large number of non-motor vehicles and the random riding, the safety risk of traffic travel is large.

[0042] In order to detect the illegal behavior of non-motor vehicles, it is necessary to identify the illegal behavior of the video of the driving process of the non-motor vehicles. However, in the scene where the non-motor vehicles are relatively dense (such as a red and green light intersection), there are a large number of occlusions between the non-motor vehicles, resulting in a low accuracy of illegal behavior identification.

[0043] In view of this, the embodiment of the application provides a method for determining abnormal behavior, comprising:

[0044] obtaining video data of a target scene; the video data comprises a plurality of image frames, each image frame comprises one or more vehicles and one or more drivers; performing limb position detection on each image frame to obtain position information of a target limb of each driver in each image frame; determining a height of a target limb of a first driver according to the position information of the target limb of the first driver; the first driver is any one of the one or more drivers; in a case where the height of the target limb of the first driver is less than a first threshold, performing vehicle position detection on each image frame to obtain position information of a first vehicle in the plurality of image frames, and determining a motion trajectory of the first vehicle in the target scene according to the position information of the first vehicle in the plurality of image frames; the first vehicle is a vehicle corresponding to the first driver; in a case where the height of the target limb of the first driver is greater than or equal to the first threshold, determining the motion trajectory of the first vehicle in the target scene according to the position information of the target limb of the first driver in the plurality of image frames; and determining whether the first vehicle has an abnormal behavior according to the motion trajectory of the first vehicle in the target scene.

[0045] The method provided by the embodiment of the application will be described in detail below with reference to the accompanying drawings.

[0046] It should be noted that the network system described in the embodiments of the application is for more clearly illustrating the technical solutions of the embodiments of the application, and does not constitute a limitation on the technical solutions provided by the embodiments of the application. It is known to those skilled in the art that, with the evolution of network systems and the appearance of other network systems, the technical solutions provided by the embodiments of the application are also applicable to similar technical problems.

[0047] Figure 1 A schematic diagram of an abnormal behavior determination system provided by the embodiment of the application is shown. As Figure 1 shown, the target user determination system can include an abnormal behavior determination device 11 (hereinafter referred to as a determination device) and a collection device 12. The determination device 11 is connected with the collection device 12. The determination device 11 and the collection device 12 can be connected in a wireless manner.

[0048] In the embodiments of the present application, the determination device 11 can also be referred to as a computer, a server, etc. In the embodiments of the present application, the specific technology and specific device form of the determination device 11 are not limited.

[0049] In the embodiments of the present application, the collection device 12 can be a camera or the like device having a video shooting function. The specific technology, specific number and specific device form of the collection device 12 are not limited in the embodiments of the present application.

[0050] The collection device 12 is configured to shoot a target scene to obtain video data of the target scene, and send the video data of the target scene to the determination device 11. The determination device 11 is configured to receive the video data of the target scene sent by the collection device 12, and determine whether a vehicle in the target scene has an abnormal behavior according to the video data of the target scene.

[0051] In different application scenarios, the determination device 11 and the collection device 12 can be independent devices or integrated into the same device. The embodiments of the present application do not make specific limitations.

[0052] It should be noted that, Figure 1 is only an exemplary block diagram, Figure 1 The names of various devices included in the embodiments of the present application are not limited, and in addition to Figure 1 the function nodes shown, other nodes can also be included. The embodiments of the present application do not make limitations on this.

[0053] Figure 2 The figure shows another abnormal behavior determination system provided by the embodiments of the present application. As shown in the figure, Figure 2 The determination system includes a single-frame information acquisition module, a tracking module, a fusion module, and an event analysis module.

[0054] The single-frame information acquisition module includes a vehicle detection model, a target limb detection model, and a license plate detection and recognition model. The vehicle detection model can be used to detect the category information and position information of each vehicle in each image frame. The target limb detection model is used to obtain the position information of the target limb of the driver and whether the driver wears a safety helmet. The license plate detection and recognition model is used to obtain the license plate position of the vehicle. If the license plate area of the vehicle meets certain conditions (such as a width greater than 60 pixels), the license plate recognition model is used to obtain the license plate content of the vehicle.

[0055] The tracking module is configured to obtain the trajectory information of the vehicle according to the position information of the vehicle, the position information of the target limb of the driver, and the license plate information and target information.

[0056] The fusion module is configured to merge multiple frames of prediction according to the target trajectory information, update the video of the safety helmet and the license plate content information.

[0057] The event analysis module is configured to determine whether the vehicle has abnormal behavior according to the position information of the vehicle and a preset rule.

[0058] In a specific implementation process, Figure 1 Each device in the device can adopt the component structure shown in Figure 2 Or include the components shown in Figure 3 . Figure 3 A component structure of a determination apparatus 200 is provided for the embodiments of the present application. The determination apparatus 200 can be a server, or the determination apparatus 200 can be a chip or a system on chip in the server. As shown in Figure 3 The determination apparatus 200 includes a processor 201, a communication interface 202 and a communication line 203.

[0059] Further, the determination apparatus 200 can further include a memory 204. The processor 201, the memory 204 and the communication interface 202 can be connected through the communication line 203.

[0060] The processor 201 is a CPU, a general processor, a network processor (NP), a digital signal processing (DSP), a microprocessor, a microcontroller, a programmable logic device (PLD) or any combination thereof. The processor 201 can also be other devices with processing functions, such as a circuit, a device or a software module, which are not limited.

[0061] The communication interface 202 is configured to communicate with other devices or other communication networks. The communication interface 202 can be a module, a circuit, a communication interface or any device capable of communication.

[0062] The communication line 203 is configured to transmit information between components included in the determination apparatus 200.

[0063] The memory 204 is configured to store instructions. The instructions can be a computer program.

[0064] The memory 204 can be a read-only memory (ROM) or other type of static storage device that can store static information and / or instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and / or instructions, an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disk storage, an optical disk storage including a compact disc, a laser disc, an optical disc, a digital versatile disc, a Blu-ray disc, and the like, a magnetic disk storage or other magnetic storage devices, and the like, without limitation.

[0065] It should be noted that the memory 204 can exist independently of the processor 201, or can be integrated with the processor 201. The memory 204 can be used to store instructions or program codes or some data, and the like. The memory 204 can be located in the determination apparatus 200, or can be located outside the determination apparatus 200, without limitation. The processor 201 is configured to execute the instructions stored in the memory 204, so as to implement the method for determining abnormal behavior provided in the embodiments of the present application.

[0066] In an example, the processor 201 can include one or more CPUs, for example, the CPU0 and the CPU1 in the CPU 201. Figure 3

[0067] As an optional implementation, the determination apparatus 200 includes a plurality of processors, for example, in addition to the processor 201 in the CPU 201, the processor 205 can also be included. Figure 3

[0068] It should be noted that the constituent structures shown in the CPU 201 do not constitute a limitation on each device in the CPU 201, except for the components shown in the CPU 201, Figure 3 Figure 1 each device in the CPU 201 can include more or fewer components, or combine some components, or different component arrangements. Figure 3 Figure 1 In the embodiments of the present application, the chip system can be composed of a chip, or can include a chip and other discrete devices. Figure 3

[0069] In the embodiments of the present application, the chip system can be composed of a chip, or can include a chip and other discrete devices.

[0070] ​​​​​In addition, the actions, terms and the like related among the embodiments of the present application can be mutually referenced and are not limited. The message names or parameter names in the messages exchanged between various devices in the embodiments of the present application are only examples, and other names can also be used in specific implementation, which are not limited.

[0071] In order to clearly describe the technical solutions of the embodiments of the present application, in the embodiments of the present application, the same items or similar items with basically the same functions and effects are distinguished by using "first", "second" and the like. The skilled in the art can understand that "first", "second" and the like do not limit the quantity and execution order, and "first", "second" and the like do not necessarily mean different.

[0072] It should be noted that in the present application, the words "exemplary" or "for example" are used to mean serving as an example, instance, or illustration. Any embodiment or design described as "exemplary" or "for example" in the present application should not be construed as being more preferred or advantageous than other embodiments or designs. Rather, the use of the words "exemplary" or "for example" is intended to present concepts in a concrete manner.

[0073] In the present application, "at least one" means one or more, and "multiple" means two or more. The association relationship between the associated objects is described by "and / or", which means that there can be three kinds of relationships, for example, A and / or B, which can represent the following three cases: A exists alone, A and B exist together, and B exists alone, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after it. "At least one of the following" or similar expressions means any combination of these items, including any combination of single item or multiple items. For example, at least one of a, b, or c, can represent a, b, c, a-b, a-c, b-c, or a-b-c, where a, b, and c can be single or multiple.

[0074] The embodiments of the present application will be described below in conjunction with Figure 1 the target user determination system shown in the figure, the determination method of abnormal behavior provided by the embodiments of the present application is described.

[0075] Figure 4 A determination method of abnormal behavior is provided for the embodiments of the present application, which is applied to a determination device and can also be applied to a server. The determination device can be a determination device 11 in Figure 1 , or a device such as a chip in the determination device 11. The embodiments of the present application are described by taking the determination device as an example, as shown in Figure 4 , the method includes the following S401-S406:

[0076] S401, determine that the device obtains video data of a target scene.

[0077] The video data includes a plurality of image frames, and each image frame includes one or more vehicles and one or more drivers. The target scene can be an intersection of a city road, etc. The plurality of vehicles can include motor vehicle categories such as cars, trucks, and buses, and non-motor vehicle categories such as bicycles and electric bicycles.

[0078] As a possible implementation, the determination device can be in communication connection with a collection device at the target scene. The determination device can send a first request message to the collection device, the first request message being used to request video data of the target scene. After receiving the first request message sent by the determination device, the collection device can send a first reply message to the determination device, the first reply message including the video data of the target scene. Correspondingly, the determination device can obtain the video data of the target scene by receiving the first reply information sent by the collection device.

[0079] It should be noted that the video data of the target scene can be pre-processed video data. For example, the pre-processed video data can be video data extracted every 4 seconds.

[0080] S402, the determination device detects the position of each image frame to obtain the position information of each driver target limb in each image frame.

[0081] Each driver target limb can be the head of the driver. The position information of an object includes the top pixel coordinates of the minimum bounding rectangle of the object. The top pixel coordinates of the minimum bounding rectangle can include the top-left corner coordinates, the bottom-left corner coordinates, the top-right corner coordinates, and the bottom-right corner coordinates.

[0082] As a possible implementation, the determination device can input each image frame into a head detection model to obtain the minimum bounding rectangle of each driver target limb in each image frame, and the top pixel coordinates of the minimum bounding rectangle.

[0083] In some embodiments, after the determination device inputs each image frame into the head detection model, the class of each driver target limb in each image frame can also be obtained. For example, in the case of a head target limb, the class of the target limb can include: the driver's head wearing a helmet, and the driver's head not wearing a helmet.

[0084] It should be noted that the head detection model can be set as needed. For example, the YOLOV5-s model can be used.

[0085] Specifically, the head detection model YOLOV5-s in the network structure for detecting the head is modified.

[0086] For example, the input feature map size of the head is (B, C, H, W), B represents the batch size, C represents the number of channels, and H and W represent the height and width of the feature map. The original YOLOV5-s model adopts a coupled manner to construct the detection head, and only one branch is included, which includes one convolution layer (C1, 3*(2+1+4), 3, 3), and the output size is (B, 3*(2+1+4), H, W). Among them, 3 represents the number of anchors, 2 represents the confidence of the two classes of wearing a helmet and not wearing a helmet, 1 represents the confidence of having a target, and 4 represents the position of the bounding rectangle.

[0087] The YOLOV5-s model of the present application adopts a decoupled manner to construct the detection head, including three branches. Branch 1 is used to predict the class confidence, including three convolution layers with parameters (C, 128, 3, 3), (128, 128, 3, 3), and (128, 3*2, 3, 3), and the output size is (B, 3*2, H, W). Branch 2 is used to predict the position of the bounding rectangle, including three convolution layers with parameters (C1, 128, 3, 3), (128, 128, 3, 3), and (128, 3*4, 3, 3), and the output size is (B, 3*4, H, W). Branch 3 is used to predict the confidence of having a target, including one convolution layer with parameters (C, 3*1, 3, 3), and the output size is (B, 3*1, H, W). Finally, the output of the second convolution layer of branch 1 is used as the feature vector required by the DeepSort algorithm, and the size is (B, 128, H, W).

[0088] S403, determining the height of the first driver target limb according to the position information of the first driver target limb.

[0089] Among them, the first driver is any one of the plurality of drivers.

[0090] As a possible implementation manner, after determining the position information of the first driver target limb, the determining apparatus can determine the height of the first driver target limb according to the vertex pixel coordinates of the minimum bounding rectangle of the first driver target limb.

[0091] It should be noted that the specific description of determining the height of the first driver target limb according to the position information of the first driver target limb in the possible implementation manner will be described in the subsequent part, and the present application will not be described here.

[0092] S404, in a case where the height of the target limb of the first driver is less than the first threshold, the determining apparatus performs vehicle position detection on each image frame to obtain position information of the first vehicle in the multiple image frames, and determines a motion trajectory of the first vehicle in the target scene according to the position information of the first vehicle in the multiple image frames.

[0093] The first vehicle is a vehicle corresponding to the first driver. The first threshold can be set as needed. For example, the first threshold can be 50 pixels.

[0094] As a possible implementation manner, the determining apparatus can perform vehicle position detection on each image frame according to a vehicle detection model to obtain the position information of the first vehicle in the multiple image frames,

[0095] Further, after obtaining the position information of the first vehicle in the multiple image frames, the determining apparatus can determine the motion trajectory of the first vehicle in the target scene according to a SORT algorithm and the front-to-back order of the multiple image frames.

[0096] For example, the multiple image frames can include image 1, image 2, and image 3 in sequence, the position information of the first vehicle in the image 1 can be (80, 80), the position information of the first vehicle in the image 2 can be (70, 70), and the position information of the first vehicle in the image 3 can be (60, 60). Then the determining apparatus can determine that the motion trajectory of the first vehicle in the target scene is (80, 80)~(70, 70)~(60, 60).

[0097] It should be noted that the specific description of the possible implementation manner of performing vehicle position detection on each image frame according to a vehicle detection model to obtain the position information of the first vehicle in the multiple image frames will be described in subsequent parts, and the present application will not be described here.

[0098] S405, in a case where the height of the target limb of the first driver is greater than or equal to the first threshold, the determining apparatus determines a motion trajectory of the first vehicle in the target scene according to the position information of the target limb of the first driver in the multiple image frames.

[0099] As a possible implementation manner, the determining apparatus can determine the first vehicle corresponding to the first driver according to a preset computer algorithm, and obtain the position information of the target limb of the first driver in the multiple image frames according to a DeepSORT algorithm to determine the motion trajectory of the first vehicle in the target scene.

[0100] It should be noted that the specific description of the possible implementation manner of determining the first vehicle corresponding to the first driver will be described in subsequent parts, and the present application will not be described here.

[0101] S406. The determining device determines whether the first vehicle has any abnormal behavior based on the movement trajectory of the first vehicle in the target scene.

[0102] Abnormal behaviors can include running red lights, driving against traffic, not wearing a helmet, and obscuring license plates.

[0103] As one possible implementation, the determining device can acquire pre-defined rules and determine that the first vehicle has abnormal behavior if the first vehicle's trajectory in the target scene does not conform to the pre-defined rules.

[0104] It should be noted that the specific details of determining whether the first vehicle has abnormal behavior based on its movement trajectory in the target scene in this possible implementation will be explained in a later section, and will not be repeated here.

[0105] In some embodiments, after determining that the first vehicle is behaving abnormally, the determining device can also identify the license plate information of the first vehicle. For example, the determining device can identify the license plate information of the first vehicle using a license plate recognition model, which can be implemented based on the CRNN algorithm. The CRNN algorithm first extracts image features using a CNN, then extracts sequence features using an RNN, and finally predicts the final text content using CTC.

[0106] It should be noted that in order to obtain the rectangular input required by the license plate recognition model, perspective transformation of the license plate image region is necessary. To obtain the perspective transformation matrix, the coordinates of four points on the original and transformed images are required. The coordinates on the original image are obtained by the license plate detection model, and the coordinates on the transformed image can be specified by the user. The calculation process for the perspective transformation described above can refer to existing technologies and will not be elaborated here. For example, the specific instructions in CN110060200A can be consulted.

[0107] Furthermore, after obtaining the results of whether a helmet is being worn and whether the license plate is being obscured in each image frame, a vote can be conducted based on the results of multiple image frames to determine information such as whether a helmet is being worn and whether the license plate is being obscured.

[0108] For example, such as Figure 5 As shown, a schematic diagram before perspective transformation is presented, such as... Figure 6 As shown, a schematic diagram after perspective transformation is presented.

[0109] Based on the technical solutions provided in the embodiments of the present application, the determination apparatus can determine the height of the first driver target limb by performing limb position detection on each image frame. In the case where the height of the first driver target limb is less than a first threshold, the determination apparatus performs vehicle position detection on each image frame to obtain position information of the first vehicle in the multiple image frames, and determines a motion trajectory of the first vehicle in the target scene according to the position information of the first vehicle in the multiple image frames. In the case where the height of the first driver target limb is greater than or equal to the first threshold, the determination apparatus determines the motion trajectory of the first vehicle in the target scene according to the position information of the first driver target limb in the multiple image frames. In this way, the determination apparatus can preferentially use the position information of the first driver target limb to determine the motion trajectory of the vehicle when the target limb region of the driver is large, i.e., when the driver is close to the collection device, thereby avoiding the situation where the target of the vehicle is large when the driver is close to the collection device, causing the vehicle to be blocked, and improving the accuracy of determining the abnormal behavior of the vehicle. When the head region is small, the position information of the vehicle is used for tracking, and the position information of the driver target limb does not need to be determined, thereby reducing the system load.

[0110] As shown in FIG. 1, in order to determine the height of the first driver target limb, the determination method of the present application can specifically include the following S501-S502. Figure 7

[0111] S501, the determination apparatus determines the maximum value of the longitudinal coordinate and the minimum value of the longitudinal coordinate in the vertex pixel coordinates of the minimum circumscribed rectangle of the first driver target limb.

[0112] As a possible implementation manner, the determination apparatus is internally provided with a comparator, and the digital extractor is used to compare the size relationship of different coordinates. The determination apparatus can determine the maximum value of the longitudinal coordinate and the minimum value of the longitudinal coordinate in the vertex pixel coordinates of the minimum circumscribed rectangle of the first driver target limb according to the comparator.

[0113] In an example, the vertex pixel coordinates of the minimum circumscribed rectangle of the first driver target limb can be (1, 1), (60, 1), (1, 80), and (60, 80) respectively. Then the determination apparatus can determine that the maximum value of the longitudinal coordinate in the vertex pixel coordinates of the minimum circumscribed rectangle of the first driver target limb is 80, and the minimum value of the longitudinal coordinate in the vertex pixel coordinates of the minimum circumscribed rectangle of the first driver target limb is 1.

[0114] S502, the determination apparatus determines the height of the first driver target limb according to the difference between the maximum value of the longitudinal coordinate and the minimum value of the longitudinal coordinate.

[0115] ​In an example, when the maximum value of the longitudinal coordinate in the top pixel coordinate of the minimum bounding rectangle of the first driver target limb is 80 and the minimum value of the longitudinal coordinate in the top pixel coordinate of the minimum bounding rectangle of the first driver target limb is 1, the determining apparatus can determine that the difference between the maximum value of the longitudinal coordinate and the minimum value of the longitudinal coordinate is 80-1=79, and determine 79 as the height of the first driver target limb.

[0116] A possible embodiment, as shown in Figure 8 To obtain the position information of the first vehicle in the plurality of image frames, the above S404 of the present application can specifically include the following S601-S604.

[0117] S601, the determining apparatus detects the positions of the plurality of vehicles in the first image frame to obtain a target vehicle set in the first image frame.

[0118] The target vehicle set includes a plurality of target vehicles and historical position information corresponding to the plurality of target vehicles; and the height of the target vehicle is greater than a third threshold value. The third threshold value can be set as needed. For example, it can be 200 pixels high.

[0119] As a possible implementation manner, the determining apparatus can detect the positions of the plurality of vehicles according to a vehicle position detection model to determine the height of each vehicle, and determine the vehicle with a height greater than the third threshold value from the heights of the plurality of vehicles as a target vehicle to obtain the target vehicle set.

[0120] S602, the determining apparatus predicts the position information of each target vehicle in the second image frame according to the Kalman filter and the target vehicle set in the first image frame to obtain a first prediction set.

[0121] The shooting time of the second image frame is after the first image frame.

[0122] As a possible implementation manner, after obtaining the target vehicle set, the determining apparatus can create a Kalman filter for each tracking target to record the historical position information of the current target vehicle. Further, the determining apparatus can use the Kalman filter to predict the position of each target vehicle in the next frame to obtain the first prediction set.

[0123] For example, the first prediction set can be a Kalman prediction set K={k i’ i=1,...,N K}N K represents the number of target vehicles.

[0124] S603, the determining apparatus determines the intersection over union of each target vehicle in the first prediction set and each target vehicle in the target vehicle set in the second image frame.

[0125] wherein the IoU is used to represent the similarity between each target vehicle in the first prediction set and each target vehicle in the target vehicle set in the second image frame. The target vehicle set in the second image frame can be detected according to the vehicle detection model, and each target vehicle in the target vehicle set in the second image frame can be D = {D j’ j = 1,..., N D}.

[0126] As a possible implementation, the determining device can determine the IoU between each target vehicle in the first prediction set and each target vehicle in the target vehicle set in the second image frame according to the following formula.

[0127]

[0128] wherein the IoU represents the IoU. AUB represents the overlapping area of the first target vehicle in the first prediction set and the first target vehicle in the target vehicle set in the second image frame. A∩B represents the intersection area of the first target vehicle in the first prediction set and the first target vehicle in the target vehicle set in the second image frame.

[0129] S604, the determining device determines the first vehicle in each image frame according to the IoU and a preset computer algorithm to obtain the position information of the first vehicle in the plurality of image frames.

[0130] wherein the preset computer algorithm can be the KM algorithm.

[0131] As a possible implementation, the determining device can use the IoU as the relationship weight, use the preset computer algorithm to obtain the best matching relationship between each target vehicle in the first prediction set and each target vehicle in the target vehicle set in the second image frame, and obtain the position information of the first vehicle in the plurality of image frames according to the best matching relationship.

[0132] It should be noted that, in the case that each target vehicle in the target vehicle set in the second image frame is D = {D j’ j = 1,..., N D}, and each target vehicle in the target vehicle set in the first image frame can be T = {t i’ i = 1,..., N T}, if D j is not matched with any T i , a new tracking target is created, an ID is assigned, and a Kalman filter is created. If T i is not matched with any D j , the tracking target is deleted from the tracking library, and the motion trajectory of T i is no longer determined.

[0133] In some embodiments, in order to avoid the interference of noise, the newly created tracking target needs to be matched for N times in succession to be saved, otherwise it is deleted; the tracking target to be deleted needs to be not matched for N frames in succession to be deleted, otherwise it is retained.

[0134] A possible embodiment, as shown in Figure 9 To determine the first driver corresponding to the first vehicle, the determination method of the application can specifically include the following S701-S703.

[0135] S701, the determining device determines the distance between each vehicle and each driver in each image frame.

[0136] As a possible implementation manner, the determining device can determine the distance between each vehicle and each driver according to the following formula two. For example, the formula two can be:

[0137]

[0138] wherein, D i,j represents the distance between vehicle i and driver j. nmv=(nmv i,x ,nmv i,y ), i=1,...,N nmv represents the position of the upper edge center point of the vehicle rectangular frame. Head j =(head j,x ,head j,y ), j=1,...,N head represents the position of the upper edge center point of the driver rectangular frame; Img w and Img h represent the width and height of the current image frame.

[0139] It should be noted that the position of the upper edge center point of the driver rectangular frame can also be the position of the upper edge center point of the target limb rectangular frame.

[0140] S702, the determining device processes the distance between each vehicle and each driver according to a preset computer algorithm to obtain the corresponding relationship between each vehicle and each driver.

[0141] As a possible implementation manner, the determining device can use the distance between each vehicle and each driver as a relationship weight, use a preset computer algorithm to determine the confidence of each vehicle and each driver, and obtain the corresponding relationship between each vehicle and each driver according to the confidence of each vehicle and each driver.

[0142] For example, after determining the confidence of each vehicle and each driver, the determining apparatus can determine the driver with the highest confidence as the driver corresponding to each vehicle.

[0143] S703, the determining apparatus determines the first driver corresponding to the first vehicle from the correspondence between each vehicle and each driver.

[0144] A possible embodiment, as shown in Figure 10 To determine whether the first vehicle has abnormal behavior, the above S406 of the present application can specifically include the following S801-S802.

[0145] S801, the determining apparatus obtains calibration data.

[0146] The calibration data includes prescribed driving direction and signal light data. The driving direction can include east, west, south, north, or northwest, southwest, southeast, and northeast. The signal light data includes first type signals and second type signals. The first type signals can be a no-go signal (e.g., a red light signal), and the second type signals can be a go signal (e.g., a green light signal).

[0147] As a possible implementation, the determining apparatus is provided with an input device (such as a keyboard), and the determining apparatus can receive the input operation of the operator through the input device and obtain the calibration data according to the input operation of the operator.

[0148] As a possible implementation, the determining apparatus can also analyze the target region in the plurality of image frames to obtain the calibration data.

[0149] S802, if the motion trajectory of the first vehicle is detected to be different from the prescribed driving direction, or the signal light data is detected to be the first type signal, and the length of the motion trajectory of the first vehicle is greater than a fourth threshold value, the determining apparatus determines that the first vehicle has abnormal behavior.

[0150] The fourth threshold value can be set as needed. For example, it can be 150 pixels high.

[0151] In an example, when the prescribed driving direction is east, if the motion trajectory of the first vehicle is detected to be west, it is determined that the first vehicle has abnormal behavior. For another example, when the signal light data is the first type signal, if the length of the motion trajectory of the first vehicle is detected to be 200 pixels high, it is determined that the first vehicle has abnormal behavior.

[0152] A possible embodiment, in order to determine whether the driver has abnormal behavior, the present application can also include the following S1-S2.

[0153] S1, the determining apparatus performs limb category detection on each image frame to obtain category information of each driver target limb in each image frame.

[0154] The category information is used to indicate whether the driver wears a helmet.

[0155] As a possible implementation, the determining apparatus inputs each image frame into a head detection model to obtain the category information of each driver target limb in each image frame.

[0156] The category information includes first category information and second category information. The first category information is used to indicate that the driver wears a helmet. The second category information is used to indicate that the driver does not wear a helmet.

[0157] The determining apparatus can determine that the driver wears a helmet when the category information of the driver target limb is the first category information. When the category information of the driver target limb is the second category information, it is determined that the driver does not wear a helmet.

[0158] S2, the determining apparatus determines that the driver has abnormal behavior when the category information indicates that the driver does not wear a helmet.

[0159] As a possible embodiment, in order to determine whether the driver has abnormal behavior, the present application can also include the following S11-S13.

[0160] S11, the determining apparatus performs license plate frame detection on each image frame to obtain position information of the license plate frame of each vehicle in each image frame.

[0161] As a possible implementation, the determining apparatus inputs each image frame into a license plate detection model to obtain the position information of the license plate frame of each vehicle in each image frame.

[0162] The license plate detection model can be set as needed. For example, it can be YOLOV5-s model.

[0163] S12, the determining apparatus performs perspective change processing on each license plate frame according to the position information of the license plate frame of each vehicle.

[0164] As a possible implementation, the determining apparatus can determine the coordinates of the four vertices of the license plate frame on the original image and the transformed image, and perform perspective change processing on each license plate frame according to the coordinates of the four vertices of the license plate frame on the original image and the transformed image.

[0165] It should be noted that the coordinates on the original image are obtained by the license plate detection model, and the converted coordinates can be specified by the user. The calculation process of the perspective change can refer to the prior art, and will not be described here.

[0166] S13, the determining device performs recognition processing on the license plate frame of each vehicle subjected to the perspective change processing, to obtain the license plate number of each vehicle.

[0167] As a possible implementation manner, the determining device inputs each image frame into a license plate recognition model to obtain the license plate number of each vehicle.

[0168] The license plate recognition model can be set as needed. For example, a CRNN algorithm model can be used.

[0169] The various schemes in the above embodiments of the application can be combined under the premise of no contradiction.

[0170] The embodiments of the application can divide the function modules or function units of the determining device according to the above method examples. For example, each function module or function unit can be divided according to each function, or two or more functions can be integrated into one processing module. The above integrated module can be realized in the form of hardware or in the form of software function module or function unit. In the embodiments of the application, the division of the module or unit is illustrative, and is only a logical function division. In actual implementation, another division manner can be used.

[0171] In the case of dividing each function module according to each function, Figure 11 A structural schematic diagram of a determining device is shown. The determining device can be a server or a chip applied in the server. The determining device can be used to execute the functions of the server involved in the above embodiments. Figure 11The determination apparatus shown can include: an acquisition unit 901, a detection unit 902, and a determination unit 903. The acquisition unit 901 is configured to acquire video data of a target scene. The video data includes a plurality of image frames, and each image frame includes one or more vehicles and one or more drivers. The detection unit 902 is configured to perform body position detection on each image frame to obtain position information of a target body of each driver in each image frame. The determination unit 903 is configured to determine a height of a target body of a first driver according to the position information of the target body of the first driver, the first driver being any one of the one or more drivers. When the height of the target body of the first driver is less than a first threshold, the determination unit 903 is further configured to perform vehicle position detection on each image frame to obtain position information of a first vehicle in the plurality of image frames, and determine a motion trajectory of the first vehicle in the target scene according to the position information of the first vehicle in the plurality of image frames, the first vehicle being a vehicle corresponding to the first driver. When the height of the target body of the first driver is greater than or equal to the first threshold, the determination unit 903 is further configured to determine the motion trajectory of the first vehicle in the target scene according to the position information of the target body of the first driver in the plurality of image frames. The determination unit 903 is further configured to determine whether the first vehicle has an abnormal behavior according to the motion trajectory of the first vehicle in the target scene.

[0172] In a possible design, the position information of the object includes pixel coordinates of vertices of a minimum bounding rectangle of the object. The determination unit 903 is specifically configured to: determine a maximum value of a vertical coordinate and a minimum value of the vertical coordinate among the pixel coordinates of the vertices of the minimum bounding rectangle of the target body of the first driver, and determine the height of the target body of the first driver according to a difference between the maximum value and the minimum value of the vertical coordinate.

[0173] In a possible design, the detection unit 902 is specifically configured to: perform position detection on the plurality of vehicles in the first image frame to obtain a target vehicle set in the first image frame, the target vehicle set including a plurality of target vehicles and historical position information corresponding to the plurality of target vehicles, and the height of the target vehicle being greater than a third threshold. The determination unit 903 is specifically configured to: predict position information of each target vehicle in a second image frame according to a Kalman filter and the target vehicle set in the first image frame to obtain a first prediction set, the shooting time of the second image frame being later than that of the first image frame; perform position detection on the plurality of vehicles in the second image frame to obtain a target vehicle set in the second image frame; determine an intersection over union of each target vehicle in the first prediction set and each target vehicle in the target vehicle set in the second image frame; and determine the first vehicle in each image frame according to the intersection over union and a preset computer algorithm to obtain the position information of the first vehicle in the plurality of image frames.

[0174] In a possible design, the determining unit 903 is further configured to determine the distance between each vehicle and each driver in each image frame, process the distance between each vehicle and each driver according to a preset computer algorithm to obtain a corresponding relationship between each vehicle and each driver, and determine the first driver corresponding to the first vehicle from the corresponding relationship between each vehicle and each driver.

[0175] In a possible design, the determining unit 903 is specifically configured to obtain calibration data, wherein the calibration data comprises a specified driving direction and signal lamp data; and determine that the first vehicle has abnormal behavior when it is detected that the motion trajectory of the first vehicle is different from the specified driving direction, or the signal lamp data is of a first type of signal and the length of the motion trajectory of the first vehicle is greater than a fourth threshold.

[0176] In a possible design, the detecting unit 902 is further configured to perform body category detection on each image frame to obtain category information of a target body of each driver in each image frame, wherein the category information is used to indicate whether the driver wears a helmet; and the determining unit 903 is further configured to determine that the driver has abnormal behavior when the category information indicates that the driver does not wear a helmet.

[0177] In a possible design, the detecting unit 902 is further configured to perform license plate frame detection on each image frame to obtain position information of a license plate frame of each vehicle in each image frame, perform perspective change processing on each license plate frame according to the position information of the license plate frame of each vehicle, and perform identification processing on each license plate frame subjected to the perspective change processing to obtain a license plate number of each vehicle.

[0178] The embodiments of the present application further provide a computer readable storage medium. All or part of the processes in the above method embodiments can be instructed by a computer program to relevant hardware to complete, the program can be stored in the above computer readable storage medium, and the program can include the processes of the above method embodiments when executed. The computer readable storage medium can be an internal storage unit of the determination apparatus (including the data sending end and / or the data receiving end) of any of the preceding embodiments, such as a hard disk or a memory of the determination apparatus. The computer readable storage medium can also be an external storage device of the terminal device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device. Further, the computer readable storage medium can include both the internal storage unit and the external storage device of the determination apparatus. The computer readable storage medium is used to store the computer program and other programs and data required by the determination apparatus. The computer readable storage medium can also be used to temporarily store data that has been output or will be output.

[0179] It should be noted that the terms "first" and "second" and the like in the specification, claims and drawings of the present application are used to distinguish different objects, and are not used to describe a particular order. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but can optionally include steps or units not listed, or can optionally include other steps or units inherent to the process, method, product or device.

[0180] It should be understood that in the present application, "at least one" means one or more, "multiple" means two or more, "at least two" means two or three and more, and "and / or" is used to describe the association relationship of the associated objects, which means that there can be three relationships, for example, "A and / or B" can mean that there are three cases of only A, only B and A and B at the same time, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can mean a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b and c can be single or multiple.

[0181] Those skilled in the art can clearly understand the above-mentioned technical solutions from the description of the above-embodiments. For the convenience and brevity of description, only the division of the above-mentioned functional modules is taken as an example. In actual application, the above-mentioned functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

[0182] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, another division mode can be used. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0183] The units described as separate components can or can not be physically separated, and the components shown as units can be one physical unit or multiple physical units, that is, they can be located in one place or distributed in multiple different places. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0184] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0185] When the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present application essentially or the parts that make contributions to the prior art or all or part of the technical solutions can be embodied in the form of a software product. The software product is stored in a storage medium, including a plurality of instructions for causing an apparatus (which can be a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the method described in the embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a ROM, a RAM, a magnetic disk or an optical disk, and various program code storage media.

[0186] The above merely provides the specific implementation of the present application, but the protection scope of the present application is not limited to this. Any change or replacement within the technical scope disclosed by the present application should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method of determining abnormal behavior, characterized by, The method comprises: acquiring video data of a target scene; the video data comprises a plurality of image frames, and each image frame comprises one or more vehicles and one or more drivers; performing limb position detection on each image frame to obtain position information of a target limb of each driver in each image frame; determining a height of the target limb of a first driver according to the position information of the target limb of the first driver; the first driver is any one of the one or more drivers; in a case where the height of the target limb of the first driver is less than a first threshold, performing vehicle position detection on each image frame to obtain position information of a first vehicle in the plurality of image frames, and determining a motion trajectory of the first vehicle in the target scene according to the position information of the first vehicle in the plurality of image frames; the first vehicle is a vehicle corresponding to the first driver; in a case where the height of the target limb of the first driver is greater than or equal to the first threshold, determining the motion trajectory of the first vehicle in the target scene according to the position information of the target limb of the first driver in the plurality of image frames; determining whether the first vehicle has an abnormal behavior according to the motion trajectory of the first vehicle in the target scene; the position information of an object comprises top pixel coordinates of a minimum bounding rectangle of the object; the determining of the height of the target limb of the first driver according to the position information of the target limb of the first driver comprises: determining a maximum value and a minimum value of a vertical coordinate among the top pixel coordinates of the minimum bounding rectangle of the target limb of the first driver; determining the height of the target limb of the first driver according to a difference between the maximum value and the minimum value of the vertical coordinate.

2. The method of claim 1, wherein, the performing of the vehicle position detection on each image frame to obtain the position information of the first vehicle in the plurality of image frames comprises: performing position detection on the plurality of vehicles of a first image frame to obtain a target vehicle set in the first image frame, the target vehicle set comprising a plurality of target vehicles and historical position information corresponding to the plurality of target vehicles; the height of the target vehicle is greater than a third threshold; predicting position information of each target vehicle in a second image frame according to a Kalman filter and the target vehicle set in the first image frame to obtain a first prediction set; the shooting time of the second image frame is located after the first image frame; performing position detection on the plurality of vehicles of the second image frame to obtain a target vehicle set in the second image frame; determining an intersection over union of each target vehicle of the first prediction set and each target vehicle of the target vehicle set in the second image frame; determining the first vehicle in each image frame according to the intersection over union and a preset computer algorithm to obtain the position information of the first vehicle in the plurality of image frames.

3. The method according to claim 1 or 2, characterized in that, The method further comprises: determining distances between each vehicle and each driver in each image frame; According to the preset computer algorithm, the distance between each vehicle and each driver is processed to obtain a corresponding relationship between each vehicle and each driver; From the corresponding relationship between each vehicle and each driver, the first driver corresponding to the first vehicle is determined.

4. The method of claim 1, wherein, The method further comprises: Obtaining calibration data; the calibration data includes a specified driving direction and signal lamp data; If the movement trajectory of the first vehicle is detected to be different from the specified driving direction, or the signal lamp data is detected to be of a first type, and the length of the movement trajectory of the first vehicle is greater than a fourth threshold, it is determined that the first vehicle has abnormal behavior.

5. The method of claim 1, wherein, The method further comprises: Performing limb category detection on each image frame to obtain category information of each driver target limb in each image frame; the category information is used to indicate whether the driver wears a helmet; In the case where the category information indicates that the driver does not wear a helmet, it is determined that the driver has abnormal behavior.

6. The method of claim 1, wherein, The method further comprises: Performing license plate frame detection on each image frame to obtain position information of the license plate frame of each vehicle in each image frame; Performing perspective change processing on each license plate frame according to the position information of the license plate frame of each vehicle; Performing identification processing on each license plate frame of each vehicle subjected to perspective change processing to obtain the license plate number of each vehicle.

7. An abnormal behavior determining apparatus characterized by comprising: The device comprises an acquisition unit, a detection unit, and a determination unit; The acquisition unit is configured to acquire video data of a target scene; the video data comprises a plurality of image frames, and each image frame comprises one or more vehicles and one or more drivers; The detection unit is configured to perform limb position detection on each image frame to obtain position information of each driver target limb in each image frame; The determination unit is configured to determine the height of the first driver target limb according to the position information of the first driver target limb; The first driver is any one of the one or more drivers; The determination unit is further configured to, in the case where the height of the first driver target limb is less than a first threshold, perform vehicle position detection on each image frame to obtain position information of a first vehicle in the plurality of image frames, and determine a movement trajectory of the first vehicle in the target scene according to the position information of the first vehicle in the plurality of image frames; the first vehicle is a vehicle corresponding to the first driver; The determination unit is further configured to, in the case where the height of the first driver target limb is greater than or equal to the first threshold, determine the movement trajectory of the first vehicle in the target scene according to the position information of the first driver target limb in the plurality of image frames; The determination unit is further configured to determine whether the first vehicle has abnormal behavior according to the movement trajectory of the first vehicle in the target scene. The position information of one object includes top pixel coordinates of a minimum bounding rectangle of the one object; and the determining unit is specifically configured to: determine a maximum value and a minimum value of a vertical coordinate among top pixel coordinates of a minimum bounding rectangle of the first driver target limb according to the position information of the first driver target limb; determine a height of the first driver target limb according to a difference between the maximum value and the minimum value of the vertical coordinate.

8. A computer readable storage medium / computer program product, characterized in that, The readable storage medium / computer program product stores instructions which, when executed, implement the method of any one of claims 1-6.

9. An electronic device, comprising: comprise: a processor, and a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement the method of any one of claims 1-6.

Images