Method and device for identifying vehicle state in emergency parking strip

Abstract

The invention provides a method and a device for identifying the state of a vehicle in an emergency parking strip. The method comprises the following steps of: acquiring monitoring video stream data including the emergency parking strip by using video monitoring equipment; constructing a projection plane, and projecting the monitoring image sequence to the projection plane in a homogeneous transformation mode to obtain a projection image sequence; and calculating the actual distance of the unit coordinates in the projection plane based on the actual length of the dotted line in the emergency parking strip, obtaining the real state vector of each target vehicle in the projection image sequence based on the actual distance of the unit coordinates, and judging the vehicle state of the corresponding target vehicle based on the real state vector of each target vehicle. According to the scheme, the monitoring video stream data is acquired and decoded, the image is subjected to preprocessing, target recognition, projection transformation and coordinate calculation, and the real state vector of the vehicle is acquired by using Kalman filtering, so that the states of the vehicle, such as driving in, stopping, driving out and retrograde running, in the emergency parking strip are judged, and accurate recognition is carried out.

Description

Technical Field

[0001] The present application relates to the field of image recognition, and in particular to a method and device for recognizing the status of a vehicle in an emergency parking zone. Background Art

[0002] In the modern transportation network, tunnels, expressways and first-class highways are undoubtedly important arteries of transportation. They carry a large amount of traffic and play a vital role in promoting economic exchanges and personnel exchanges between regions. In order to maximize the driving safety on these roads, relevant departments usually plan and set up emergency parking lanes on both sides of the road.

[0003] The existence of emergency parking lanes is of great significance. When a vehicle encounters an emergency such as mechanical failure, tire blowout, or a driver or passenger suffers from a sudden illness during driving, the driver can park the vehicle smoothly in the emergency parking lane in the first time, thereby temporarily leaving the normal driving lane and avoiding direct obstruction of smooth traffic due to the vehicle staying in the middle of the road. At the same time, it can also greatly reduce the risk of accidents such as collisions caused by subsequent vehicles due to failure to avoid the vehicle in time, providing a key safety guarantee for drivers and passengers as well as passing vehicles.

[0004] However, there are some problems in reality that cannot be ignored. In the emergency stop strip, which was originally used for emergency avoidance, improper or dangerous behaviors of drivers and passengers often occur, and these behaviors are quietly increasing the risk of accidents. For example, some drivers, due to negligence or fluke, will choose to go against the flow in the emergency stop strip, completely ignoring normal traffic rules and the serious consequences of doing so. You must know that in a high-speed traffic environment, it is easy for a reverse vehicle to collide head-on with a normal vehicle or a passing vehicle that uses the emergency stop strip. Once such an accident occurs, the consequences are often disastrous; another common situation is that the vehicle stays in the emergency stop strip for a long time, which may be because the maintenance problem has not been solved for a long time, or the driver and passengers carry out some activities unrelated to emergency avoidance after parking. In this way, the emergency parking lane cannot play its due role normally. Vehicles that need to use the parking lane in subsequent emergency situations may not be able to stop in time due to these vehicles that have been parked for a long time, and may be forced to stop on the driving lane, which will instantly cause congestion and chaos on the road. Drivers of subsequent vehicles may not have time to make correct responses to such emergencies, ultimately leading to secondary accidents such as chain collisions.

[0005] Based on such a complex and severe reality, it is necessary to accurately identify the status of vehicles in the emergency parking lane and quickly trigger the alarm mechanism when improper behaviors such as driving in the wrong direction or staying for a long time without reason are detected. The relevant management departments and passing vehicles can be notified in time to make preparations in advance. This is undoubtedly crucial to effectively prevent the occurrence of secondary accidents and ensure the safe and orderly operation of the entire transportation network. Summary of the invention

[0006] The embodiments of the present application provide a method and device for identifying the state of a vehicle in an emergency parking zone. The method obtains and decodes surveillance video stream data, performs image preprocessing, target recognition, projection transformation, and coordinate calculation, and uses Kalman filtering to obtain the true state vector of the vehicle, thereby accurately identifying the states of the vehicle entering, stopping, exiting, or reversing in the emergency parking zone.

[0007] In a first aspect, an embodiment of the present application provides a method for identifying a vehicle state in an emergency parking zone, the method comprising: Using a video surveillance device to acquire surveillance video stream data including the emergency parking strip, and decoding the surveillance video stream data to obtain a surveillance image sequence including a plurality of image frames; Constructing a projection plane, projecting the monitoring image sequence onto the projection plane in a homogeneous transformation manner to obtain a projection image sequence, wherein the projection plane is on a horizontal plane of the video monitoring device; The actual distance of the unit coordinates in the projection plane is calculated based on the actual length of the dotted line in the emergency parking zone, the real state vector of each target vehicle in the projection image sequence is obtained based on the actual distance of the unit coordinates, and the vehicle state of the corresponding target vehicle is determined based on the real state vector of each target vehicle, wherein the real state vector includes a position vector, a velocity vector and an acceleration vector, and the vehicle state includes entering, stopping, exiting and reversing.

[0008] In a second aspect, an embodiment of the present application provides a vehicle state recognition device in an emergency parking zone, comprising: An acquisition module, used to acquire monitoring video stream data including the emergency parking strip using a video monitoring device, and decode the monitoring video stream data to obtain a monitoring image sequence including a plurality of image frames; A construction module, used for constructing a projection plane, projecting the monitoring image sequence onto the projection plane in a homogeneous transformation manner to obtain a projection image sequence, wherein the projection plane is on a horizontal plane of the video monitoring device; The state recognition module calculates the actual distance of the unit coordinates in the projection plane based on the actual length of the dotted line in the emergency parking zone, obtains the real state vector of each target vehicle in the projection image sequence based on the actual distance of the unit coordinates, and determines the vehicle state of the corresponding target vehicle based on the real state vector of each target vehicle, wherein the real state vector includes a position vector, a velocity vector and an acceleration vector, and the vehicle state includes entering, stopping, exiting and reversing.

[0009] In a third aspect, an embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute a method for identifying the status of a vehicle in an emergency parking zone.

[0010] In a fourth aspect, an embodiment of the present application provides a readable storage medium, in which a computer program is stored. The computer program includes a program code for controlling a process to execute a process, and the process includes a method for identifying the status of a vehicle in an emergency parking zone.

[0011] The main contributions and innovations of the present invention are as follows: The embodiment of the present application obtains the pixel brightness distribution of the image frame and constructs a cumulative distribution function, remaps the pixel brightness level, improves the image light ratio, makes the pixel distribution more uniform, and highlights the details of the bright and dark parts. This effectively reduces the difficulty of vehicle recognition in the dim environment of the emergency parking strip, provides a clear image basis for the subsequent accurate recognition of vehicles, emergency parking strip areas and dotted lines, and improves the accuracy of image information extraction; this solution constructs a projection plane, projects the monitoring image into it through homogeneous transformation, defines the boundary point coordinates and reference coordinates to construct a homogeneous coordinate matrix, and realizes the pixel coordinate conversion. This eliminates the perspective distortion of the image, so that the coordinates in the projection plane have an accurate correspondence with the actual distance. Based on this calculation of the actual distance of the unit coordinate, it lays the foundation for accurately measuring the vehicle position, speed and acceleration, and improves the reliability of vehicle state judgment; this solution obtains the vehicle observation state vector at the actual distance of the unit coordinate, combines Kalman filtering, uses the estimated state vector of the previous frame to calculate the estimated state vector of the current frame, and corrects the observation state vector through the Kalman gain. The noise and jitter problems of the target coordinates are effectively handled, and the real state vectors such as the vehicle position, speed and acceleration are accurately obtained, providing a reliable basis for vehicle state judgment.

[0012] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 is a flow chart of a method for identifying a vehicle state in an emergency parking zone according to an embodiment of the present application; Figure 2 is a schematic diagram of an image frame not processed using a cumulative distribution function according to the present application; Figure 3 is a schematic diagram of an image frame processed using a cumulative distribution function according to the present application; Figure 4 is a schematic diagram of projecting an image frame onto a projection plane; Figure 5 is a structural block diagram of a vehicle state recognition device in an emergency parking zone according to an embodiment of the present application; Figure 6 It is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0014] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with one or more embodiments of this specification. Instead, they are merely examples of devices and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0015] It should be noted that: in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0016] Embodiment 1 The embodiment of the present application provides a method for identifying the state of a vehicle in an emergency parking zone, by acquiring and decoding monitoring video stream data, performing image preprocessing, target recognition, projection transformation and coordinate calculation, and using Kalman filtering to obtain the real state vector of the vehicle, and then accurately identifying the state of the vehicle entering, stopping, exiting, or reversing in the emergency parking zone. Specifically, refer to Figure 1 , the method comprising: Using a video surveillance device to acquire surveillance video stream data including the emergency parking strip, and decoding the surveillance video stream data to obtain a surveillance image sequence including a plurality of image frames; Constructing a projection plane, projecting the monitoring image sequence onto the projection plane in a homogeneous transformation manner to obtain a projection image sequence, wherein the projection plane is on a horizontal plane of the video monitoring device; The actual distance of the unit coordinates in the projection plane is calculated based on the actual length of the dotted line in the emergency parking zone, the real state vector of each target vehicle in the projection image sequence is obtained based on the actual distance of the unit coordinates, and the vehicle state of the corresponding target vehicle is determined based on the real state vector of each target vehicle, wherein the real state vector includes a position vector, a velocity vector and an acceleration vector, and the vehicle state includes entering, stopping, exiting and reversing.

[0017] In some specific embodiments, before projecting the monitoring image sequence onto the projection plane, the pixel brightness distribution of each image frame in the monitoring image sequence is obtained, the pixel brightness distribution records the number of times pixels with different brightness levels appear in the corresponding image frame, and a cumulative distribution function is constructed based on the pixel brightness distribution. The cumulative distribution function is then used to remap the brightness level of each pixel in the corresponding image frame.

[0018] Specifically, since emergency parking lanes are generally set up in closed roads such as tunnels and highways, the surveillance video streams obtained by fixed cameras in these closed roads will result in poor image presentation due to factors such as light and weather. Therefore, in order to ensure that the pixel distribution of the image frame is more uniform, more details of the bright and dark areas can be obtained.

[0019] Specifically, the brightness level of each pixel in the image frame is traversed to obtain the pixel brightness distribution of each image frame. The cumulative distribution function constructed based on the pixel brightness distribution represents the proportion of the number of pixels with different brightness levels in the same image frame to the number of all pixels, which is expressed by the formula:

[0020] Among them, CDF is the pixel brightness distribution, i represents any brightness level, j is the number of all brightness levels, represents the number of pixels with brightness level i in the image frame, and N is the total number of pixels in the image frame, so It represents the ratio of the number of pixels at pixel level i in the image frame to the total number of pixels.

[0021] Furthermore, the cumulative distribution function represents the ratio of the number of pixels of different brightness levels in the image frame to the total number of pixels, and the pixel value of the pixel with pixel level i is remapped to the product of the corresponding cumulative distribution function and 255, where i is any pixel level in the image frame.

[0022] Specifically, since the current video surveillance equipment hardware is eight-bit deep, the brightness level is generally [0, 255], that is, 256 pixel levels, so j is 256, so when using the cumulative distribution function to remap the brightness level of each pixel in the corresponding image frame, the formula is expressed as: ×255 Where i is any pixel level, CDF is the cumulative distribution function, is the pixel value after mapping of all pixels at pixel level i.

[0023] Specifically, after the cumulative distribution function is processed, the image can obtain more details of the bright and dark parts. The image frame without the cumulative distribution function is as follows: Figure 2 As shown, Figure 2 The darker areas in the image contain information such as vehicle outlines and license plates that are difficult to distinguish. The image frame processed using the cumulative distribution function is as follows: Figure 3 As shown by Figure 3 It can be seen that the processed image becomes clearer and some dark details can be seen more clearly, which provides a better image basis for subsequent image-based operations such as emergency parking strip area recognition, vehicle recognition, and vehicle status judgment, and improves the accuracy and reliability of these operations.

[0024] In some embodiments, a pre-trained convolutional neural network is used to identify vehicles, emergency parking strip areas, and dashed lines of the emergency parking strip areas in each image frame of the surveillance image sequence, and vehicles whose center points are not within the emergency parking strip area are removed.

[0025] Specifically, a pre-trained convolutional neural network is used to identify vehicles in emergency parking lanes and image frames, and irrelevant vehicles are removed to improve the efficiency of subsequent data processing, optimize the projection effect, and provide more accurate data support for subsequent vehicle status judgment, thereby improving the reliability and accuracy of the entire system.

[0026] In some embodiments, four boundary point coordinates are defined in the projection plane, and the coordinates of the four intersection points of the extended line of the emergency parking strip boundary and the image edge of each image frame in the monitoring image sequence are obtained as reference coordinates. A homogeneous coordinate matrix is ​​constructed based on the reference coordinates and the boundary point coordinates. Based on the homogeneous coordinate matrix, a homogeneous transformation is performed on each pixel point of each image frame to obtain homogeneous coordinates, and then the homogeneous coordinates of each pixel point are converted into two-dimensional coordinates in the projection plane.

[0027] Specifically, the schematic diagram of projecting each image frame onto the projection plane is as follows: Figure 4As shown, that is to say, since each image frame in the present scheme includes an emergency parking strip, and since the two boundary lines of the emergency parking strip are parallel, a homogeneous coordinate matrix can be obtained by using the coordinates of the four intersection points of the extended lines of the two boundary lines with the edge of the image and the coordinates of the four boundary points defined in the projection plane. According to the obtained homogeneous coordinate matrix, each pixel point in the image frame can be mapped to the projection plane, thereby eliminating the perspective distortion of the video surveillance equipment.

[0028] Furthermore, the coordinates of the four boundary points defined in the projection plane are (0,0), (w,0), (0,h), (w,h), and the coordinates of the four boundary points form a coordinate axis, where w is the narrowest width of the emergency parking strip, and h is the pixel height of the image captured by the camera. The formula for obtaining the homogeneous coordinate matrix is ​​expressed as follows:

[0029]

[0030] in, are the 8 parameters of the homogeneous coordinate matrix, and the values ​​of Z and z are set to 1. The default value of is 1, M is the homogeneous coordinate matrix, and the coordinates of the four boundary points and the reference coordinates defined in the projection plane are substituted into the above formula to obtain the values ​​of the eight parameters and thus obtain the homogeneous coordinate matrix.

[0031] After obtaining the homogeneous coordinate matrix, the coordinates of each pixel point are substituted into the homogeneous coordinate matrix and then a two-dimensional transformation is performed to obtain the two-dimensional coordinates of the pixel point on the projection plane.

[0032] Specifically, by determining the correspondence between the four points where the extended line of the emergency parking strip area boundary intersects the image on the original image and the projection plane, a set of equations is established to solve the values ​​of these matrix elements, thereby obtaining a homogeneous coordinate matrix that can accurately map the three-dimensional area where the emergency parking strip is located to the target plane, thereby eliminating the problem of uneven actual distance between pixels caused by perspective distortion of the image.

[0033] In some embodiments, in the step of "calculating the actual distance of the unit coordinates in the projection plane based on the actual length of the dotted line in the emergency parking zone", since the actual length of the dotted line in the emergency parking zone is fixed, the y-axis coordinate values ​​of the uppermost and lowermost edges of the dotted line are obtained in the projection plane, and the actual length of the dotted line is divided by the difference between the y-axis coordinate value of the uppermost edge of the dotted line and the y-axis coordinate value of the lowermost edge of the dotted line to obtain the actual distance of the unit coordinates in the projection plane, and the formula is expressed as follows:

[0034] in, is the actual distance in unit coordinates, is the actual length of the dotted line, is the y-axis coordinate value of the top of the dotted line, It is the y-axis coordinate value of the bottom of the dotted line.

[0035] Specifically, since image distortion has been eliminated in the projection plane, the actual distance represented by each unit coordinate is the same, and the actual distance of each unit coordinate in the projection plane can be calculated by the length of the dotted line in the projection plane, thereby accurately calculating the actual moving distance of the vehicle in the continuous image.

[0036] In some specific embodiments, in a projected image sequence, the observed state vector of each vehicle in the current image frame is first obtained based on the actual distance of the unit coordinates, and then the estimated state vector of the corresponding vehicle in the previous image frame is used to calculate the estimated state vector of each vehicle in the current image frame. The estimated state vector of the first image frame is equal to the observed state vector. Finally, the estimated state vector of each vehicle in the current image frame is used through the Kalman gain to correct the observed state vector of the corresponding vehicle to obtain the true state vector of each vehicle in the current image frame.

[0037] Specifically, the projection plane in this scheme is a two-dimensional plane, so the position vector in this scheme is (x, y) used to indicate the position of the center point of the vehicle, the velocity vector is (vx, vy), vx represents the velocity component of the vehicle in the x direction, vy represents the velocity component of the vehicle in the y direction, and the acceleration vector is (ax, ay), ax represents the acceleration component of the vehicle in the x direction, and ay represents the acceleration component of the vehicle in the y direction.

[0038] Specifically, the observed state vector of the vehicle is calculated based on the difference between the state vectors of the current image frame and the previous image frame. For example, the vehicle position in the previous image frame is (0, 0), and the vehicle position in the current image frame is (0, 1). The actual distance of each unit is 50 cm, so it can be known that the vehicle has moved 50 cm in the y direction, thereby obtaining the state vector of the current image frame. However, in actual detection, there will be a lot of noise and jitter in the video surveillance equipment, and these noises or jitters will cause the detected coordinates to jump instantaneously, and short-term target loss will cause coordinate loss. Therefore, in this scheme, Kalman filtering is used to effectively smooth the coordinate trajectory and predict the position of short-term lost targets, thereby improving tracking stability.

[0039] Specifically, this scheme uses the uniformly accelerated motion equation to design the transfer matrix F, and then calculates the estimated state vector based on the transfer matrix F. The formula of the transfer matrix is ​​expressed as follows:

[0040] in, Indicates the time interval between two adjacent image frames.

[0041] The calculation formula for the estimated state vector is as follows:

[0042] in, represents the estimated state vector of the previous image frame, Represents the estimated state vector of the current image frame.

[0043] In this solution, when a vehicle just enters the emergency parking zone, it is the first frame image relative to the vehicle, and its speed and acceleration are set to 0.

[0044] In this scheme, when calculating the Kalman gain, we must first obtain the observation matrix H and the observation equation To estimate the noise, the formula is as follows:

[0045]

[0046] in, is the observation noise, is the observed state vector of the target vehicle.

[0047] Then the covariance Initialize, the formula is as follows:

[0048] Calculate the prior covariance based on the covariance , the formula is as follows:

[0049] Where Q represents the process noise covariance,

[0050] in, , is the current maximum acceleration absolute value of the target vehicle, and so on.

[0051] The formula for calculating the Kalman gain is as follows:

[0052] Where R is the observation noise, R=2*coordinate standard deviation.

[0053] Finally, the Kalman gain is used to correct the observed state vector using the estimated state vector. The formula is as follows:

[0054] in, is the observed state vector of the current image frame, is the estimated state vector, and K is the Kalman gain.

[0055] After calculating the true state vector of the current image frame, the covariance is updated, and the formula is as follows:

[0056] Among them, I is the true state vector, which is used to ensure the dimension matching of matrix operations and achieve the correct update of covariance, A correction matrix represents the correction of prediction uncertainty by observations. When the Kalman gain K is large (observations are more reliable), the correction is larger, and when K is small (predictions are more reliable), the correction is smaller.

[0057] Furthermore, when the target vehicle is lost in the current image frame, if the target vehicle position coordinates in the previous image frame are at the edge of the emergency parking strip, it is considered that the target vehicle has left the parking strip, otherwise The predicted value is still generated through the state transition equation , until it exceeds the set tracking threshold, this solution passes The recursive update of can achieve continuous tracking of the vehicle's motion state and noise suppression.

[0058] In some embodiments, when a target vehicle is added to the current frame, the target vehicle is identified as entering; when the speed of the target vehicle is 0 and the acceleration is 0, the target vehicle is identified as stopping; when the speed of the target vehicle is negative and the acceleration is positive, the target vehicle is identified as going in the opposite direction; when the speed of the target vehicle is positive and the acceleration is positive, the target vehicle is identified as leaving.

[0059] Embodiment 2 Based on the same idea, refer to Figure 5 The present application also proposes a vehicle status recognition device in an emergency parking zone, comprising: An acquisition module, used to acquire monitoring video stream data including the emergency parking strip using a video monitoring device, and decode the monitoring video stream data to obtain a monitoring image sequence including a plurality of image frames; A construction module, used for constructing a projection plane, projecting the monitoring image sequence onto the projection plane in a homogeneous transformation manner to obtain a projection image sequence, wherein the projection plane is on a horizontal plane of the video monitoring device; The state recognition module calculates the actual distance of the unit coordinates in the projection plane based on the actual length of the dotted line in the emergency parking zone, obtains the real state vector of each target vehicle in the projection image sequence based on the actual distance of the unit coordinates, and determines the vehicle state of the corresponding target vehicle based on the real state vector of each target vehicle, wherein the real state vector includes a position vector, a velocity vector and an acceleration vector, and the vehicle state includes entering, stopping, exiting and reversing.

[0060] Embodiment 3 This embodiment also provides an electronic device, referring to Figure 6 , comprises a memory 404 and a processor 402, wherein the memory 404 stores a computer program, and the processor 402 is configured to run the computer program to execute the steps in any of the above method embodiments.

[0061] Specifically, the processor 402 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0062] Among them, the memory 404 may include a large capacity memory 404 for data or instructions. For example, but not limitation, the memory 404 may include a hard disk drive (HDD), a floppy disk drive, a solid state drive (SSD), a flash memory, an optical disk, a magneto-optical disk, a tape, or a universal serial bus (USB) drive, or a combination of two or more of these. In appropriate cases, the memory 404 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 404 may be inside or outside the data processing device. In a specific embodiment, the memory 404 is a non-volatile memory. In a specific embodiment, the memory 404 includes a read-only memory (ROM) and a random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (Programmable Read-Only Memory, PROM for short), an erasable PROM (Erasable Programmable Read-Only Memory, EPROM for short), an electrically erasable PROM (Electrically Erasable Programmable Read-Only Memory, EEPROM for short), an electrically alterable ROM (Electrically Alterable Read-Only Memory, EAROM for short) or a flash memory (FLASH) or a combination of two or more of these. In appropriate circumstances, the RAM may be a static random access memory (SRAM) or a dynamic random access memory (DRAM), wherein the DRAM may be a fast page mode dynamic random access memory 404 (FPMDRAM), an extended data output dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0063] The memory 404 may be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor 402 .

[0064] The processor 402 reads and executes the computer program instructions stored in the memory 404 to implement any one of the vehicle status recognition methods in the emergency stop zone in the above embodiments.

[0065] Optionally, the electronic device may further include a transmission device 406 and an input / output device 408 , wherein the transmission device 406 is connected to the processor 402 , and the input / output device 408 is connected to the processor 402 .

[0066] The transmission device 406 can be used to receive or send data via a network. The specific examples of the above-mentioned network may include a wired or wireless network provided by a communication provider of the electronic device. In one example, the transmission device includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 406 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0067] The input / output device 408 is used to input or output information. In this embodiment, the input information may be monitoring video stream data, etc., and the output information may be the vehicle status of the target vehicle, etc.

[0068] Optionally, in this embodiment, the processor 402 may be configured to perform the following steps through a computer program: Using a video surveillance device to acquire surveillance video stream data including the emergency parking strip, and decoding the surveillance video stream data to obtain a surveillance image sequence including a plurality of image frames; Constructing a projection plane, projecting the monitoring image sequence onto the projection plane in a homogeneous transformation manner to obtain a projection image sequence, wherein the projection plane is on a horizontal plane of the video monitoring device; The actual distance of the unit coordinates in the projection plane is calculated based on the actual length of the dotted line in the emergency parking zone, the real state vector of each target vehicle in the projection image sequence is obtained based on the actual distance of the unit coordinates, and the vehicle state of the corresponding target vehicle is determined based on the real state vector of each target vehicle, wherein the real state vector includes a position vector, a velocity vector and an acceleration vector, and the vehicle state includes entering, stopping, exiting and reversing.

[0069] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0070] In general, various embodiments may be implemented in hardware or dedicated circuits, software, logic, or any combination thereof. Some aspects of the invention may be implemented in hardware, while other aspects may be implemented in firmware or software executed by a controller, microprocessor, or other computing device, but the invention is not limited thereto. Although various aspects of the invention may be shown and described as block diagrams, flow charts, or using some other graphical representation, it should be understood that, as non-limiting examples, the boxes, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuits or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0071] Embodiments of the present invention may be implemented by computer software that is executable by a data processor of a mobile device, such as in a processor entity, or by hardware, or by a combination of software and hardware. Computer software or programs (also referred to as program products) including software routines, applets and / or macros may be stored in any device-readable data storage medium, and they include program instructions for performing specific tasks. A computer program product may include one or more computer executable components configured to perform an embodiment when the program is run. One or more computer executable components may be at least one software code or a portion thereof. In addition, at this point, it should be noted that, for example, Figure 6 Any block of the logic flow in the program may represent program steps, or interconnected logic circuits, blocks and functions, or a combination of program steps and logic circuits, blocks and functions. The software may be stored on physical media such as memory chips or storage blocks implemented within a processor, magnetic media such as hard disks or floppy disks, and optical media such as, for example, DVDs and their data variants, CDs, etc. Physical media are non-transitory media.

[0072] Those skilled in the art should understand that the technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0073] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for identifying vehicle status in an emergency parking zone, characterized in that: The following steps are involved: Using a video surveillance device to acquire surveillance video stream data including the emergency parking strip, and decoding the surveillance video stream data to obtain a surveillance image sequence including a plurality of image frames; Constructing a projection plane, projecting the monitoring image sequence onto the projection plane in a homogeneous transformation manner to obtain a projection image sequence, wherein the projection plane is on a horizontal plane of the video monitoring device; The actual distance of the unit coordinates in the projection plane is calculated based on the actual length of the dotted line in the emergency parking zone, the real state vector of each target vehicle in the projection image sequence is obtained based on the actual distance of the unit coordinates, and the vehicle state of the corresponding target vehicle is determined based on the real state vector of each target vehicle, wherein the real state vector includes a position vector, a velocity vector and an acceleration vector, and the vehicle state includes entering, stopping, exiting and reversing.

2. The method for identifying vehicle status in an emergency parking zone according to claim 1, characterized in that: Before projecting the surveillance image sequence onto the projection plane, the pixel brightness distribution of each image frame in the surveillance image sequence is obtained, the pixel brightness distribution records the number of occurrences of pixels with different brightness levels in the corresponding image frame, a cumulative distribution function is constructed based on the pixel brightness distribution, and the brightness level of each pixel in the corresponding image frame is remapped using the cumulative distribution function.

3. The method for identifying vehicle status in an emergency parking zone according to claim 2, characterized in that: After remapping the brightness level of each pixel in the corresponding image frame using the cumulative distribution function, a pre-trained convolutional neural network is used to identify vehicles, emergency parking areas, and dotted lines of the emergency parking areas in each image frame of the surveillance image sequence, and vehicles whose center points are not within the emergency parking area are removed.

4. The method for identifying vehicle status in an emergency parking zone according to claim 1, characterized in that: Four boundary point coordinates are defined in the projection plane, and the coordinates of the four intersection points of the emergency parking strip boundary extension line and the image edge of each image frame in the monitoring image sequence are obtained as reference coordinates. A homogeneous coordinate matrix is ​​constructed based on the reference coordinates and the boundary point coordinates. Based on the homogeneous coordinate matrix, a homogeneous transformation is performed on each pixel point of each image frame to obtain homogeneous coordinates, and the homogeneous coordinates of each pixel point are converted into two-dimensional coordinates in the projection plane.

5. The method for identifying vehicle status in an emergency parking zone according to claim 1, characterized in that: Get the y-axis coordinate values ​​of the top and bottom edges of the dotted line in the projection plane, and use the actual length of the dotted line to divide the difference between the y-axis coordinate value of the top edge of the dotted line and the y-axis coordinate value of the bottom edge of the dotted line to get the actual distance of the unit coordinate in the projection plane.

6. The method for identifying vehicle status in an emergency parking zone according to claim 1, characterized in that: In the projected image sequence, the observed state vector of each vehicle in the current image frame is obtained based on the actual distance of the unit coordinates, and the estimated state vector of the corresponding vehicle in the previous image frame is used to calculate the estimated state vector of each vehicle in the current image frame. The estimated state vector of each vehicle in the current image frame is first used through the Kalman gain to correct the observed state vector of the corresponding vehicle to obtain the true state vector of each vehicle in the current image frame.

7. The method for identifying vehicle status in an emergency parking zone according to claim 1, characterized in that: When a position vector of a new target vehicle is added in the current frame, the new target vehicle is identified as entering; if the speed of the target vehicle is judged to be 0 and the acceleration is 0 based on the speed vector and acceleration vector of the target vehicle in the current frame, the target vehicle is identified as stopping; if the speed of the target vehicle is judged to be negative and the acceleration is positive based on the speed vector and acceleration vector of the target vehicle in the current frame, the target vehicle is identified as going in the opposite direction; if the speed of the target vehicle is judged to be positive and the acceleration is positive based on the speed vector and acceleration vector of the target vehicle in the current frame, the target vehicle is identified as leaving.

8. A vehicle status recognition device in an emergency parking zone, characterized in that: include: An acquisition module, used to acquire monitoring video stream data including the emergency parking strip using a video monitoring device, and decode the monitoring video stream data to obtain a monitoring image sequence including a plurality of image frames; A construction module, used for constructing a projection plane, projecting the monitoring image sequence onto the projection plane in a homogeneous transformation manner to obtain a projection image sequence, wherein the projection plane is on a horizontal plane of the video monitoring device; The state recognition module calculates the actual distance of the unit coordinates in the projection plane based on the actual length of the dotted line in the emergency parking zone, obtains the real state vector of each target vehicle in the projection image sequence based on the actual distance of the unit coordinates, and determines the vehicle state of the corresponding target vehicle based on the real state vector of each target vehicle, wherein the real state vector includes a position vector, a velocity vector and an acceleration vector, and the vehicle state includes entering, stopping, exiting and reversing.

9. An electronic device comprising a memory and a processor, characterized in that: The memory stores a computer program, and the processor is configured to run the computer program to execute the method for identifying the state of a vehicle in an emergency stop zone according to any one of claims 1 to 7.

10. A readable storage medium, characterized in that: The readable storage medium stores a computer program, wherein the computer program includes a program code for controlling a process to execute a process, wherein the process includes a method for identifying a vehicle state in an emergency stop zone according to any one of claims 1-7.

Images