Motion state detection method, device, equipment, medium and product

Abstract

The invention discloses a motion state detection method and device, equipment, a medium and a product, and relates to the field of intelligent traffic, the method is applied to first equipment, the first equipment is deployed on the roadside, and the method comprises the steps of obtaining a video image collected by camera equipment; target detection is carried out on the video image, first information of a traffic participant contained in the video image is obtained, and the first information comprises pixel coordinates of the chassis center of the traffic participant; converting the pixel coordinates into bird's-eye view angle BEV visual coordinates based on camera equipment parameters; and determining the motion state of the traffic participant according to the BEV visual coordinates. The problem that hardware cost is high due to the fact that an existing motion state detection mode excessively depends on a laser radar is solved.

Description

Technical Field

[0001] The present application relates to the field of intelligent transportation technology, and in particular to a motion state detection method, device, equipment, medium and product. Background Art

[0002] The intelligent traffic roadside perception system provides structured data for networked autonomous vehicles, covering target type, ID, location, speed, acceleration and other contents. Speed ​​and acceleration reflect the motion status information of traffic participants. These data can provide spatiotemporal reference and motion feature description for networked autonomous vehicles, and assist in the realization of collision warning, road condition warning and other functions. However, at present, most of the above data rely on hardware such as LiDAR for collection and calculation, resulting in high hardware costs. How to calculate the accurate speed and acceleration of the target is a difficult problem that needs to be solved urgently. Summary of the invention

[0003] The present application provides a motion state detection method, device, equipment, medium and product, which solves the problem that the current motion state detection method relies too much on laser radar, resulting in high hardware costs.

[0004] In a first aspect, an embodiment of the present application provides a motion state detection method, which is applied to a first device, where the first device is deployed on a roadside, and the method includes:

[0005] Acquire video images captured by a camera device;

[0006] Performing target detection on the video image to obtain first information of a traffic participant contained in the video image, wherein the first information includes pixel coordinates of a chassis center of the traffic participant;

[0007] Based on the camera device parameters, the pixel coordinates are converted into bird's-eye view (BEV) visual coordinates;

[0008] The motion state of the traffic participant is determined based on the BEV visual coordinates.

[0009] The step of performing target detection on the video image to obtain first information of traffic participants contained in the video image includes:

[0010] Using a target detection algorithm to identify and detect traffic participants in the video image, obtain identification ID information of the traffic participant, at least one 3D bounding box, and pixel coordinates of the center of the chassis corresponding to each of the 3D bounding boxes;

[0011] Using the non-maximum suppression (NMS) algorithm, redundant 3D bounding boxes of the detected traffic participants are filtered to obtain a target 3D bounding box corresponding to the traffic participant;

[0012] Continuously tracking the detected traffic participant using a target tracking algorithm to obtain the target 3D bounding box of the traffic participant in different frame images of the video image and the pixel coordinates of the corresponding chassis center;

[0013] The first information includes: the ID information of the traffic participant, the target 3D bounding box and the pixel coordinates of the center of the chassis.

[0014] The pixel coordinates are converted into bird's-eye view (BEV) visual coordinates based on the camera device parameters, including:

[0015] According to the intrinsic parameters and distortion parameters in the camera equipment parameters, the pixel coordinates of the center of the chassis are converted into coordinates in the camera equipment coordinate system;

[0016] Based on the external parameters in the camera device parameters, the coordinates in the camera device coordinate system are converted into coordinates in the Northeastern ENU coordinate system;

[0017] Determine the BEV visual coordinates projected from the coordinates in the ENU coordinate system to the BEV visual angle.

[0018] Wherein, the method further comprises:

[0019] Acquiring a first image;

[0020] The first image is used to calibrate parameters of the camera device to obtain parameters of the camera device, where the parameters of the camera device include intrinsic parameters, distortion parameters, and extrinsic parameters.

[0021] The step of calibrating the camera device using the first image to obtain parameters of the camera device includes:

[0022] In the first image, obtain pixel coordinates of N feature points and world coordinates corresponding to each of the pixel coordinates; N is a positive integer;

[0023] Convert the world coordinates to ENU local coordinates;

[0024] The pixel coordinates and the ENU local coordinates corresponding to the pixel coordinates are input into a camera calibration algorithm to obtain the camera parameters output by the camera calibration algorithm.

[0025] Wherein, determining the motion state of the traffic participant according to the BEV visual coordinates includes:

[0026] Calculating a speed vector of the traffic participant according to the BEV visual coordinates corresponding to two adjacent frames of images;

[0027] Calculating the acceleration vector of the traffic participant according to the calculated two adjacent velocity vectors of the traffic participant;

[0028] filtering the velocity vector and the acceleration vector;

[0029] The motion state of the traffic participant is determined according to the filtered velocity vector and the acceleration vector.

[0030] In a second aspect, an embodiment of the present application provides a motion state detection device, which is applied to a first device, the first device is deployed on the road side, and the device includes:

[0031] An acquisition module, for acquiring video images captured by a camera device;

[0032] A detection module, configured to perform target detection on the video image to obtain first information of a traffic participant contained in the video image, wherein the first information includes pixel coordinates of a chassis center of the traffic participant;

[0033] A conversion module, configured to convert the pixel coordinates into a bird's-eye view (BEV) visual coordinates based on camera device parameters;

[0034] A calculation module is used to determine the motion state of the traffic participant according to the BEV visual coordinates.

[0035] In a third aspect, an embodiment of the present application provides a motion state detection device, comprising a transceiver, a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor implements the motion state detection method as described in the first aspect when executing the computer program.

[0036] In a fourth aspect, an embodiment of the present application provides a readable storage medium having a program or instruction stored thereon, wherein the program or instruction, when executed by a processor, implements the motion state detection method as described in the first aspect.

[0037] In a fifth aspect, an embodiment of the present application provides a computer program product, comprising computer instructions, which, when executed by a processor, implement the motion state detection method as described in the first aspect.

[0038] The beneficial effects of the above technical solution of the present application are:

[0039] In the motion state detection method of the embodiment of the present application, first, the first device obtains the video image captured by the camera device; second, the first device performs target detection on the video image to obtain the first information of the traffic participant contained in the video image, and the first information includes the pixel coordinates of the center of the chassis of the traffic participant; third, the first device converts the pixel coordinates into the bird's-eye view BEV visual coordinates based on the camera device parameters; finally, the first device determines the motion state of the traffic participant according to the BEV visual coordinates. In this way, the sensor involved in the motion state detection method of the embodiment of the present application is only the camera device, which greatly reduces the use cost compared with the existing solution that requires the combination of camera and laser radar. In addition, the motion state of the traffic participant is calculated based on the BEV visual coordinates, so that the calculated parameters representing the motion state are more accurate, so that the motion state of the traffic participant can be determined in a low-cost manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is one of the flow charts of the motion state detection method according to an embodiment of the present application;

[0041] Figure 2 This is a second flow chart of the motion state detection method according to an embodiment of the present application;

[0042] Figure 3 This is a schematic diagram of the process of performing target detection by the AI ​​module in an embodiment of the present application;

[0043] Figure 4 A schematic diagram of the process of camera calibration in an embodiment of the present application;

[0044] Figure 5 A schematic diagram of the coordinate conversion process in an embodiment of the present application;

[0045] Figure 6 A schematic diagram of the working principle of the Kalman filter in an embodiment of the present application;

[0046] Figure 7 A schematic diagram of the structure of a motion state detection device according to an embodiment of the present application;

[0047] Figure 8 This is a schematic diagram of the structure of a motion state detection device according to an embodiment of the present application. DETAILED DESCRIPTION

[0048] In order to make the technical problems, technical solutions and advantages to be solved by the application clearer, the following will be described in detail in conjunction with the accompanying drawings and specific embodiments. In the following description, specific details such as specific configurations and components are provided only to help fully understand the embodiments of the application. Therefore, it should be clear to those skilled in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the application. In addition, for clarity and brevity, the description of known functions and structures has been omitted.

[0049] It should be understood that the references to "one embodiment" or "an embodiment" throughout the specification mean that the specific features, structures, or characteristics associated with the embodiment are included in at least one embodiment of the present application. Therefore, the references to "in one embodiment" or "in an embodiment" appearing throughout the specification do not necessarily refer to the same embodiment. In addition, these specific features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0050] In the various embodiments of the present application, it should be understood that the size of the serial numbers of the following processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0051] In the embodiments provided in the present application, it should be understood that "B corresponding to A" means that B is associated with A, and B can be determined according to A. However, it should also be understood that determining B according to A does not mean determining B only according to A, and B can also be determined according to A and / or other information.

[0052] Below, in conjunction with the accompanying drawings, the motion state detection method, device, equipment, medium and product of the embodiments of the present application are described in detail.

[0053] The embodiment of the present application provides a motion state detection method, which is applied to a first device, wherein the first device is deployed on the road side, for example, the first device may be a road side unit (RSU), such as Figure 1 As shown, the method includes:

[0054] Step 101, obtaining a video image captured by a camera device;

[0055] The "camera device" in the above step 101 is, for example, a camera, etc., and the above "camera device" can be integrated in the first device, or can be independent of the first device and able to communicate with the first device; specifically, the camera device is used to capture video images on the road.

[0056] Step 102, performing target detection on the video image to obtain first information of a traffic participant contained in the video image, wherein the first information includes pixel coordinates of a chassis center of the traffic participant;

[0057] The "chassis center" in the above step 102 is the center of the traffic participant on the road surface or on a plane parallel to the road surface. Here, taking the traffic participant as a pedestrian as an example, the chassis center is the center between the two feet of the pedestrian. Taking the traffic participant as a car as an example, the chassis center may be the center of the plane where the chassis of the car is located.

[0058] The above-mentioned first information may include, in addition to the pixel coordinates of the center of the chassis, a 3D bounding box of the traffic participant and / or the ID of the traffic participant, wherein the 3D bounding box is used to identify the outline of the traffic participant, and the ID is used to identify the traffic participant to achieve tracking of the traffic participant in continuous frame images.

[0059] Step 103, based on the camera device parameters, convert the pixel coordinates into Bird Eye View (BEV) visual coordinates;

[0060] Here, BEV is a technology that converts sensor input to a unified overhead angle for processing. By converting pixel coordinates into BEV visual coordinates, on the one hand, it can improve the positioning accuracy of traffic participants in physical space and provide a solid foundation for subsequent real-time speed and acceleration calculations based on multi-frame image data; on the other hand, it can more intuitively display the motion status of traffic participants, which is especially suitable for scenarios with strict requirements for high-precision motion analysis such as traffic monitoring.

[0061] Step 104 , determining the motion state of the traffic participant according to the BEV visual coordinates.

[0062] Here, the motion state of the traffic participant can be specifically represented by a velocity vector and / or an acceleration vector of the traffic participant.

[0063] The above step 104 determines the motion state of the traffic participant based on the BEV visual coordinates, which can ensure that the speed and acceleration and other parameters of the traffic participant are accurately calculated under this perspective, and provide an efficient and accurate motion analysis tool for traffic monitoring.

[0064] In the motion state detection method of the embodiment of the present application, first, a video image captured by a camera device is obtained; second, target detection is performed on the video image to obtain the first information of the traffic participant contained in the video image, wherein the first information includes the pixel coordinates of the center of the chassis of the traffic participant; third, based on the camera device parameters, the pixel coordinates are converted into bird's eye view (Bird Eye View, BEV) visual coordinates; finally, the motion state of the traffic participant is determined according to the BEV visual coordinates. In this way, on the one hand, the detection of the motion state of the traffic participant only involves the camera device, and there is no need to combine the camera device with a sensor such as a laser radar, thereby reducing the cost of use; on the other hand, compared with directly determining the motion state based on pixel coordinates or corresponding world coordinates, the embodiment of the present application converts the pixel coordinates into BEV visual coordinates and determines the motion state based on the BEV visual coordinates, thereby realizing a more intuitive display of the motion state of the traffic participant and improving the accuracy of the motion state determination.

[0065] As an optional implementation, step 102 includes:

[0066] Using a target detection algorithm to identify and detect traffic participants in the video image, obtain identification ID information of the traffic participant, at least one 3D bounding box, and pixel coordinates of the center of the chassis corresponding to each of the 3D bounding boxes;

[0067] In the above steps, the target detection algorithm is specifically, for example, the YOLOv8 target detection algorithm, which can identify and detect objects in videos or images. Among them, the algorithm can accurately capture the shape, position and boundary of the object, and is particularly suitable for the detection of traffic participants such as vehicles.

[0068] Using a non-maximum suppression (NMS) algorithm, redundant 3D bounding boxes of the detected traffic participants are filtered to obtain a target 3D bounding box corresponding to the traffic participant;

[0069] Continuously tracking the detected traffic participant using a target tracking algorithm to obtain the target 3D bounding box of the traffic participant in different frame images of the video image and the pixel coordinates of the corresponding chassis center;

[0070] The target tracking algorithm in the above steps is an improved multi-target tracking algorithm, such as the OcSort target tracking algorithm, so that the detected traffic participants can be continuously tracked to ensure that even if the traffic participants move in multiple frames, their position changes can still be continuously identified and recorded.

[0071] The first information includes: the ID information of the traffic participant, the target 3D bounding box and the pixel coordinates of the center of the chassis.

[0072] A specific example of the above optional implementation is as follows Figure 3 As shown in the figure, first, the image is acquired, then the image is preprocessed, and then the YOLOv8 algorithm is used to detect the target on the preprocessed image to obtain the shape, position and boundary of the traffic participants in the image; then, the NMS algorithm is used to filter the redundant boundaries of the traffic participants (the borders that identify the traffic participants) to obtain the appropriate detection results; then, the OcSort tracking algorithm is used to continuously track the detected traffic participants to ensure that the traffic participants can continue to identify and record their position changes when they move in multiple frames. That is, the ID of the traffic participant and the 3D frame and chassis center in each frame image are obtained.

[0073] Here, it should be noted that the above optional implementation method can be specifically Figure 2 The AI ​​module is implemented in .

[0074] As an optional implementation, step 103 includes:

[0075] According to the intrinsic parameters and distortion parameters in the camera equipment parameters, the pixel coordinates of the center of the chassis are converted into coordinates in the camera equipment coordinate system;

[0076] Based on the external parameters in the camera device parameters, the coordinates in the camera device coordinate system are converted into coordinates in the Northeastern ENU coordinate system;

[0077] Determine the BEV visual coordinates projected from the coordinates in the ENU coordinate system to the BEV visual angle.

[0078] The above optional implementation method is to first convert the pixel coordinates of the center of the chassis into world coordinates based on the camera device parameters. In this step, the coordinates in the camera device coordinate system can be converted into ENU coordinates, and the ENU coordinates can be further converted into BEV visual coordinates.

[0079] The optional implementation method is to multiply the pixel coordinates of the chassis center by the camera parameters to convert the pixel coordinates of the chassis and the chassis center of the traffic participant into coordinates in the ENU coordinate system, and further project the coordinates in the ENU coordinate system to the BEV vision to obtain the BEV vision coordinates. Figure 2 As a specific example, if the camera is a camera, such as Figure 5As shown in the figure, first, the pixel coordinates are multiplied with the camera intrinsic parameters and distortion parameters to obtain the camera coordinate system coordinates; then, the camera coordinate system coordinates are multiplied with the inverse matrix of the camera extrinsic parameters to obtain the ENU coordinates corresponding to the pixel coordinates, and the ENU coordinates are further projected to the BEV perspective to obtain the BEV visual coordinates.

[0080] As an optional implementation, before step 101, the method further includes:

[0081] Acquire a first image; for example, the first image is a road image taken by a camera;

[0082] The first image is used to calibrate parameters of the camera device to obtain parameters of the camera device, where the parameters of the camera device include intrinsic parameters, distortion parameters, and extrinsic parameters.

[0083] Compared with the method of performing parameter calibration using a chessboard or the like, the above optional implementation method performs parameter calibration on the camera device based on the acquired first image, thereby simplifying the parameter calibration process.

[0084] As a specific implementation manner, calibrating the camera device using the first image to obtain the camera device parameters includes:

[0085] In the first image, pixel coordinates of N feature points and world coordinates corresponding to each of the pixel coordinates are obtained; N is a positive integer; here, the world coordinates are, for example, WGS84 coordinates, where WGS84 coordinates are a coordinate system established for use with the Global Positioning System (GPS).

[0086] The world coordinates are converted into ENU local coordinates; wherein the ENU coordinate system is also called the station center coordinate system, which is a coordinate system with the measuring station as the origin, that is, the prepared base is used to locate the point and observe and measure. This step can improve the accuracy of calibration by converting the world coordinates into ENU local coordinates.

[0087] The pixel coordinates and the ENU local coordinates corresponding to the pixel coordinates are input into a camera calibration algorithm to obtain the camera parameters output by the camera calibration algorithm.

[0088] An example of the above optional implementation is as follows Figure 4As shown, first, feature points are collected, for example, 20 to 30 feature points are collected in the first image; secondly, the pixel coordinates and world coordinates (latitude and longitude) of the collected feature points are obtained respectively; thirdly, a certain feature point is selected as the reference point; then, based on the coordinate conversion algorithm, the WGS84 coordinates (world coordinates) are converted to ENU coordinates; then, the pixel coordinates and the ENU coordinates matching each pixel coordinate are input into the camera calibration algorithm for camera calibration, and finally, the camera intrinsic parameters, extrinsic parameters and distortion (parameters) output by the camera calibration algorithm are obtained. In order to convert the chassis center of the traffic participant from pixel coordinates to world coordinates based on the calibrated parameters in the future, this conversion process is based on the camera imaging model and the principle of geometric projection, and the pixel coordinates are mapped to the ENU local coordinate system of the actual physical space through data calculation.

[0089] As an optional implementation, the above step 104 includes:

[0090] (1) calculating the speed vector of the traffic participant according to the BEV visual coordinates corresponding to two adjacent frames of images;

[0091] Speed ​​is the rate of change of position over time and can be expressed as the displacement of a traffic participant between two points in time divided by the time interval:

[0092]

[0093] in:

[0094] ν(t)=(ν,ν y ,ν z ) is the velocity vector at time t;

[0095] r(t) and r(t-Δt) are the position vectors at time t and time t-Δt;

[0096] Δt is the time interval between two time points.

[0097] (2) calculating the acceleration vector of the traffic participant based on the two adjacent velocity vectors of the traffic participant calculated;

[0098] Acceleration is the rate of change of velocity over time and can also be calculated from the change in velocity:

[0099]

[0100] in:

[0101] a(t)=(a,a y ,a z ) is the acceleration vector at time t;

[0102] ν(t) and ν(t-Δt) are the velocity vectors at time t and time t-Δt;

[0103] Δt is the time interval between two time points.

[0104] (3) filtering the velocity vector and the acceleration vector;

[0105] Since noise and uncertainty may affect the accuracy of the results in the calculation of velocity and acceleration, the above step (3) uses a Kalman filter as a filtering module to smooth the calculation of velocity and acceleration and improve the estimation accuracy of the target motion state. The Kalman filter is a recursive algorithm that can estimate the state of the system through the measured noise and the dynamic model of the system. Figure 6 As shown in Figure 1, the Kalman filter consists of two main steps: prediction and update. The prediction step estimates the current state based on the previous state, and the update step corrects the predicted state using new measurements.

[0106] (4) Determine the motion state of the traffic participant based on the filtered velocity vector and the acceleration vector.

[0107] As an example, Figure 2 As shown, the motion state detection method of the embodiment of the present application mainly involves four modules: an AI module, a camera calibration module, a BEV conversion module and a speed calculation module; each module is described below:

[0108] AI module: Identify and detect objects in videos or images through the YOLOv8 target detection algorithm. This algorithm can accurately capture the shape, position and boundaries of objects, and is particularly suitable for the detection of traffic participants such as vehicles. Next, the system uses the NMS algorithm to filter out redundant frames to obtain appropriate detection results. Finally, the system uses the OcSort target tracking algorithm to continuously track the detected targets, ensuring that even if the target moves in multiple frames, its position changes can still be continuously identified and recorded. The overall process of the AI ​​module is as follows: Figure 3 shown.

[0109] Camera calibration module: In order to realize the conversion from image coordinates to world coordinates, the system needs to perform camera calibration. During the calibration process, the system collects the pixel coordinates of 20-30 feature points in the image and their corresponding world coordinates, and converts these world coordinates into ENU local coordinates. By matching the pixel coordinates and the ENU local coordinates, the system performs camera calibration to obtain the internal and external parameters and distortion parameters of the camera. Based on these parameters, the system can convert the chassis center point of the target from pixel coordinates to world coordinates. This conversion process is based on the camera imaging model and the principle of geometric projection, and uses mathematical calculations to map the pixel coordinates to the ENU local coordinate system of the actual physical space. The overall process is as follows: Figure 4 shown.

[0110] BEV conversion module: multiply the pixel points detected by the AI ​​module with the internal and external parameters of the camera output by the calibration module, and finally project the chassis and chassis center of the target to the BEV vision. Based on the BEV vision, the system calculates the motion information of the target. The overall process is as follows: Figure 5 shown.

[0111] Speed ​​calculation module: Calculates the speed and acceleration of the target in the world coordinate system based on multiple frames of data. Since the system can continuously obtain the spatial position of the target, the motion state of the target can be calculated using physical formulas using these position change data.

[0112] The implementation process of the motion state detection method based on the above four modules is simply described as including: inputting the image to the AI ​​module to obtain the target 3D bounding box, chassis center, and ID information; then passing through the camera calibration module, the target chassis is converted to BEV (bird's eye view) with the chassis center; finally, through the speed module, key information is obtained: speed and acceleration. The above embodiments provide efficient and accurate solutions for application scenarios such as traffic monitoring, autonomous driving, and motion analysis. Specifically, the specific implementation process of the above embodiments includes:

[0113] First, acquiring a first image, where the first image is used to calibrate the camera;

[0114] Second, the first image is input into a camera calibration module to calibrate the camera and obtain camera parameters;

[0115] Third, collect video images;

[0116] Fourth, the collected video images are input into the AI ​​module, which performs target recognition, redundant frame filtering and continuous target tracking on the images to obtain the 3D frame, chassis center and ID of the target (traffic participant);

[0117] Fifth, the camera parameters and chassis center are input into the BEV conversion module to obtain the BEV visual coordinates. Specifically, based on the product transformation of the chassis center and the camera parameters, the coordinates of the chassis center in the ENU coordinate system are obtained, and the coordinates in the ENU coordinate system are projected to the BEV perspective to obtain the BEV visual coordinates.

[0118] Sixth, the BEV visual coordinates are input into the speed calculation module to calculate the speed and acceleration of the target;

[0119] Seventh, obtain the target's 3D box, chassis center, ID, velocity, and acceleration.

[0120] In the motion state detection method of the embodiment of the present application, firstly, accurate target detection is performed by using the YOLOv8 algorithm, and combined with NMS to suppress redundant frames, and then OcSort is used to achieve cross-frame continuous tracking of the target; high-precision target detection and continuous tracking can be achieved in complex traffic scenes. This combination not only improves the recognition accuracy of dynamic targets, but also effectively tracks the motion trajectory of the target in multiple frames. Secondly, the WGS84 coordinates (world coordinates) are converted to ENU local coordinates, and then matched with the pixel coordinates in the image, and the internal and external parameters are obtained through camera calibration, and finally the accurate conversion from pixel coordinates to world coordinates is achieved; on this basis, the system further converts the converted world coordinates into BEV vision. This process not only improves the positioning accuracy of the object in the physical space, but also provides a solid foundation for the subsequent real-time speed and acceleration calculation based on multi-frame image data. Through this precise conversion, the system can more intuitively display the motion state of the target from the BEV perspective, which is particularly suitable for scenes such as traffic monitoring that have strict requirements for high-precision motion analysis. In this way, the embodiments of the present application can capture video images using only a camera device, and then analyze the video images based on an algorithm to obtain the precise motion status of traffic participants, thereby reducing the cost of detecting the motion status.

[0121] The embodiment of the present application also provides a motion state detection device, which is applied to a first device, and the first device is deployed on the road side, such as Figure 7 As shown, the device comprises:

[0122] An acquisition module 701 acquires a video image captured by a camera device;

[0123] A detection module 702 is used to perform target detection on the video image to obtain first information of a traffic participant contained in the video image, wherein the first information includes pixel coordinates of a chassis center of the traffic participant;

[0124] A conversion module 703, configured to convert the pixel coordinates into bird's-eye view (BEV) visual coordinates based on camera device parameters;

[0125] The calculation module 704 is used to determine the motion state of the traffic participant according to the BEV visual coordinates.

[0126] Wherein, the detection module 702 includes:

[0127] A detection submodule, used to identify and detect traffic participants in the video image using a target detection algorithm, and obtain identification ID information of the traffic participants, at least one 3D bounding box, and pixel coordinates of the center of the chassis corresponding to each of the 3D bounding boxes;

[0128] A filtering submodule, used to filter the redundant 3D bounding boxes of the detected traffic participants by using a non-maximum suppression (NMS) algorithm, to obtain a target 3D bounding box corresponding to the traffic participant;

[0129] A tracking submodule, used to continuously track the detected traffic participant using a target tracking algorithm, and obtain the target 3D bounding box of the traffic participant in different frame images of the video image and the pixel coordinates of the corresponding chassis center;

[0130] The first information includes: the ID information of the traffic participant, the target 3D bounding box and the pixel coordinates of the center of the chassis.

[0131] Wherein, the conversion module 703 includes:

[0132] A first conversion submodule, used for converting the pixel coordinates of the center of the chassis into coordinates in a camera device coordinate system according to the internal parameters and distortion parameters in the camera device parameters;

[0133] A second conversion submodule, for converting the coordinates in the camera coordinate system into coordinates in the Northeastern ENU coordinate system based on the external parameters in the camera parameters;

[0134] The first determination submodule is used to determine the BEV visual coordinates projected from the coordinates in the ENU coordinate system to the BEV visual angle.

[0135] Wherein, the device further comprises:

[0136] An acquisition submodule, used for acquiring a first image;

[0137] The calibration submodule is used to calibrate the parameters of the camera device using the first image to obtain the camera device parameters, where the camera device parameters include internal parameters, distortion parameters and external parameters.

[0138] Wherein, the calibration submodule includes:

[0139] A first acquisition unit, configured to acquire pixel coordinates of N feature points and world coordinates corresponding to each of the pixel coordinates in the first image; N is a positive integer;

[0140] A conversion unit, used for converting the world coordinates into ENU local coordinates;

[0141] The second acquisition unit is used to input the pixel coordinates and the ENU local coordinates corresponding to the pixel coordinates into a camera calibration algorithm to obtain the camera parameters output by the camera calibration algorithm.

[0142] Wherein, the determining module 704 includes:

[0143] A first calculation submodule, used for calculating the speed vector of the traffic participant according to the BEV visual coordinates corresponding to two adjacent frames of images;

[0144] A second calculation submodule, used for calculating the acceleration vector of the traffic participant according to the calculated two adjacent velocity vectors of the traffic participant;

[0145] A filtering submodule, used for filtering the velocity vector and the acceleration vector;

[0146] The second determination submodule is used to determine the motion state of the traffic participant according to the filtered velocity vector and the acceleration vector.

[0147] It should be noted here that the above-mentioned motion state detection device provided in the embodiment of the present application can implement all the method steps implemented in the above-mentioned motion state detection method embodiment applied to the first device, and can achieve the same technical effect. The parts and beneficial effects of this embodiment that are the same as the method embodiment will not be described in detail here.

[0148] An embodiment of the present application also provides a motion state detection device, including a transceiver 810, a processor 800, a memory 820, and a program stored on the memory 820 and executable on the processor 800; wherein, when the processor 800 executes the program, the motion state detection method applied to the first device as described above is implemented.

[0149] The transceiver 810 is used to receive and send data under the control of the processor 800.

[0150] Among them, Figure 8In the embodiment of the present invention, the bus architecture may include any number of interconnected buses and bridges, specifically linking various circuits of one or more processors represented by processor 800 and memory represented by memory 820. The bus architecture may also link various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are not further described herein. The bus interface provides an interface. The transceiver 810 may be a plurality of components, namely, a transmitter and a receiver, providing a unit for communicating with various other devices over a transmission medium.

[0151] The processor 800 is responsible for managing the bus architecture and general processing, and the memory 820 can store data used by the processor 800 when performing operations.

[0152] It should be noted here that the above-mentioned motion state detection device provided in the embodiment of the present application can implement all the method steps implemented in the above-mentioned motion state detection method embodiment applied to the first device, and can achieve the same technical effect. The parts and beneficial effects of this embodiment that are the same as the method embodiment will not be repeated here.

[0153] The embodiment of the present application also provides a readable storage medium, on which a program is stored, and when the program is executed by the processor, each process of the above-mentioned motion state detection method embodiment applied to the first device is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. Among them, the readable storage medium, such as a read-only memory (Read-Only Memory, referred to as ROM), a random access memory (Random Access Memory, referred to as RAM), a disk or an optical disk, etc.

[0154] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course, by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, disk, CD), and includes a number of instructions for executing the methods described in each embodiment of the present application.

[0155] Therefore, an embodiment of the present application also provides a computer program product, including computer instructions, which, when executed by a processor, implement the steps in the motion state detection method applied to the first device as described above, and can achieve the same technical effect. To avoid repetition, they are not repeated here.

[0156] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0157] The above is a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles described in the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A motion state detection method, characterized in that: Applied to a first device, the first device is deployed on a roadside, and the method includes: Acquire video images captured by a camera device; Performing target detection on the video image to obtain first information of a traffic participant contained in the video image, wherein the first information includes pixel coordinates of a chassis center of the traffic participant; Based on the camera device parameters, the pixel coordinates are converted into bird's-eye view (BEV) visual coordinates; The motion state of the traffic participant is determined based on the BEV visual coordinates.

2. The method according to claim 1, characterized in that: Performing target detection on the video image to obtain first information of traffic participants contained in the video image includes: Using a target detection algorithm to identify and detect traffic participants in the video image, obtain identification ID information of the traffic participant, at least one 3D bounding box, and pixel coordinates of the center of the chassis corresponding to each of the 3D bounding boxes; Using the non-maximum suppression (NMS) algorithm, redundant 3D bounding boxes of the detected traffic participants are filtered to obtain a target 3D bounding box corresponding to the traffic participant; Continuously tracking the detected traffic participant using a target tracking algorithm to obtain the target 3D bounding box of the traffic participant in different frame images of the video image and the pixel coordinates of the corresponding chassis center; The first information includes: the ID information of the traffic participant, the target 3D bounding box and the pixel coordinates of the center of the chassis.

3. The method according to claim 1, characterized in that Based on the camera device parameters, the pixel coordinates are converted into bird's-eye view BEV visual coordinates, including: According to the intrinsic parameters and distortion parameters in the camera equipment parameters, the pixel coordinates of the center of the chassis are converted into coordinates in the camera equipment coordinate system; Based on the external parameters in the camera device parameters, the coordinates in the camera device coordinate system are converted into coordinates in the Northeastern ENU coordinate system; Determine the BEV visual coordinates projected from the coordinates in the ENU coordinate system to the BEV visual angle.

4. The method according to claim 1 or 3, characterized in that: The method further comprises: acquiring a first image; The first image is used to calibrate parameters of the camera device to obtain parameters of the camera device, where the parameters of the camera device include intrinsic parameters, distortion parameters, and extrinsic parameters.

5. The method according to claim 4, characterized in that Calibrating the camera device using the first image to obtain parameters of the camera device includes: In the first image, obtain pixel coordinates of N feature points and world coordinates corresponding to each of the pixel coordinates; N is a positive integer; Convert the world coordinates to ENU local coordinates; The pixel coordinates and the ENU local coordinates corresponding to the pixel coordinates are input into a camera calibration algorithm to obtain the camera parameters output by the camera calibration algorithm.

6. The method according to claim 1, characterized in that Determining the motion state of the traffic participant according to the BEV visual coordinates includes: Calculating a speed vector of the traffic participant according to the BEV visual coordinates corresponding to two adjacent frames of images; Calculating the acceleration vector of the traffic participant according to the calculated two adjacent velocity vectors of the traffic participant; filtering the velocity vector and the acceleration vector; The motion state of the traffic participant is determined according to the filtered velocity vector and the acceleration vector.

7. A motion state detection device, characterized in that: Applied to a first device, the first device is deployed on a road side, and the apparatus includes: An acquisition module, for acquiring video images captured by a camera device; A detection module, configured to perform target detection on the video image to obtain first information of a traffic participant contained in the video image, wherein the first information includes pixel coordinates of a chassis center of the traffic participant; A conversion module, configured to convert the pixel coordinates into a bird's-eye view (BEV) visual coordinates based on camera device parameters; A calculation module is used to determine the motion state of the traffic participant according to the BEV visual coordinates.

8. A motion state detection device, comprising a transceiver, a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the motion state detection method according to any one of claims 1 to 6 is implemented.

9. A readable storage medium having a program or instruction stored thereon, characterized in that: When the program or instruction is executed by a processor, the motion state detection method according to any one of claims 1 to 6 is implemented.

10. A computer program product, characterized in that The method comprises computer instructions, which, when executed by a processor, implement the motion state detection method according to any one of claims 1 to 6.

Images