Vehicle longitudinal and lateral speed prediction method and system based on radar and vision integrated machine and readable storage medium

Abstract

This application relates to the field of data processing technology, and in particular to a method, system, and readable storage medium for predicting the lateral and longitudinal speeds of vehicles based on a radar-view integrated camera. The method utilizes radar-view integrated cameras installed upstream and downstream of a road segment to collect vehicle driving data within the detection range, forming single-source bidirectional fragmented data, and treating the area between adjacent detectors as a blind zone. A state-space model is used to deeply mine the multidimensional spatiotemporal correlation features implicit in the multidimensional data. The input multidimensional spatiotemporal correlation features are discretized, and then a selective mechanism and time-varying module are used to learn the key information of the bidirectional data and extract local and global temporal features, finally outputting the lateral and longitudinal speed results of vehicles within the blind zone. The aim is to solve the problem of how to predict the lateral and longitudinal speeds of vehicles within the detection blind zone between radar-view integrated cameras.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a method, system, and readable storage medium for predicting the lateral and longitudinal speeds of vehicles based on a radar-visual integrated machine. Background Technology

[0002] With the advancement of intelligent transformation of highways and the construction of digital twins, the ways of obtaining vehicle driving status and road traffic information on the road are becoming more and more diversified. Vehicle full sample lateral and longitudinal speed data have emerged, which refers to data covering the lateral and longitudinal speeds of vehicles at every moment during the driving process. It can completely record the macro and micro motion characteristics of vehicles in continuous spatiotemporal dimensions and is the basic data source for analyzing complex multi-lane traffic systems.

[0003] Currently, the Radar-Video Integration Device (RDD), a new generation of equipment used for fixed-point traffic data collection on highways, can continuously collect multi-dimensional trajectory parameters such as vehicle ID, timestamp, longitudinal and lateral position (lane information), and speed within its monitoring range. However, the application of full-sample vehicle speed data in complex scenarios such as highways currently faces certain technical limitations. The effective monitoring range of the RDD is generally around 400 meters, but due to practical constraints such as equipment cost, installation conditions, and privacy protection, its actual coverage is low. Furthermore, the distance between adjacent RDDs often exceeds 500 meters. This low coverage characteristic results in significant detection blind spots on highways, meaning that the RDD can only acquire fragmented vehicle speed data, which is significantly different from full-sample vehicle speed data.

[0004] In view of this, this application proposes a method for predicting the lateral and longitudinal speeds of vehicles based on a radar-visual integrated camera, aiming to achieve high-precision prediction of the lateral and longitudinal speeds of vehicles within the blind zone based on data collected by a low-coverage radar-visual integrated camera. Summary of the Invention

[0005] The main purpose of this application is to provide a method for predicting the lateral and longitudinal speeds of vehicles based on data from a radar-visual integrated machine, aiming to solve the problem of how to predict the lateral and longitudinal speeds of vehicles within the detection blind zone of the radar-visual integrated machine.

[0006] To achieve the above objectives, this application provides a method for predicting the lateral and longitudinal speeds of vehicles based on data from a radar-visual integrated machine. The method includes the following steps:

[0007] Acquire vehicle driving data of target vehicles collected by adjacent upstream / downstream integrated radar-visual cameras on expressways, as well as the detection blind spots between the upstream / downstream integrated radar-visual cameras;

[0008] Temporal modeling is performed on the measured lateral and longitudinal velocities of the vehicles in the vehicle driving data to extract multidimensional spatiotemporal correlation features. These multidimensional spatiotemporal correlation features include vehicle ID, timestamp, lateral and longitudinal positions, and velocities of the vehicles in the data collected by single-source, bidirectional, and fragmented detectors.

[0009] The multidimensional spatiotemporal correlation features are input into a preset model to obtain the vehicle lateral and longitudinal speed prediction results of the detection blind zone output by the preset model. The preset model discretizes the multidimensional spatiotemporal correlation features and extracts local and global temporal features through a selective mechanism and a time-varying module.

[0010] Optionally, the step of performing time-series modeling on the measured lateral and longitudinal velocities of the vehicles in the vehicle driving data, and extracting multi-dimensional spatiotemporal correlation features such as vehicle ID, timestamp, lateral and longitudinal position, and velocity from the data collected by single-source, bidirectional, and fragmented detectors includes:

[0011] Discover the hidden state of the previous time step t-1 Multidimensional spatiotemporal characteristics of vehicle lateral and longitudinal speeds Output the current state Then through and Mapping to output response :

[0012]

[0013]

[0014] in, , These represent multidimensional time series data, respectively, as input and output. Represents the current state of the transportation system, and indicates the relationship with... The result of combined action; matrix Used to adjust the dynamic state changes of the observation equation; and These are the input projection matrix and the output projection matrix, respectively; Let R be the residual, and R be a real number.

[0015] Optionally, the preset model, when discretizing the multidimensional spatiotemporal correlation features, includes the following steps:

[0016] Matrix A and matrix B are transformed into discrete parameters using the zero-order preservation technique. and The sequence value is maintained each time a discrete signal is received until the next new discrete signal is received:

[0017]

[0018]

[0019] in, and These are the methods used to preserve the matrix using zero-order preservation techniques. and The parameters after discretization, Δ t A is the increment of the state matrix, Δ t B represents the increment of the input matrix, and I represents the identity matrix.

[0020] Optionally, when the preset model uses a selective mechanism and a time-varying module to extract local and global temporal features from the discretized multidimensional spatiotemporal correlation features, it includes the following steps:

[0021] Introducing step size As a learnable parameter, according to The magnitude of the input is used to sample the multidimensional spatiotemporal correlation features after discretization, resulting in a discretized input velocity sequence;

[0022] Incorporate a selectivity mechanism to handle discrete parameters and The parameters become input-dependent, allowing them to be dynamically adjusted based on the input speed data during deep learning. The selection mechanism is implemented using a set of input-sensitive gating vectors, represented as:

[0023]

[0024] Among them, u t The multidimensional input features (horizontal and vertical velocities) at time t. For the Sigmoid function, W g and b g Here, d represents the learnable parameter, and d represents the state dimension. The selective mechanism maps instantaneous inputs to gated parameters g. t The system matrix is ​​selected time-by-time.

[0025] Optionally, the selective mechanism and time-varying module are represented as follows:

[0026]

[0027]

[0028] in, express l Hidden states within a time frame; and They represent lThe input and output lateral and longitudinal velocity sequences within a time interval. and With Change with change.

[0029] Optionally, the expressions for the measured lateral velocity and the measured longitudinal velocity of the target vehicle are:

[0030]

[0031]

[0032] In the formula, and These are the measured lateral speed and the measured longitudinal speed of the vehicle, respectively. It is time, measured in frames; and These represent the horizontal and vertical distances, respectively; 'a' is the conversion factor between frames and seconds.

[0033] In addition, to achieve the above objectives, this application also provides a computer system, the computer system comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, it implements the steps of the vehicle lateral and longitudinal speed prediction method based on radar-visual integrated machine data as described in any of the preceding claims.

[0034] In addition, to achieve the above objectives, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the vehicle lateral and longitudinal speed prediction method based on radar-visual integrated machine data as described in any of the preceding claims.

[0035] This application has at least the following beneficial effects:

[0036] (1) Analyze the characteristics of the data collected by the upstream and downstream integrated radar and vision machines, take into account the time sequence of data collected by the single source detector at different installation locations, and unify the ID of the same vehicle when it is collected by the upstream and downstream integrated radar and vision machines to obtain the known information of the vehicle at the upstream and downstream locations, thus expanding the universality.

[0037] (2) By deeply mining the spatiotemporal correlation characteristics of the data collected by the upstream and downstream integrated radar and vision systems, a "forward-backward" bidirectional learning prediction model is proposed to construct a bidirectional collaborative perception mechanism. This application not only achieves high-resolution prediction of the lateral and longitudinal speeds of vehicles within the blind zone, but also effectively solves the problem of result instability in single-path prediction through bidirectional information complementarity.

[0038] (3) Using bidirectional fragmented vehicle driving data collected by the radar-visual integrated machine, the known data is learned by the advanced deep learning framework - Mamba model. The long-range spatiotemporal dependence between data is effectively captured by state space modeling. Then, the lateral and longitudinal speeds of all vehicles within the blind zone are predicted, providing a general and practical deep learning prediction method for the perception of vehicle motion state in the blind zone of the detector in complex highway environments.

[0039] (4) The radar-visual integrated machine can report detection data in real time. This application can be used in both online and offline environments, thereby expanding its application in traffic state estimation. Attached Figure Description

[0040] Figure 1 This is a flowchart illustrating the steps of the first embodiment of this application.

[0041] Figure 2 This application describes the acquisition of known data, determination of blind zone range, and reconstruction of target map in the detection environment of the integrated radar-visual machine involved in the embodiments of this application;

[0042] Figure 3 This is a framework diagram of the Mamba model used in this application to predict the lateral and longitudinal velocities of a vehicle.

[0043] Figure 4 This application provides a real-world road environment and schematic diagram related to the embodiments of this application.

[0044] Figure 5 This is a partial prediction diagram of the lateral and longitudinal speeds of vehicles in a real road environment involved in the embodiments of this application.

[0045] Figure 6 This is a contour map comparing the actual and predicted longitudinal speeds of all vehicles within the detection blind zone area involved in the embodiments of this application;

[0046] Figure 7 This is a flowchart illustrating an embodiment of the computer system involved in this application.

[0047] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0048] To better understand the above technical solutions, exemplary embodiments of this disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of this disclosure to those skilled in the art.

[0049] First Embodiment

[0050] Reference Figure 1 This embodiment provides a method for predicting the lateral and longitudinal speeds of vehicles based on data from a radar-visual integrated machine. The method includes the following steps:

[0051] Step S10: Obtain vehicle driving data of target vehicles collected by adjacent upstream / downstream integrated radar-visual cameras on the expressway, as well as the detection blind spots between the upstream / downstream integrated radar-visual cameras;

[0052] In this embodiment, multi-dimensional dynamic data of vehicles within the detection range are collected by radar-visual integrated machines installed upstream and downstream of the road section, including ID, time, location and speed data, forming single-source bidirectional fragmented data, and the area between adjacent detectors is regarded as the blind zone.

[0053] Specifically, since the time format of the detection data from adjacent integrated radar-viewing machines is not uniform, it is necessary to unify the collected data according to the time order. The time unit of the collected data from the integrated radar-viewing machines is set to a frame, and the position unit is set to feet, as shown in equations (1) and (2):

[0054] (1)

[0055] (2)

[0056] in, It is time, measured in frames. and These are the horizontal and vertical distances, respectively, in feet; and These are the horizontal and vertical speeds, respectively, in feet per second; 'a' is the conversion factor between frames and seconds.

[0057] In this embodiment, a is 0.1.

[0058] In the process of predicting vehicle lateral and longitudinal speeds, blind zone ranges are determined using integrated radar-visual cameras installed at upstream and downstream locations. The time range of the blind zone is determined by the time each vehicle leaves the upstream radar-visual camera's acquisition range and enters the downstream radar-visual camera's acquisition range, while the spatial range of the blind zone is determined by the interval between the upstream and downstream radar-visual cameras. The vehicle data collected by the integrated radar-visual cameras includes vehicle ID, timestamp, lateral and longitudinal speeds, and position, i.e. ( , , , , , ), of which, the raw data when the vehicle passes the upstream radar-visual integrated machine, i.e. ( , , , , , (), , , , , , ), ..., ( , , , , , ), , , , , and These are, respectively, the ID, timestamp, lateral speed, longitudinal speed, lateral position, and longitudinal position of the i-th vehicle within the range collected by the upstream radar-view integrated machine; and the raw data of the vehicle when it passes the downstream radar-view integrated machine, i.e. ( , , , , , (), , , , , , ), ..., ( , , , , , ), , , , , and These are the ID, timestamp, lateral velocity, longitudinal velocity, lateral position, and longitudinal position of the i-th vehicle within the downstream radar-vision integrated camera's acquisition range. The longitudinal velocities of all vehicles are plotted on the same spatiotemporal map, as shown below. Figure 2 As shown.

[0059] Step S20: Perform time-series modeling on the measured lateral and longitudinal speeds of the vehicles in the vehicle driving data, and extract multi-dimensional spatiotemporal correlation features. The multi-dimensional spatiotemporal correlation features include vehicle ID, timestamp, lateral and longitudinal position and speed of the vehicle in the data collected by single-source, bidirectional and fragmented detectors.

[0060] Temporal modeling of the vehicle's lateral and longitudinal velocities is performed, and the hidden state of the previous time step t-1 is mined. Vehicle lateral and longitudinal velocity input features Output the current state Then through and Mapping to output response As shown in equations (3) and (4):

[0061] (3)

[0062] (4)

[0063] in, , These represent the input and predicted output time series data, respectively. This indicates the current state of the transportation system. and The result of combined action; matrix Used to adjust the dynamic state changes of the observation equation; and These are the projection matrices for the input and output, respectively; The value is the residual, which is generally taken as 0; R represents a real number.

[0064] Step S30: Input the multidimensional spatiotemporal correlation features into a preset model and obtain the vehicle lateral and longitudinal speed prediction results of the detection blind zone output by the preset model. The preset model discretizes the multidimensional spatiotemporal correlation features and extracts local and global temporal features through a selective mechanism and a time-varying module.

[0065] In this embodiment, the preset model can be the Mamba model. The input data is discretized using the zero-order hold technique in the Mamba model, and then the key information of the bidirectional data is learned by using a selective mechanism and a time-varying module to extract local and global temporal features. Finally, the high spatiotemporal resolution lateral and longitudinal velocity results of the vehicle in the blind spot are output.

[0066] In this embodiment, the zero-order hold technique in the Mamba model is used to discretize the input data. Then, a selective mechanism and a time-varying module are used to learn the key information of the bidirectional data and extract the local and global temporal features. Finally, the high spatiotemporal resolution lateral and longitudinal velocity results of the vehicle in the blind spot are output.

[0067] In addition, selective mechanisms and time-varying modules are used to learn bidirectional fragmented data to predict vehicle lateral and longitudinal velocities with high spatiotemporal resolution within blind spots, specifically including:

[0068] By introducing a new learnable parameter—step size To determine the time to save this value, the input continuous velocity timing data is based on... The magnitude of the input velocity is sampled to obtain a discretized input velocity sequence;

[0069] It should be noted that the first use of the zero-order hold technique for discretization transforms the continuous-time problem into a discrete-time problem, which applies to matrices A and B; while the second use of Δt discretization discretizes the input velocity sequence data by sampling at Δt intervals.

[0070] Local features refer to the changes in speed and instantaneous interactions within a short time window surrounding the currently sampled data. These features are used to characterize "imminent" action patterns, with a short time scale and rapid response. Global features refer to slow variables and trend information that gradually accumulate over a longer time range, manifested as the overall speed baseline of the road segment, congestion mitigation / spreading trends, the inertia and memory effects of stable cruising / queue following, and the consistency of multiple sensors on the time axis. These features are used to characterize "slow evolution," with a long time scale and gradual changes.

[0071] On the one hand, the selective mechanism generates a gating vector at each time step based on the current input features and internal state, and performs dimension-wise weighting on each channel of the discrete system, thereby enabling the model to generate higher gains for local disturbances. On the other hand, the time-varying module dynamically changes the effective time span of state updates by inputting the relevant sampling step size and the corresponding ZOH discretization: when the traffic state is stable, the step size is appropriately increased, allowing the state to propagate and accumulate over a longer time scale, strengthening the integration of global trends and slow variables; when the state changes abruptly, the step size automatically shrinks, improving the time resolution, making it easier for the selective mechanism to capture and process high-frequency changes in the short to medium term.

[0072] Therefore, a selection mechanism is added to the discrete parameters. and The parameters become input-dependent, allowing them to be dynamically adjusted based on the input speed data during deep learning. This enables them to perform better on long sequence modeling tasks, thereby predicting high-resolution vehicle lateral and longitudinal speed data within the blind zone.

[0073] Specifically, a selective mechanism and a time-varying module are added, as shown in equations (7) and (8):

[0074] (7)

[0075] (8)

[0076] in, This represents the hidden state at time l; and These represent the input and output lateral and longitudinal velocity sequences at time l, respectively.

[0077] In this embodiment, =10.

[0078] It should be noted that the selective mechanism, centered on a gating vector, is used to adaptively adjust the "opening degree" of the dynamic channel at each sampling time according to the current scene. In some optional implementations, the system first embeds the multidimensional spatiotemporal features and inputs the hidden state from the previous time step into a nonlinear mapping to generate a gating vector g with values ​​in the interval (0,1). t After the gating vector is generated, the selective mechanism does not simply replace the system parameters, but performs a dimension-wise weighted fusion of the time-varying discrete system matrix: on the one hand, it uses the main channel matrix obtained by zero-order preservation and learnable step size, and on the other hand, it retains the residual matrix with robust priors.

[0079] In the technical solution provided in this embodiment, vehicle driving data within the detection range is collected by radar-visual integrated machines installed upstream and downstream of the road section, forming single-source bidirectional fragmented data, and the area between adjacent detectors is regarded as the blind zone. The state space model is used to deeply mine the multidimensional spatiotemporal correlation features hidden in the multidimensional data. The input multidimensional spatiotemporal correlation features are discretized, and then the key information of the bidirectional data is learned by using a selective mechanism and a time-varying module to extract the local and global temporal features, and finally output the lateral and longitudinal speed results of vehicles in the blind zone.

[0080] Verification Implementation Examples

[0081] Based on the vehicle lateral and longitudinal velocity prediction method using radar-visual integrated machine data involved in the first embodiment, this embodiment verifies the prediction effect of the method as follows:

[0082] Using highways from the NGSIM dataset as the experimental subject, one radar-visual integrated machine was set up upstream and one downstream. The detection range of the radar-visual integrated machine was 400 feet, and the spatial interval of the blind zone range between adjacent radar-visual integrated machines was 1000 feet. Using bidirectional fragmented vehicle data collected by the radar-visual integrated machines as input, the high-resolution lateral and longitudinal velocities of vehicles within the blind zone range were predicted.

[0083] The actual environment and schematic diagram of the highway section in the embodiment are as follows: Figure 4 As shown.

[0084] First, some of the original data required for this application in the NGSIM dataset are shown in Table 1:

[0085] Table 1. Partial raw trajectory data from the NGSIM dataset

[0086]

[0087] The formulas for calculating the lateral and longitudinal velocities of vehicles collected by the radar-visual integrated machine are as follows:

[0088]

[0089]

[0090] Secondly, the upstream radar-visual integrated camera was selected and deployed at 200 feet, while the downstream radar-visual integrated camera was deployed at 1600 feet, both with a detection range of 400 feet. After standardizing the data format and determining the time sequence of the collected data, the time range was determined by the time the vehicle left the upstream detection range and entered the downstream detection range, which, together with the spatial range of the radar-visual integrated camera, formed the blind zone range. As shown in Table 1, the spatial range of the blind zone in this example is between 400 feet and 1400 feet, and the time range of the vehicle with ID=2 within the blind zone is from 102 frames to 322 frames.

[0091] Next, a state-space model is used to capture the long-range temporal correlation between bidirectional known fragmented data. Then, the selection mechanism and time-varying module in the Mamba model are applied to learn the key information of the bidirectional known fragmented data and extract local and global temporal features, thereby predicting the high-precision lateral and longitudinal velocities of vehicles within the blind zone. The lateral and longitudinal velocity prediction results for some vehicles are as follows: Figure 5 As shown in the figure. The results demonstrate that this application can fully utilize bidirectional fragmented detector data and use the Mamba model to predict high-resolution lateral and longitudinal vehicle velocity data within the detection blind zone.

[0092] Finally, based on the steps of this application, the high-resolution longitudinal velocity prediction results for all vehicles within the blind spot area are obtained as follows: Figure 6 As shown. Since the predicted lateral and longitudinal velocity data are microscopic data, the mean absolute error (MAE) and root mean square error (RMSE) are used to quantify the accuracy of the lateral and longitudinal velocity prediction of vehicles within the blind zone in this application. Within this blind zone, the spatial range of the blind zone is 1000 feet, and the detection range of the radar-visual integrated machine is 400 feet. The quantification indicators MAE and RMSE are 2.03 feet / s and 4.15 feet / s, respectively.

[0093] As one implementation scheme, Figure 7 This is a schematic diagram of the hardware operating environment of the computer system involved in the embodiments of this application.

[0094] like Figure 7As shown, the computer system may include: a processor 1001, such as a CPU; a memory 1005; a user interface 1003; a network interface 1004; and a communication bus 1002. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or non-volatile memory, such as a disk drive. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0095] Those skilled in the art will understand that Figure 7 The computer system architecture shown does not constitute a limitation on the computer system and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0096] like Figure 7 As shown, the memory 1005, as a storage medium, may include an operating system, a network communication module, a user interface module, and computer programs. The operating system is a program that manages and controls the hardware and software resources of the computer system, as well as the operation of the computer programs and other software or programs.

[0097] exist Figure 7 In the computer system shown, the user interface 1003 is mainly used to connect to the terminal and communicate data with the terminal; the network interface 1004 is mainly used to communicate data with the backend server; and the processor 1001 can be used to call the computer program stored in the memory 1005.

[0098] In this embodiment, the computer system includes: a memory 1005, a processor 1001, and a computer program stored in the memory and executable on the processor, wherein:

[0099] When processor 1001 calls a computer program stored in memory 1005, it performs the following operations:

[0100] Acquire vehicle driving data of the target vehicle collected by the upstream / downstream integrated radar-visual equipment, as well as the detection blind spots between the upstream / downstream integrated radar-visual equipment;

[0101] Time-series modeling is performed on the measured lateral and longitudinal velocities of the vehicles in the vehicle driving data to extract multidimensional spatiotemporal correlation features;

[0102] The multidimensional spatiotemporal correlation features are input into a preset model to obtain the vehicle lateral and longitudinal speed prediction results of the detection blind zone output by the preset model. The preset model discretizes the multidimensional spatiotemporal correlation features and extracts local and global temporal features through a selective mechanism and a time-varying module.

[0103] When processor 1001 calls a computer program stored in memory 1005, it performs the following operations:

[0104] Discover the hidden state of the previous time step t-1 Multidimensional spatiotemporal characteristics of vehicle lateral and longitudinal speeds Output the current state Then through and Mapping to output response :

[0105]

[0106]

[0107] in, , These represent multidimensional time series data, respectively, as input and output. Represents the current state of the transportation system, and indicates the relationship with... The result of combined action; matrix Used to adjust the dynamic state changes of the observation equation; and These are the input projection matrix and the output projection matrix, respectively; Let R be the residual, and R be a real number.

[0108] When processor 1001 calls a computer program stored in memory 1005, it performs the following operations:

[0109] Matrix A and matrix B are transformed into discrete parameters using the zero-order preservation technique. and The sequence value is maintained each time a discrete signal is received until the next new discrete signal is received:

[0110]

[0111]

[0112] in, and These are the methods used to preserve the matrix using zero-order preservation techniques. and The parameters after discretization, Δ t A is the increment of the state matrix, Δt B represents the increment of the input matrix, and I represents the identity matrix.

[0113] When processor 1001 calls a computer program stored in memory 1005, it performs the following operations:

[0114] Introducing step size As a learnable parameter, according to The magnitude of the input is used to sample the multidimensional spatiotemporal correlation features after discretization, resulting in a discretized input velocity sequence;

[0115] Incorporate a selectivity mechanism to handle discrete parameters and The parameters become input-dependent, allowing them to be dynamically adjusted based on the input speed data during deep learning. The selection mechanism is implemented using a set of input-sensitive gating vectors, represented as:

[0116]

[0117] Among them, u t The multidimensional input features (horizontal and vertical velocities) at time t. For the Sigmoid function, W g and b g Here, d represents the learnable parameter, and d represents the state dimension. The selective mechanism maps instantaneous inputs to gated parameters g. t The system matrix is ​​selected time-by-time.

[0118] When processor 1001 calls a computer program stored in memory 1005, it performs the following operations:

[0119] The selective mechanism and time-varying module are represented as follows:

[0120]

[0121]

[0122] in, express l Hidden states within a time frame; and They represent l The input and output lateral and longitudinal velocity sequences within a time interval. and With Change with change.

[0123] When processor 1001 calls a computer program stored in memory 1005, it performs the following operations:

[0124] The expressions for the measured lateral velocity and measured longitudinal velocity of the target vehicle are as follows:

[0125]

[0126]

[0127] In the formula, and These are the measured lateral speed and the measured longitudinal speed of the vehicle, respectively. It is time, measured in frames; and These represent the horizontal and vertical distances, respectively, in feet; 'a' is the conversion factor between frames and seconds.

[0128] Furthermore, those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program includes program instructions and can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in a computer system to implement the process steps of the embodiments of the above methods.

[0129] Therefore, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the various steps of the vehicle lateral and longitudinal speed prediction method based on radar-visual integrated machine data as described in the above embodiments.

[0130] The computer-readable storage medium can be any computer-readable storage medium capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory (ROM), magnetic disk, or optical disk.

[0131] It should be noted that, since the storage medium provided in the embodiments of this application is the storage medium used to implement the methods of the embodiments of this application, those skilled in the art can understand the specific structure and variations of the storage medium based on the methods described in the embodiments of this application, and therefore will not be repeated here. All storage media used in the methods of the embodiments of this application fall within the scope of protection of this application.

[0132] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0133] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0134] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0135] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0136] Although preferred embodiments of this application have been described, those skilled in the art, once they have learned the basic inventive concept, can make other changes and modifications to these embodiments.

[0137] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of this application and its equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for predicting the lateral and longitudinal speeds of vehicles based on a radar-visual integrated machine, characterized in that, The method includes the following steps: Acquire vehicle driving data of target vehicles collected by adjacent upstream / downstream integrated radar-visual cameras on expressways, as well as the detection blind spots between the upstream / downstream integrated radar-visual cameras; Temporal modeling is performed on the measured lateral and longitudinal velocities of the vehicles in the vehicle driving data to extract multidimensional spatiotemporal correlation features. These multidimensional spatiotemporal correlation features include vehicle ID, timestamp, lateral and longitudinal positions, and velocities of the vehicles in the data collected by single-source, bidirectional, and fragmented detectors. The multidimensional spatiotemporal correlation features are input into a preset model to obtain the vehicle lateral and longitudinal speed prediction results of the detection blind zone output by the preset model. The preset model discretizes the multidimensional spatiotemporal correlation features and extracts local and global temporal features through a selective mechanism and a time-varying module. The steps of performing time-series modeling on the measured lateral and longitudinal velocities of the vehicles in the vehicle driving data, and extracting multi-dimensional spatiotemporal correlation features of vehicle ID, timestamp, lateral and longitudinal position, and velocity from the data collected by single-source, bidirectional, and fragmented detectors include: Discover the hidden state of the previous time step t-1 Multidimensional spatiotemporal characteristics of vehicle lateral and longitudinal speeds Output the hidden state at the current time step. Then through and Mapping to output response : ； ； in, , These represent multidimensional time series data, respectively, as input and output. Represents the hidden state of the transportation system at the current moment, and represents the relationship with... The result of combined action; matrix Used to adjust the dynamic state changes of the observation equation; and These are the input projection matrix and the output projection matrix, respectively; R represents the residual; When the preset model performs discretization on the multidimensional spatiotemporal correlation features, it includes the following steps: Matrix A and matrix B are transformed into discrete parameters using the zero-order preservation technique. and The sequence value is maintained each time a discrete signal is received until the next new discrete signal is received: ； ； in, and These are the methods used to preserve the matrix using zero-order preservation techniques. and The parameters after discretization, Δ t A is the increment of the state matrix, Δ t B is the increment of the input matrix, and I represents the identity matrix; When the preset model uses a selective mechanism and a time-varying module to extract local and global temporal features from the discretized multidimensional spatiotemporal correlation features, it includes the following steps: Introducing step size As a learnable parameter, according to The magnitude of the input is used to sample the multidimensional spatiotemporal correlation features after discretization, resulting in a discretized input velocity sequence; Incorporate a selectivity mechanism to handle discrete parameters and The parameters become input-dependent, allowing them to be dynamically adjusted based on the input speed data during deep learning. The selection mechanism is implemented using a set of input-sensitive gating vectors, represented as: ； Among them, u t For the multidimensional input features at time t, For the Sigmoid function, W g and b g Here, d represents the learnable parameter, and d represents the state dimension. The selective mechanism maps instantaneous inputs to gated parameters g. t The system matrix is ​​selected time-by-time; The selective mechanism and time-varying module are represented as follows: ； ； in, express l Hidden states within a time frame; and They represent l The input and output lateral and longitudinal velocity sequences within a time interval. and With Change with change.

2. The method as described in claim 1, characterized in that, The expressions for the measured lateral velocity and measured longitudinal velocity of the target vehicle are as follows: ； ； In the formula, and These are the measured lateral speed and the measured longitudinal speed of the vehicle, respectively. It is time, measured in frames; and These represent the horizontal and vertical distances, respectively; 'a' is the conversion factor between frames and seconds.

3. A computer system, characterized in that, The computer system includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the steps of the vehicle lateral and longitudinal speed prediction method based on the radar-visual integrated machine as described in any one of claims 1 to 2.

4. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the vehicle lateral and longitudinal speed prediction method based on the radar-visual integrated machine as described in any one of claims 1 to 2.

Images