Vehicle status determination method, device, electronic equipment and readable storage medium

Abstract

This application provides a vehicle state determination method, apparatus, electronic device, and readable storage medium. The method includes: acquiring positioning points in a target region of a target vehicle image; calculating the offset distance value of the positioning points in the historical preset frames using position data of the positioning points in the target vehicle image of historical preset frames; and determining the vehicle state corresponding to the shaking of the target vehicle based on the relationship between the offset distance value and a distance threshold. This application's embodiment determines the offset distance value of the positioning points in historical frames based on the position data of the positioning points in the vehicle image, and then determines the shaking intensity of the target vehicle through the offset distance value of the positioning points. This achieves vehicle state determination during shaking, reduces misjudgments during vehicle shaking, and improves the accuracy of vehicle state determination during shaking.

Description

Technical Field

[0001] This application relates to the field of autonomous driving, and more specifically, to a method, apparatus, electronic device, and readable storage medium for determining the state of a vehicle. Background Technology

[0002] Vehicle status assessment is a prerequisite for autonomous driving systems, providing a valid basis for path planning and vehicle control. Accurate and reliable vehicle status detection technology plays a crucial role in controlling autonomous vehicles to perform actions such as parking and driving. The accuracy of vehicle status directly affects the safety, operational efficiency, and driving experience of the autonomous driving system. Current technologies for vehicle status detection mostly rely on tracking two-dimensional bounding boxes. However, in practical applications, when the vehicle vibrates, the detection of these bounding boxes often deviates, leading to misjudgments of the vehicle status. Summary of the Invention

[0003] In view of this, the purpose of the embodiments of this application is to provide a vehicle state determination method, device, electronic device and readable storage medium, which can reduce misjudgment of the vehicle when it is shaking and improve the accuracy of vehicle state determination when it is shaking.

[0004] In a first aspect, embodiments of this application provide a vehicle state determination method, comprising: acquiring a positioning point in a target region of a target vehicle in a target vehicle image; calculating an offset distance value of the positioning point in a historical preset frame using position data of the positioning point in a target vehicle image of a historical preset frame; and determining the vehicle state corresponding to the shaking of the target vehicle based on the relationship between the offset distance value and a distance threshold.

[0005] In the above implementation process, the offset distance value of the positioning point in the historical frame is determined based on the position data of the positioning point in the vehicle image. Then, the shaking intensity of the target vehicle can be determined by the offset distance value of the positioning point. Since the shaking intensity of the vehicle is different in different states, the vehicle state of the target vehicle can be determined by the relationship between the offset distance value and the distance threshold. This realizes the determination of the vehicle state when shaking, reduces the misjudgment of the vehicle when shaking, and improves the accuracy of the determination of the vehicle state when shaking.

[0006] In one embodiment, the offset distance value includes an average offset distance value and an Euclidean offset distance value. Calculating the offset distance value of the target position in the historical preset frame using the position data of the positioning points in the target vehicle image of the historical preset frame includes: performing a straight line fit on the position data using the least squares method to obtain a target straight line; determining the average offset distance value of all positioning points to the target straight line; and calculating the Euclidean offset distance value of the positioning point using the position data.

[0007] In the above implementation process, by calculating the average offset distance between the positioning point and the target line, the jitter intensity of the target vehicle along its left and right sides during vibration can be determined. Furthermore, by calculating the distance between positioning points in consecutive frames, the jitter intensity of the target vehicle along its longitudinal direction during vibration can be determined. In other words, the calculation of the target vehicle's offset distance considers the jitter intensity of the positioning point in multiple dimensions, enabling a more accurate determination of the positioning point's offset distance value and improving its accuracy.

[0008] In one embodiment, calculating the Euclidean offset distance value of the positioning point using the location data includes: dividing the location data of the positioning point in the target vehicle image of a historical preset frame into a first part of historical preset frame location data and a second part of historical preset frame location data; determining the first part of the historical preset frame average location data of the first part of the historical preset frame location data, and the second part of the historical preset frame average location data of the second part of the historical preset frame location data; calculating the Euclidean offset distance value between the first part of the historical preset frame average location data and the second part of the historical preset frame average location data; wherein, the formula for calculating the Euclidean offset distance value between the first part of the historical preset frame average location data and the second part of the historical preset frame average location data is: Where, d c c is the Euclidean offset distance value. 1x c represents the value of the average position data of the first set of historical preset frames on the first coordinate axis. 2x The value of the average position data of the second part of the historical preset frames on the first coordinate axis, c 1y c represents the value of the average position data of the second set of historical preset frames on the second coordinate axis. 2y The value of the average position data of the second set of historical preset frames on the second coordinate axis.

[0009] In the above implementation process, when dividing the position data of historical preset frames, the previous and subsequent preset frames are divided equally, so that the weights of the first and second parts of the historical preset frame position data are the same, thereby improving the accuracy of the average position data of the first and second parts of the historical preset frames. Furthermore, the Euclidean offset distance is calculated using the average position data of the first and second parts of the historical preset frames, improving the accuracy of the Euclidean offset distance.

[0010] In one embodiment, the formula for determining the average offset distance value of all positioning points from the target line is: Where A, B, and C are the parameters of the target line, x iLet y be the value of the i-th positioning point on the first coordinate axis. i Let d be the value of the i-th positioning point on the second coordinate axis. i Let d be the distance between the i-th positioning point and the target line, N be the number of positioning points in the target vehicle image of the historical preset frame, and d be the distance between the i-th positioning point and the target line. avg This represents the average offset distance from the positioning point to the target line.

[0011] In the above implementation process, by calculating the average offset distance from all positioning points to the target line, the average offset distance of multiple positioning points along the left and right sides of the target vehicle when the target vehicle shakes can be determined. This method of determining the shaking intensity of the target vehicle along the left and right sides of the vehicle by using the average offset distance of all positioning points in historical preset frames improves the accuracy of the average offset distance value because it integrates the offset distances of all positioning points in historical preset frames.

[0012] In one embodiment, the distance threshold includes an average distance threshold and an Euclidean distance threshold. Determining the vehicle state corresponding to the target vehicle vibration based on the relationship between the offset distance value and the distance threshold includes: if the average offset distance value is less than the average distance threshold and the Euclidean offset distance value is greater than the Euclidean distance threshold, determining that the target vehicle is in a motion vibration state.

[0013] In the above implementation process, when the average offset distance value is less than the average distance threshold, it can be determined that the target vehicle is experiencing minor vibration. Then, when the target vehicle's vibration is minor, if the Euclidean offset distance value is greater than the Euclidean distance threshold, it can be determined that the target vehicle's Euclidean offset distance value includes not only the offset caused by vibration but also the displacement caused by the target vehicle's movement. Therefore, it can be determined that the target vehicle is in a state of motion vibration. Determining the target vehicle's state using both the average offset distance value and the Euclidean offset distance value improves the accuracy of the target vehicle state assessment.

[0014] In one embodiment, obtaining the localization point in the target region of the target vehicle image includes: inputting the target vehicle image into a deep learning network, and outputting a target detection box after NMS maximum suppression; and determining the point at a preset position in the target detection box as the localization point.

[0015] In the above implementation process, the target bounding boxes in the target vehicle image are detected by using NMS maximum suppression. Since NMS maximum suppression can effectively remove the dark bounding boxes in the target vehicle image, the accuracy of target bounding box detection is improved.

[0016] In one embodiment, after acquiring the positioning point in the target region of the target vehicle image, the method further includes: projecting the positioning point onto a local coordinate system to obtain the target position corresponding to the positioning point, wherein the local coordinate system is a coordinate system established by the installation device where the image acquisition device is located; calculating the offset distance value of the positioning point in the historical preset frame using the position data of the positioning point in the target vehicle image of the historical preset frame includes: calculating the offset distance value of the target position in the historical preset frame using the position data of the target position in the target vehicle image of the historical preset frame; determining the vehicle state corresponding to the target vehicle vibration based on the relationship between the offset distance value and a distance threshold includes: determining the vehicle state corresponding to the target vehicle vibration based on the relationship between the offset distance value of the target position and a distance threshold.

[0017] In the above implementation process, the positioning point is projected onto the target coordinate system to calculate the offset distance value in the target coordinate system. Since the target coordinate system is a local coordinate system or a world coordinate system, it is more intuitive. Therefore, calculating the offset distance value through the target coordinate system can be done more intuitively and simply, reducing the difficulty of calculating the offset distance value.

[0018] Secondly, embodiments of this application also provide a vehicle state determination device, comprising: an acquisition module for acquiring a positioning point in a target area of ​​a target vehicle in a target vehicle image; a calculation module for calculating an offset distance value of the positioning point in a historical preset frame using position data of the positioning point in a target vehicle image of a historical preset frame; and a determination module for determining the vehicle state corresponding to the shaking of the target vehicle based on the relationship between the offset distance value and a distance threshold.

[0019] Thirdly, embodiments of this application also provide an electronic device, including: a processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the machine-readable instructions are executed by the processor to perform the steps of the method described in the first aspect above, or any possible implementation of the first aspect.

[0020] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the vehicle state determination method described in the first aspect or any possible implementation of the first aspect.

[0021] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, specific embodiments are described below in conjunction with the accompanying drawings. Attached Figure Description

[0022] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a schematic diagram illustrating the interaction between the image acquisition device and the electronic device provided in the embodiments of this application.

[0024] Figure 2 A block diagram illustrating an electronic device provided in an embodiment of this application;

[0025] Figure 3 A flowchart of the vehicle status determination method provided in the embodiments of this application;

[0026] Figure 4 This is a schematic diagram of the functional modules of the vehicle status determination device provided in the embodiments of this application. Detailed Implementation

[0027] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0028] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0029] With the popularization of autonomous driving technology, autonomous vehicle control has also been widely applied. Autonomous driving requires real-time acquisition of precise coordinates, direction of travel, and vehicle status information to control the vehicle to travel or stop along the desired route. However, some deviations still exist in the current control of vehicles in autonomous driving systems.

[0030] Through long-term research, the inventors of this application have discovered that in autonomous driving, errors in vehicle state determination will affect the automatic control of the vehicle. In other words, the accuracy of vehicle state is fundamental to ensuring automatic vehicle control. However, current methods for determining vehicle state often struggle to distinguish between stationary, trembling vehicles and slowly moving vehicles.

[0031] In view of this, the inventors of this application propose a vehicle state determination method. This method calculates the offset distance value of a positioning point based on the location data of positioning points in historical preset frames, and determines the vehicle state corresponding to the target vehicle based on the relationship between the offset distance value and a distance threshold. Since the shaking intensity of a vehicle varies under different states, the vehicle state of the target vehicle can be determined based on the relationship between the offset distance value and the distance threshold. This achieves vehicle state determination during shaking, reduces misjudgments during vehicle shaking, and improves the accuracy of vehicle state determination during shaking.

[0032] To facilitate understanding of this embodiment, the operating environment for implementing the vehicle state determination method disclosed in this application embodiment will first be described in detail.

[0033] like Figure 1 The diagram illustrates the interaction between an image acquisition device and an electronic device according to an embodiment of this application. The electronic device 100 communicates with one or more image acquisition devices 200 via a network for data communication or interaction. The electronic device 100 can be a web server, database server, personal computer (PC), tablet computer, smartphone, personal digital assistant (PDA), vehicle computer, vehicle navigation system, etc. The image acquisition device can be a camera, webcam, or other device for acquiring target objects.

[0034] The image acquisition device 200 is installed on one or both sides of the target road surface. It can also be installed above the target road surface or at a designated location on the target vehicle (such as the front or chassis). The installation method of the image acquisition device 200 can be adjusted according to actual conditions, and this application does not impose specific limitations.

[0035] The aforementioned electronic device 100 is used to acquire images of the target vehicle captured by the image acquisition device, so as to determine the vehicle status of the target vehicle based on the image of the target vehicle.

[0036] The electronic device that performs the method disclosed in the embodiments of this application will be described in detail below.

[0037] like Figure 2 The diagram shown is a block diagram of an electronic device. The electronic device 100 may include a memory 111, a memory controller 112, a processor 113, a peripheral interface 114, and an input / output unit 115. Those skilled in the art will understand that... Figure 2 The structure shown is for illustrative purposes only and does not limit the structure of the electronic device 100. For example, the electronic device 100 may also include components that are more... Figure 2 The more or fewer components shown, or having the same Figure 2 The different configurations shown.

[0038] The aforementioned memory 111, memory controller 112, processor 113, peripheral interface 114, and input / output unit 115 are electrically connected directly or indirectly to each other to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines. The aforementioned processor 113 is used to execute executable modules stored in the memory.

[0039] The memory 111 can be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The memory 111 stores programs, and the processor 113 executes these programs upon receiving execution instructions. The methods executed by the electronic device 100 as defined in any embodiment of this application can be applied to or implemented by the processor 113.

[0040] The aforementioned processor 113 may be an integrated circuit chip with signal processing capabilities. The processor 113 may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a digital signal processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor.

[0041] The peripheral interface 114 described above couples various input / output devices to the processor 113 and the memory 111. In some embodiments, the peripheral interface 114, the processor 113, and the memory controller 112 can be implemented on a single chip. In other instances, they can be implemented on separate chips.

[0042] The input / output unit 115 described above is used to provide user input data. The input / output unit 115 may be, but is not limited to, a mouse and keyboard, etc.

[0043] The electronic device 100 in this embodiment can be used to execute various steps in the various methods provided in the embodiments of this application. The implementation process of the vehicle state determination method is described in detail below through several embodiments.

[0044] Please see Figure 3 This is a flowchart of the vehicle state determination method provided in the embodiments of this application. The following will describe... Figure 3 The specific process shown will be explained in detail.

[0045] Step 201: Obtain the location points in the target area of ​​the target vehicle in the target vehicle image.

[0046] The target vehicle here is the vehicle that is vibrating. The state of the vehicle when it is vibrating can include a stationary state and a moving state.

[0047] The aforementioned target area refers to the region of the target vehicle where the vibration is most pronounced. For example, the target area could be the engine area, the front of the vehicle, or the cylinder area, etc. The selection of the target area can be adjusted according to the actual situation, and this application does not impose specific restrictions.

[0048] Understandably, if there are obvious markers in the target area of ​​the target vehicle, those markers can be selected as the location points for that target area. For example, if the target area is the front of the target vehicle, the location point could be a point on the vehicle's license plate, such as one of the four vertices of the license plate or the midpoint of one of the four sides of the license plate.

[0049] If there are no obvious markers in the target area of ​​the target vehicle, then a point on the target frame containing that target area can be determined as the positioning point of the target vehicle. If the target area is the cylinder area of ​​the target vehicle, then the positioning point can be one of the four corners of the target frame containing the cylinder, or the midpoint of one of the four sides of the target frame, etc. Here, the target frame refers to the image frame containing the target area detected by the image acquisition device after acquiring the image of the target vehicle.

[0050] The aforementioned target vehicle image is acquired through an image acquisition device. This image can be acquired in real-time, at preset time intervals, or after a vehicle vibration signal is detected. The method of acquiring the target vehicle image can be adjusted according to actual circumstances, and this application does not impose specific limitations.

[0051] Step 202: Calculate the offset distance value of the positioning point in the historical preset frame using the position data of the positioning point in the target vehicle image of the historical preset frame.

[0052] The offset distance value includes an average offset distance value and a Euclidean offset distance value. The average offset distance value is used to determine the intensity of the vibration along the left and right sides of the vehicle when it vibrates. The Euclidean offset distance value is used to determine the intensity of the vibration along the front and rear of the vehicle when it vibrates.

[0053] The location data of the aforementioned positioning point can be either the coordinates of the positioning point in a local coordinate system or the coordinates of the positioning point in a global coordinate system. This local coordinate system is established using the installation equipment where the image acquisition device is located.

[0054] The historical preset frames here refer to a preset time period preceding the current frame. These preset frames can be 5, 10, 20, etc. The setting of these historical preset frames can be adjusted according to parameters such as the target vehicle's speed and the road conditions of the target surface; this application does not impose specific limitations on this.

[0055] Understandably, each frame corresponds to one image, so the corresponding historical target vehicle image can be determined through a historical preset frame. For example, if the historical preset frame is 10 frames, it can correspond to all historical target vehicle images in the 10 frames preceding the current frame, i.e., 10 historical target vehicle images. Each historical target vehicle image corresponds to a positioning point, so 10 positioning points can be further determined.

[0056] After determining the positioning points corresponding to the historical preset frames, the center line of the target vehicle's vibration can be fitted based on these positioning points. The vibration intensity along both sides of the target vehicle can then be determined based on the distance between each positioning point and this center line. Alternatively, the vibration intensity along the front and rear of the target vehicle can be determined based on the Euclidean distance between positioning points in consecutive historical preset frames.

[0057] Step 203: Determine the vehicle state corresponding to the target vehicle vibration based on the relationship between the offset distance value and the distance threshold.

[0058] The distance threshold includes an average distance threshold and a Euclidean distance threshold. The average distance threshold is the maximum force of the target vehicle shaking along both sides of the vehicle when it is in motion, and the Euclidean distance threshold is the maximum force of the target vehicle shaking along the front and rear of the vehicle when it is stationary.

[0059] In some embodiments, if the average offset distance value is less than the average distance threshold and the Euclidean offset distance value is greater than the Euclidean distance threshold, it is determined that the target vehicle is in a state of motion vibration.

[0060] In other embodiments, the target vehicle is determined to be in a stationary shaking state if the average offset distance value is greater than the average distance threshold, or the Euclidean offset distance value is less than the Euclidean distance threshold, or the average offset distance value is greater than the average distance threshold and the Euclidean offset distance value is less than the Euclidean distance threshold, or the average offset distance value is greater than the average distance threshold and the Euclidean offset distance value is less than the Euclidean distance threshold, or the average offset distance value is greater than the average distance threshold and the Euclidean offset distance value is greater than the Euclidean distance threshold.

[0061] In the above implementation process, the offset distance value of the positioning point in the historical frame is determined based on the position data of the positioning point in the vehicle image. Then, the shaking intensity of the target vehicle can be determined by the offset distance value of the positioning point. Since the shaking intensity of the vehicle is different in different states, the vehicle state of the target vehicle can be determined by the relationship between the offset distance value and the distance threshold. This realizes the determination of the vehicle state when shaking, reduces the misjudgment of the vehicle when shaking, and improves the accuracy of the determination of the vehicle state when shaking.

[0062] In one possible implementation, step 202 includes: performing a line fitting on the position data using the least squares method to obtain a target line; determining the average offset distance value of all positioning points from the target line; and calculating the Euclidean offset distance value of the positioning points using the position data.

[0063] In some embodiments, the target straight line can be obtained by fitting the position data using the least squares method. This can be achieved in the following ways:

[0064] A location data sequence is established based on the location data of the target positioning points in historical preset frames. This location data sequence can be represented as follows:

[0065] X = x1, x2, x3…x n ;

[0066] Where X is the location data sequence, x n This is the location data for the nth positioning point, where n = 1, 2, 3, ...

[0067] The target straight line is determined by performing a least-squares fit on the above location data sequence. The formula for this target straight line can be expressed as:

[0068] Ax i -By i +C;

[0069] Where A, B, and C are the parameters of the target line, xi Let y be the value of the i-th positioning point on the first coordinate axis. i The value of the i-th positioning point on the second coordinate axis.

[0070] Understandably, the aforementioned target straight line is obtained by fitting the position data of the positioning points in the historical preset frames. Therefore, this target straight line can be identified as the center line of the target vehicle's vibration in the historical preset frames. Consequently, the offset distance of the target vehicle along its left and right sides can be determined based on the distance between the positioning points and the target straight line.

[0071] In some embodiments, the offset distance values ​​between each positioning point and the target line may be different in the historical preset frame. By determining the average offset distance from all positioning points to the target line in the historical preset frame, the offset distance values ​​along the left and right sides of the target vehicle when it shakes can be determined.

[0072] The aforementioned Euclidean offset distance is the actual distance between two points in two-dimensional or three-dimensional space.

[0073] Understandably, by determining the distance between the positioning points of consecutive frames, the offset distance of the target vehicle along the forward and backward direction of the vehicle during shaking can be determined.

[0074] The location data of the target vehicle image in this historical preset frame can be divided into previous preset frame location data and subsequent preset frame location data. These two sets of location data together constitute the location data of the historical preset frame. Optionally, the previous preset frame location data and subsequent preset frame location data can be divided equally according to the historical preset frames, or they can be divided unevenly.

[0075] For example, if the historical preset frame is 10 frames, then the position data of the previous preset frame can be the position data of the first 5 frames, and the position data of the subsequent preset frame can be the position data of the last 5 frames. Alternatively, the position data of the previous preset frame can be the position data of the first 4 frames, and the position data of the subsequent preset frame can be the position data of the last 6 frames. The position data of the previous preset frame can also be the position data of the first 7 frames, and the position data of the subsequent preset frame can be the position data of the last 3 frames, etc. The division of the position data of the previous and subsequent preset frames can be adjusted according to the actual situation, and this application does not impose specific limitations.

[0076] In the above implementation process, by calculating the average offset distance between the positioning point and the target line, the jitter intensity of the target vehicle along its left and right sides during vibration can be determined. Furthermore, by calculating the distance between positioning points in consecutive frames, the jitter intensity of the target vehicle along its longitudinal direction during vibration can be determined. In other words, the calculation of the target vehicle's offset distance considers the jitter intensity of the positioning point in multiple dimensions, enabling a more accurate determination of the positioning point's offset distance value and improving its accuracy.

[0077] In one possible implementation, calculating the Euclidean offset distance value of the positioning point using position data includes: dividing the position data of the positioning point in the target vehicle image of a historical preset frame into a first part of historical preset frame position data and a second part of historical preset frame position data; determining the first part of the historical preset frame position data as the average position data of the first part of the historical preset frame position data, and the second part of the historical preset frame position data as the average position data of the second part of the historical preset frame position data; and calculating the Euclidean offset distance value between the first part of the historical preset frame average position data and the second part of the historical preset frame average position data.

[0078] The formula for calculating the Euclidean offset distance between the average position data of the first part of historical preset frames and the average position data of the second part of historical preset frames is as follows:

[0079]

[0080] Where, d c c is the Euclidean offset distance value. 1x c represents the value of the average position data of the first set of historical preset frames on the first coordinate axis. 2x The value of the average position data of the second part of the historical preset frames on the first coordinate axis, c 1y c represents the value of the average position data of the second set of historical preset frames on the second coordinate axis. 2y The value of the average position data of the second part of the historical preset frames on the second coordinate axis. Understandably, in order to ensure that the position data of the first part of the historical preset frames has the same weight after being divided, and to reduce the deviation generated during the calculation process, the position data of the historical preset frames can usually be divided equally to obtain the position data of the first part of the historical preset frames and the position data of the second part of the historical preset frames.

[0081] After dividing the position data of historical preset frames, both the first and second parts of the historical preset frame position data include multiple positioning points. By calculating the average position data of multiple positioning points in the first and second parts of the historical preset frame position data, the Euclidean offset distance between the two parts can be determined. Then, based on this Euclidean offset distance, the distance the positioning points move along the longitudinal direction of the vehicle when the target vehicle vibrates can be determined.

[0082] In the above implementation process, when dividing the position data of historical preset frames, the previous and subsequent preset frames are divided equally, so that the weights of the first and second parts of the historical preset frame position data are the same, thereby improving the accuracy of the average position data of the first and second parts of the historical preset frames. Furthermore, the Euclidean offset distance is calculated using the average position data of the first and second parts of the historical preset frames, improving the accuracy of the Euclidean offset distance.

[0083] In one possible implementation, the formula for determining the average offset distance of all positioning points from the target line is:

[0084]

[0085]

[0086] Where A, B, and C are the parameters of the target line, x i Let y be the value of the i-th positioning point on the first coordinate axis. i Let d be the value of the i-th positioning point on the second coordinate axis. i Let d be the distance between the i-th positioning point and the target line, N be the number of positioning points in the target vehicle image of the historical preset frame, and d be the distance between the i-th positioning point and the target line. avg This represents the average offset distance from the positioning point to the target line.

[0087] In the above implementation process, by calculating the average offset distance from all positioning points to the target line, the average offset distance of multiple positioning points along the left and right sides of the target vehicle when the target vehicle shakes can be determined. This method of determining the shaking intensity of the target vehicle along the left and right sides of the vehicle by using the average offset distance of all positioning points in historical preset frames improves the accuracy of the average offset distance value because it integrates the offset distances of all positioning points in historical preset frames.

[0088] In one possible implementation, step 203 includes: if the average offset distance value is less than the average distance threshold and the Euclidean offset distance value is greater than the Euclidean distance threshold, it is determined that the target vehicle is in a state of motion vibration.

[0089] Understandably, when the average offset distance is less than the average distance threshold, it can be determined that the target vehicle exhibits relatively low shaking intensity along its sides. Therefore, it can be preliminarily concluded that the target vehicle is not in a static shaking state.

[0090] When the Euclidean offset distance value is greater than the Euclidean distance threshold, it can be determined that the target vehicle has a large shaking distance along the front-rear direction, indicating that the target vehicle may be moving in addition to shaking. By comprehensively analyzing the shaking of the target vehicle along the front-rear direction and the smaller shaking intensity along the sides, it can be determined whether the target vehicle is in a stationary shaking state or a moving shaking state.

[0091] Obviously, after determining that the vibration intensity of the target vehicle is relatively small along its sides, it can be concluded that the overall vibration intensity of the target vehicle is relatively small. Further determining that the vibration distance of the target vehicle is relatively large along its front-to-back direction, it can be concluded that this vibration distance includes not only the offset distance caused by the vibration of the target vehicle, but also the displacement caused by the vehicle's movement. Therefore, it can be determined that the target vehicle is in a state of motion vibration.

[0092] In the above implementation process, when the average offset distance value is less than the average distance threshold, it can be determined that the target vehicle is experiencing minor vibration. Then, when the target vehicle's vibration is minor, if the Euclidean offset distance value is greater than the Euclidean distance threshold, it can be determined that the target vehicle's Euclidean offset distance value includes not only the offset caused by vibration but also the displacement caused by the target vehicle's movement. Therefore, it can be determined that the target vehicle is in a state of motion vibration. Determining the target vehicle's state using both the average offset distance value and the Euclidean offset distance value improves the accuracy of the target vehicle state assessment.

[0093] In one possible implementation, step 201 includes: inputting the target vehicle image into a deep learning network, and outputting a target detection box after NMS maximum suppression; determining the point at a preset position in the target detection box as a localization point.

[0094] Here, NMS (Non-Maximum Suppression) refers to a technique that removes redundant bounding boxes without affecting the target bounding boxes in the image.

[0095] Understandably, after acquiring an image of the target vehicle, it is necessary to extract the localization points within that image. This can be achieved by inputting the target vehicle image into a deep learning network and performing NMS (Non-Maximum Suppression) processing, which allows the determination of the bounding boxes containing the localization points.

[0096] In some embodiments, to facilitate the determination of positioning points within a target bounding box, points at specific locations within the target bounding box can be designated as positioning points. For example, these preset locations could be vertices or edges of the target bounding box. The positioning point could be a vertex of the target bounding box, the midpoint of one of its edges, etc.

[0097] In the above implementation process, the target bounding boxes in the target vehicle image are detected by using NMS maximum suppression. Since NMS maximum suppression can effectively remove the dark bounding boxes in the target vehicle image, the accuracy of target bounding box detection is improved.

[0098] In one possible implementation, after step 201, the method further includes: projecting the positioning point onto the target coordinate system to obtain the target position corresponding to the positioning point.

[0099] In some embodiments, if the positioning point is a positioning point in the target vehicle image, since the positioning point in the target vehicle image is in the image coordinate system, the offset distance value of the positioning point is not intuitively calculated directly through the image coordinate system due to the limitations of the image coordinate system. In this case, the positioning point can be projected onto the target coordinate system first.

[0100] The target coordinate system is either a local coordinate system or a world coordinate system established by the installation equipment where the image acquisition device is located.

[0101] The local coordinate system here is established using the installation equipment where the image acquisition device is located. For example, the origin of this local coordinate system is the intersection of the installation equipment where the image acquisition device is located and the target road surface. The direction in which the installation equipment moves along the target road surface is the first coordinate axis of the target coordinate system. The direction of the first coordinate axis is the front of the camera of the image acquisition device. The direction of the installation equipment perpendicular to the direction of movement of the target road surface is the second coordinate axis of the ground coordinate system. The direction of the second coordinate axis is the direction of the camera of the image acquisition device.

[0102] The establishment of the local coordinate system described above is merely illustrative; the local coordinate system can also be translated, flipped, rotated, etc., based on this coordinate system.

[0103] In some embodiments, step 202 includes: calculating the offset distance value of the target position in the historical preset frame using the position data of the target position in the target vehicle image of the historical preset frame. Step 203 includes: determining the vehicle state corresponding to the target vehicle shaking based on the relationship between the offset distance value of the target position and a distance threshold.

[0104] In the above implementation process, the positioning point is projected onto the target coordinate system to calculate the offset distance value in the target coordinate system. Since the target coordinate system is a local coordinate system or a world coordinate system, it is more intuitive. Therefore, calculating the offset distance value through the target coordinate system can be done more intuitively and simply, reducing the difficulty of calculating the offset distance value.

[0105] Based on the same application concept, this application also provides a vehicle state determination device corresponding to the vehicle state determination method. Since the principle of the device in this application is similar to that of the aforementioned vehicle state determination method, the implementation of the device in this application can refer to the description in the above method embodiments, and the repeated parts will not be described again.

[0106] Please see Figure 4 This is a functional module diagram of the vehicle state determination device provided in this application embodiment. Each module in the vehicle state determination device in this embodiment is used to execute the steps in the above method embodiments. The vehicle state determination device includes an acquisition module 301, a calculation module 302, and a determination module 303; wherein,

[0107] The acquisition module 301 is used to acquire the positioning points in the target area of ​​the target vehicle image.

[0108] The calculation module 302 is used to calculate the offset distance value of the positioning point in the historical preset frame using the position data of the positioning point in the target vehicle image of the historical preset frame.

[0109] The determination module 303 is used to determine the vehicle state corresponding to the target vehicle vibration based on the relationship between the offset distance value and the distance threshold.

[0110] In one possible implementation, the calculation module 302 is further configured to: perform linear fitting on the position data using the least squares method to obtain a target straight line; determine the average offset distance value of all positioning points to the target straight line; and calculate the Euclidean offset distance value of the positioning points using the position data.

[0111] In one possible implementation, the calculation module 302 is specifically configured to: divide the position data of the target location in the target vehicle image of a historical preset frame into a first part of historical preset frame position data and a second part of historical preset frame position data; determine the first part of the historical preset frame average position data of the first part of the historical preset frame position data, and the second part of the historical preset frame average position data of the second part of the historical preset frame position data; calculate the Euclidean offset distance value between the first part of the historical preset frame average position data and the second part of the historical preset frame average position data; wherein, the formula for calculating the Euclidean offset distance value between the first part of the historical preset frame average position data and the second part of the historical preset frame average position data is: Where, d c c is the Euclidean offset distance value. 1x c represents the value of the average position data of the first set of historical preset frames on the first coordinate axis. 2x The value of the average position data of the second part of the historical preset frames on the first coordinate axis, c 1y c represents the value of the average position data of the second set of historical preset frames on the second coordinate axis. 2y The value of the average position data of the second set of historical preset frames on the second coordinate axis.

[0112] In one possible implementation, the determining module 303 is further configured to: determine that the target vehicle is in a state of motion vibration if the average offset distance value is less than the average distance threshold and the Euclidean offset distance value is greater than the Euclidean distance threshold.

[0113] In one possible implementation, the acquisition module 301 is further configured to: input the target vehicle image into a deep learning network, and output a target detection box after NMS maximum suppression; and determine the point at a preset position in the target detection box as a positioning point.

[0114] In one possible implementation, the vehicle state determination device further includes a projection module for: projecting the positioning point onto a local coordinate system to obtain the target position corresponding to the positioning point, wherein the local coordinate system is a coordinate system established by the installation device where the image acquisition device is located.

[0115] In one possible implementation, the calculation module 302 is specifically used to: calculate the offset distance value of the target position in the historical preset frame using the position data of the target position in the target vehicle image of the historical preset frame.

[0116] In one possible implementation, the determining module 303 is specifically used to: determine the vehicle state corresponding to the target vehicle vibration based on the relationship between the offset distance value of the target position and the distance threshold.

[0117] Furthermore, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the vehicle state determination method described in the above method embodiments.

[0118] The computer program product of the vehicle state determination method provided in this application includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the steps of the vehicle state determination method described in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0119] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0120] In addition, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0121] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks. It should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0122] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application. It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0123] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for determining the state of a vehicle, characterized in that, include: Obtain the location points within the target region of the target vehicle in the target vehicle image; The offset distance value of the positioning point in the target vehicle image of the historical preset frame is calculated using the position data of the positioning point in the historical preset frame. The vehicle state corresponding to the target vehicle vibration is determined based on the relationship between the offset distance value and the distance threshold. The offset distance value includes an average offset distance value and a Euclidean offset distance value. The step of calculating the offset distance value of the positioning point in the historical preset frame using the position data of the positioning point in the target vehicle image of the historical preset frame includes: The position data is fitted with a straight line using the least squares method to obtain the target straight line; Determine the average offset distance from all positioning points to the target line; The Euclidean offset distance of the positioning point is calculated using the location data.

2. The method according to claim 1, characterized in that, The step of calculating the Euclidean offset distance value of the positioning point using the location data includes: The location data of the positioning point in the target vehicle image of the historical preset frame is divided into the first part of the historical preset frame location data and the second part of the historical preset frame location data. Determine the first part of the historical preset frame position data average position data, and the second part of the historical preset frame position data average position data. Calculate the Euclidean offset distance between the first part of the historical preset frame average position data and the second part of the historical preset frame average position data; The formula for calculating the Euclidean offset distance between the first part of the historical preset frame average position data and the second part of the historical preset frame average position data is as follows: ; in, This is the Euclidean offset distance value. The first part is the value of the average position data of the historical preset frames on the first coordinate axis. The second part is the value of the historical preset frame average position data on the first coordinate axis. The value of the average position data of the first set of historical preset frames on the second coordinate axis. The value of the average position data of the second set of historical preset frames on the second coordinate axis.

3. The method according to claim 1, characterized in that, The formula for determining the average offset distance from all positioning points to the target line is as follows: Where A, B, and C are the parameters of the target line. For the first The values ​​of each positioning point on the first coordinate axis For the first The values ​​of each positioning point on the second coordinate axis For the first The distance between each positioning point and the target line. The number of location points in the target vehicle image of the historical preset frame. This represents the average offset distance from the positioning point to the target line.

4. The method according to claim 1, characterized in that, The distance threshold includes an average distance threshold and a Euclidean distance threshold. Determining the vehicle state corresponding to the target vehicle vibration based on the relationship between the offset distance value and the distance threshold includes: If the average offset distance value is less than the average distance threshold and the Euclidean offset distance value is greater than the Euclidean distance threshold, the target vehicle is determined to be in a state of motion vibration.

5. The method according to claim 1, characterized in that, The step of acquiring the location points in the target region of the target vehicle image includes: The target vehicle image is input into a deep learning network, and after NMS maximum suppression, a target detection box is output. The point at the preset position in the target detection frame is determined as the positioning point.

6. The method according to any one of claims 1-5, characterized in that, After obtaining the location points in the target region of the target vehicle image, the method further includes: The positioning point is projected onto the target coordinate system to obtain the target position corresponding to the positioning point. The target coordinate system is a local coordinate system or a world coordinate system established by the installation equipment where the image acquisition device is located. The step of calculating the offset distance value of the positioning point in the historical preset frame using the position data of the positioning point in the target vehicle image of the historical preset frame includes: The offset distance value of the target position in the historical preset frame is calculated using the position data of the target position in the target vehicle image of the historical preset frame; Determining the vehicle state corresponding to the target vehicle vibration based on the relationship between the offset distance value and the distance threshold includes: The vehicle state corresponding to the target vehicle vibration is determined based on the relationship between the offset distance value of the target position and the distance threshold.

7. A vehicle state determination device, characterized in that, include: The acquisition module is used to acquire the positioning points in the target area of ​​the target vehicle in the target vehicle image; The calculation module is used to calculate the offset distance value of the positioning point in the historical preset frame using the position data of the positioning point in the target vehicle image of the historical preset frame; The determination module is used to determine the vehicle state corresponding to the target vehicle vibration based on the relationship between the offset distance value and the distance threshold; Specifically, the calculation module is used for: The offset distance value includes the average offset distance value and the Euclidean offset distance value; The position data is fitted with a straight line using the least squares method to obtain the target straight line; Determine the average offset distance from all positioning points to the target line; The Euclidean offset distance of the positioning point is calculated using the location data.

8. An electronic device, characterized in that, include: The processor and memory, wherein the memory stores machine-readable instructions executable by the processor, wherein when the electronic device is running, the machine-readable instructions are executed by the processor to perform the steps of the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the method as described in any one of claims 1 to 6.

Images