Training method and device for vehicle driving direction detection model and detection method and device for vehicle running status

Abstract

The embodiments of the present invention disclose a training method, a detection method, and an apparatus for a vehicle operation status detection model. The method involves acquiring image data of a vehicle at multiple moments; then performing convolution calculations on the image data using multiple convolution kernels included in a preset initial vehicle operation status detection model to obtain time-series features of the vehicle at multiple moments; then determining the first operation status information of the vehicle based on the time-series features of the vehicle at multiple moments; and finally updating the parameters of the initial vehicle operation status detection model based on the first operation status information and pre-stored second operation status information corresponding to the image data to obtain a trained vehicle operation status detection model for detecting the vehicle operation status. The embodiments of the present invention solve the problem of the inability to accurately identify the vehicle's driving direction in current technical solutions, thereby achieving accurate identification of the vehicle's driving direction.

Description

Technical Field

[0001] The present invention relates to the field of target recognition technology, and in particular to a training method, detection method, device, equipment and storage medium for a vehicle operating status detection model. Background Art

[0002] Autonomous driving technology in cars will benefit society, drivers and pedestrians because it will reduce the incidence of traffic accidents.

[0003] Currently, research on vehicle direction recognition in autonomous driving technology is limited by preconditions or the target object needs to be in a relatively simple environment. Therefore, when faced with special situations such as the target's posture changes, scale changes, in-plane rotation, and occlusion, the current algorithms cannot accurately identify the vehicle's direction, making target tracking technology difficult to apply in real life.

[0004] Therefore, the current technical solution has the problem of being unable to accurately identify the direction of vehicle travel. Summary of the Invention

[0005] The embodiments of the present invention provide a training method, detection method, device, equipment and storage medium for a vehicle operation status detection model, which solves the problem of the inability to accurately identify the vehicle's driving direction in the current technical solution, thereby achieving accurate identification of the vehicle's driving direction.

[0006] In order to solve the above technical problems, the present invention:

[0007] In a first aspect, a method for training a vehicle operation status detection model is provided, the training method comprising:

[0008] Acquire image data of the vehicle at multiple times;

[0009] Perform convolution calculations on the image data using multiple convolution kernels included in the preset initial vehicle operation status detection model to obtain temporal features of the vehicle at multiple moments;

[0010] Determining first operating state information of the vehicle based on time-series characteristics of the vehicle at multiple moments;

[0011] According to the first operating state information and pre-stored second operating state information corresponding to the image data, the parameters of the initial vehicle operating state detection model are updated to obtain a trained vehicle operating state detection model.

[0012] In some implementations of the first aspect, the training method further includes:

[0013] Performing grayscale value balancing processing on the image data to obtain first image data;

[0014] The first image data is convolved using multiple convolution kernels included in the initial vehicle detection model to obtain temporal features of the vehicle at multiple moments.

[0015] In some implementations of the first aspect, the method further includes:

[0016] removing background feature data from the image data based on the grayscale value of each pixel in the image data at each moment, the average grayscale value of the image data, and a preset background feature threshold, to obtain second image data, wherein the average grayscale value is determined based on the grayscale values ​​of pixels whose grayscale value change rate between the image data at two moments is less than the threshold;

[0017] The second image data is convolved using multiple convolution kernels included in the initial vehicle running state detection model to obtain temporal features of the vehicle at multiple moments.

[0018] In some implementations of the first aspect, determining first operating state information of the vehicle based on time-series features of the vehicle at multiple moments includes:

[0019] Superimpose the time series features of the vehicle at multiple moments to obtain the superimposed features;

[0020] First operating state information of the vehicle is determined according to the superimposed features.

[0021] In some implementations of the first aspect, the preset initial vehicle operating state detection model includes multiple convolution kernels in a first direction and / or multiple convolution kernels in a second direction; the first direction and the second direction are perpendicular to each other; the time series feature includes a first time series feature and a second time series feature; and the training method further includes:

[0022] Performing convolution calculation on the image data using multiple convolution kernels in a first direction to obtain first temporal features of the vehicle at multiple moments, and / or performing convolution calculation on the image data using multiple convolution kernels in a second direction to obtain second temporal features of the vehicle at multiple moments;

[0023] First operating state information of the vehicle is determined based on the first time-series characteristics and / or the second time-series characteristics of the vehicle at multiple moments.

[0024] In some implementations of the first aspect, updating parameters of an initial vehicle operating state detection model based on the first operating state information and pre-stored second operating state information corresponding to the image data to obtain a trained vehicle detection model includes:

[0025] determining a loss value between the first operating status information and the second operating status information;

[0026] Update the parameters of the convolution kernel according to the loss value;

[0027] When the loss value meets the preset conditions, the vehicle operation status detection model corresponding to the loss value meeting the preset conditions is used as the trained vehicle operation status detection model.

[0028] In a second aspect, a method for detecting a vehicle operating state is provided, the method comprising:

[0029] Acquire image data of the vehicle at multiple times;

[0030] The image data is input into a vehicle operation status detection model to obtain the vehicle operation status, wherein the vehicle operation status detection model is obtained based on the first aspect and any one of the vehicle operation status detection model training methods in some implementations of the first aspect.

[0031] In a third aspect, a training device for a vehicle operating state detection model is provided, the training device comprising:

[0032] An acquisition module, used to acquire image data of the vehicle at multiple times;

[0033] A processing module, configured to perform convolution calculations on the image data using multiple convolution kernels included in a preset initial vehicle operation state detection model to obtain time series features of the vehicle at multiple moments;

[0034] The processing module is further configured to determine first operating state information of the vehicle based on time-series characteristics of the vehicle at multiple moments;

[0035] The processing module is further used to update the parameters of the initial vehicle operating status detection model according to the first operating status information and the pre-stored second operating status information corresponding to the image data to obtain a trained vehicle operating status detection model.

[0036] In some implementations of the third aspect, the processing module is further used to superimpose the time-series features of the vehicle at multiple moments to obtain superimposed features; and determine the first operating state information of the vehicle based on the superimposed features.

[0037] In some implementations of the third aspect, the preset initial vehicle operating status detection model includes multiple convolution kernels in a first direction and / or multiple convolution kernels in a second direction; the first direction and the second direction are perpendicular to each other; the timing features include a first timing feature and a second timing feature.

[0038] Therefore, in some implementations of the third aspect, the processing module is also used to perform convolution calculations on the image data using multiple convolution kernels in a first direction to obtain first temporal features of the vehicle at multiple moments, and / or to perform convolution calculations on the image data using multiple convolution kernels in a second direction to obtain second temporal features of the vehicle at multiple moments; and determine the first operating status information of the vehicle based on the first temporal features and / or second temporal features of the vehicle at multiple moments.

[0039] In some implementations of the third aspect, the processing module is further used to determine a loss value between the first operating status information and the second operating status information; update the parameters of the convolution kernel according to the loss value; and when the loss value meets a preset condition, use the vehicle operating status detection model corresponding to the loss value that meets the preset condition as the trained vehicle operating status detection model.

[0040] In some implementations of the third aspect, the processing module is further used to perform grayscale value balancing on the image data to obtain first image data, and perform convolution calculation on the first image data using multiple convolution kernels included in the initial vehicle detection model to obtain temporal features of the vehicle at multiple moments.

[0041] In some implementations of the third aspect, the processing module is further used to remove background feature data in the image data based on the grayscale value of each pixel in the image data at each moment, the average grayscale value of the image data, and a preset background feature threshold to obtain second image data, wherein the average grayscale value is determined based on the grayscale value of the pixel whose grayscale value change rate between the image data at two moments is less than the threshold; and use multiple convolution kernels included in the initial vehicle operation status detection model to perform convolution calculation on the second image data to obtain temporal features of the vehicle at multiple moments.

[0042] In a fourth aspect, a vehicle operating status detection device is provided, the detection device comprising:

[0043] An acquisition module, used to acquire image data of the vehicle at multiple times;

[0044] A processing module is used to input image data into a vehicle operation status detection model to obtain the vehicle operation status, wherein the vehicle operation status detection model is obtained based on the first aspect and any one of the vehicle operation status detection model training methods in some implementation methods of the first aspect.

[0045] In a fifth aspect, an electronic device is provided, the device comprising: a processor and a memory storing computer program instructions;

[0046] When the processor executes the computer program instructions, it implements the first aspect and the training method of the vehicle operation status detection model in some implementations of the first aspect, or implements the vehicle operation status detection method of the second aspect.

[0047] In a sixth aspect, a computer storage medium is provided, characterized in that computer program instructions are stored on the computer storage medium, and when the computer program instructions are executed by a processor, the training method of the vehicle operation status detection model in the first aspect and some implementation methods of the first aspect is implemented, or the vehicle operation status detection method of the second aspect is implemented.

[0048] Embodiments of the present invention provide a training method, detection method, apparatus, device, and storage medium for a vehicle operating state detection model. Image data of a vehicle at multiple moments is acquired; then, multiple convolution kernels included in a preset initial vehicle operating state detection model are used to perform convolution calculations on the image data to obtain time-series features of the vehicle at multiple moments. Because features are obtained by directly performing convolution calculations on the image data, the target positioning step is skipped, thereby improving detection efficiency. First operating state information of the vehicle is then determined based on the time-series features of the vehicle at multiple moments; parameters of the initial vehicle operating state detection model are updated based on the first operating state information and pre-stored second operating state information corresponding to the image data, thereby obtaining a trained vehicle operating state detection model for detecting the vehicle's operating state. Furthermore, because the first operating state information of the vehicle is determined based on the obtained time-series features of the vehicle at multiple moments, and the parameters of the trained vehicle operating state detection model are updated based on pre-stored second operating state information corresponding to the image data, detection accuracy is also improved, resolving the problem of the inability to accurately identify the vehicle's driving direction in current technical solutions and improving recognition efficiency and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0050] Figure 1 1 is a flow chart of a method for training a vehicle operation status detection model provided by an embodiment of the present invention;

[0051] Figure 2 This is a schematic diagram of superimposing X-axis features at multiple moments provided by an embodiment of the present invention;

[0052] Figure 3This is a schematic diagram of superimposing Y-axis features at multiple moments provided by an embodiment of the present invention;

[0053] Figure 4 is a grayscale histogram of image data before grayscale value balancing processing provided by an embodiment of the present invention;

[0054] Figure 5 A grayscale histogram of image data after grayscale value balancing processing is provided in an embodiment of the present invention;

[0055] Figure 6 1 is a flow chart of a method for detecting a vehicle operating state provided by an embodiment of the present invention;

[0056] Figure 7 is a schematic diagram of a training device for a vehicle operating state detection model provided by an embodiment of the present invention;

[0057] Figure 8 1 is a schematic diagram of a vehicle operating status detection device provided by an embodiment of the present invention;

[0058] Figure 9 This is a structural diagram of a computing device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0059] The features and exemplary embodiments of various aspects of the present invention will be described in detail below. In order to make the objects, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below in conjunction with the accompanying drawings and Examples. It should be understood that the specific embodiments described herein are only configured to explain the present invention and are not configured to limit the present invention. For those skilled in the art, the present invention can be implemented without the need for some of these specific details. The following description of the embodiments is merely to provide a better understanding of the present invention by illustrating examples of the present invention.

[0060] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, the elements defined by the phrase "comprising..." do not exclude the presence of other identical elements in the process, method, article, or device comprising the elements.

[0061] Autonomous vehicles are intelligent vehicles that use onboard sensor systems to perceive the road environment, automatically plan routes, and control the vehicle to reach a predetermined destination. They utilize onboard sensors to sense the vehicle's surroundings and, based on this information about the road, vehicle position, and obstacles, control the vehicle's steering and speed, enabling safe and reliable road travel. Integrating numerous technologies, including automatic control, architecture, artificial intelligence, and visual computing, autonomous vehicles are a product of the advanced development of computer science, pattern recognition, and intelligent control. They are also a key indicator of a country's scientific research strength and industrial development, and have broad application prospects in national defense and the national economy.

[0062] The accident rate associated with self-driving cars could drop to virtually zero. Even with the impact of other car accident rates, the rapid growth of the self-driving car market share will steadily reduce the overall accident rate. Therefore, the driving mode of self-driving cars can be more energy-efficient, reducing traffic congestion and air pollution. Consequently, the widespread adoption of self-driving cars will significantly reduce government investment in traffic infrastructure such as extra-wide lanes, guardrails, speed bumps, wide shoulders, and even stop signs. Therefore, self-driving cars benefit society, drivers, and pedestrians alike.

[0063] In the process of self-driving cars achieving autonomous driving, target detection is an important process because it can identify the location of the object to be identified and other information about the object to be identified.

[0064] Specifically, object detection refers to the process of finding, identifying, and locating an object in an image. This complex process requires not only identifying the object but also defining its specific location within the image using a bounding box. The main process consists of three steps: first, extracting candidate regions from the image, then identifying the category to which the candidate regions belong, and finally, accurately locating the valid candidate regions.

[0065] When detecting objects to identify vehicle direction, traditional techniques first use a brute-force search to select a candidate region in the image. Feature extraction is then performed on the candidate region. Next, classification methods such as SVM are used to classify the candidate region, determining whether the candidate region contains the target and its category. Finally, methods such as bounding box regression and non-maximum suppression are used to accurately locate the target. Traditional object detection algorithms primarily include the following five: optical flow, inter-frame difference, background difference, SVM feature detection, and AdaBoost feature detection. These five algorithms are described below, focusing on their applicable scenarios and their limitations, as shown in Table 1.

[0066] Table 1

[0067]

[0068]

[0069] As shown in Table 1, most traditional target detection algorithms used to detect vehicle direction are subject to preconditions or require the target object to be in a relatively simple environment, such as a static background or tracking a non-deformable target. This shows that target tracking algorithms face the impact of unconstrained conditions in complex scenarios, especially in real-world scenarios. These conditions, such as changes in the target's posture, scale, in-plane rotation, and occlusion, pose significant challenges to the research and application of target tracking technology, making it difficult to apply in real life.

[0070] Therefore, the current technical solution has the problem of being unable to accurately identify the direction of vehicle travel.

[0071] In order to solve the problem that the current technical solution cannot accurately identify the direction of vehicle travel, the embodiment of the present invention provides a training method, detection method, device, equipment and storage medium for a vehicle operation status detection model. By acquiring image data of the vehicle at multiple moments; then using multiple convolution kernels included in the preset initial vehicle operation status detection model to perform convolution calculation on the image data, the time-series features of the vehicle at multiple moments are obtained. Because the features are obtained by directly performing convolution calculation on the image data, the target positioning step is skipped, so the efficiency of detection is improved. Then, the first operation status information of the vehicle is determined based on the time-series features of the vehicle at multiple moments; based on the first operation status information and the pre-stored second operation status information corresponding to the image data, the parameters of the initial vehicle operation status detection model are updated to obtain a trained vehicle operation status detection model for detecting the operation status of the vehicle. In addition, because the first operating status information of the vehicle is determined based on the temporal characteristics of the vehicle at multiple moments, and the parameters are updated based on the second operating status information corresponding to the image data stored in advance to obtain a trained vehicle operating status detection model for detecting the operating status of the vehicle, the detection accuracy is also improved, which solves the problem of the current technical solution that cannot accurately identify the vehicle's driving direction, and improves the recognition efficiency and accuracy.

[0072] The technical solutions provided by the embodiments of the present invention are described below with reference to the accompanying drawings.

[0073] Figure 1 FIG. 1 is a flow chart of a method for training a vehicle operation status detection model provided by an embodiment of the present invention. Figure 1As shown, the execution subject of the method may include a terminal device, such as a vehicle terminal or a server, and the method includes:

[0074] S101: Acquire image data of the vehicle at multiple moments.

[0075] Because certain specific features appear regularly at different times when a vehicle moves in the direction of travel, image data including vehicle information to be identified at a certain time interval, i.e., multiple moments, can be obtained through a camera component or from a video stream of offline data.

[0076] In one embodiment, in order to facilitate subsequent calculations, the image data of the vehicle at multiple moments can be expressed using formula (1).

[0077] Seq{I0,I1,I2,...,I n},n≥1 (1)

[0078] In formula (1), Seq{} represents the image data sequence, I0, I1, I2, ..., I n Represents the elements in the sequence, which respectively represent the image data of frames 0 to n in the video stream.

[0079] In addition, in order to make the collected image data take into account the actual situation, include as few repeated features as possible and as many high-value features as possible, so when collecting image data, the I can be determined based on information such as vehicle speed and the properties of the collection equipment. n with I n-1 The interval between frames is m.

[0080] In one embodiment, the process of determining the interval frame number m may be as shown in formula (2).

[0081] v×m / p / u≥p_mi (2)

[0082] That is, the interval frame number m must satisfy m ≥ p_mi × p × u × v. Here, v is the minimum vehicle speed, measured in meters per second (m / s); p is the frame rate of the video capture device; u is the image resolution, representing the distance of one pixel in the actual scene, measured in meters per pixel (m / pix); and p_mi is the smallest processing unit supported by the algorithm, measured in pixels (pix).

[0083] In addition, it should be noted that since m means a specific number of frames, it needs to be an integer. Therefore, in one embodiment, m also needs to satisfy m≥ceil(p_mi×p×u / v), where ceil(x) means the smallest integer not less than x.

[0084] After acquiring the image data of the vehicle at multiple moments, subsequent calculations may be performed on the image data to train a vehicle operation status detection model, ie, proceed to S102 .

[0085] S102: Perform convolution calculation on the image data using multiple convolution kernels included in a preset initial vehicle running state detection model to obtain temporal features of the vehicle at multiple moments.

[0086] Specifically, multiple convolution kernels included in the preset initial vehicle running state detection model can be used to calculate the image data at multiple moments respectively to obtain the temporal features of the vehicle at multiple moments.

[0087] In one embodiment, taking the use of a convolution kernel to calculate a certain pixel in the image data as an example, the calculation process can be shown as formula (3).

[0088]

[0089] Among them, i represents the pixel in the i-th row of the image data, j represents the pixel in the j-th column of the image data, and a i,j It represents the matrix after convolution calculation of the pixel at position i row and column j in a frame of image data. d represents the channel of the convolution kernel, m is the width of the convolution kernel, n is the height of the convolution kernel, F is the width of the filter, and w is the width of the filter. b is the bias value, w d,m,n Represents a convolution kernel with d channels, m width, and n height.

[0090] According to formula (3), convolution calculation is performed once for each pixel in a frame of image data, and finally the image data after convolution calculation is obtained. Because it is for a single frame, this process only obtains the temporal features of the vehicle at a single moment. Then, convolution calculation is performed on each frame of image data in multiple frames using formula (3) to obtain the temporal features of multiple moments.

[0091] In addition, it should be noted that the height and width of the matrix determinant of the image data (Feature Map) after convolution calculation can be shown as formula (4):

[0092]

[0093] Among them, W2 is the width of the Feature Map after convolution; W1 is the width of the image before convolution; H2 is the height of the Feature Map after convolution; H1 is the height of the image before convolution; F is the width of the filter; P is the number of Zero Paddings, which refers to the number of circles of zeros padded around the original image. If the value of P is 1, then 1 circle of zeros is padded; Sx is the stride of the convolution kernel in the x direction, Sy is the stride of the convolution kernel in the y direction; H2 is the height of the Feature Map after convolution.

[0094] In addition, in order to more efficiently determine the vehicle operating status, in one embodiment, the multiple convolution kernels in the preset initial vehicle operating status detection model can be divided into multiple convolution kernels in the first direction and multiple convolution kernels in the second direction according to different detection functions. Therefore, the preset initial vehicle operating status detection model may include multiple convolution kernels in the first direction and / or multiple convolution kernels in the second direction. Among them, the multiple convolution kernels in the first direction can be used to determine the characteristics of the vehicle on the X-axis, and the multiple convolution kernels in the second direction can be used to determine the characteristics of the vehicle on the Y-axis, and the X-axis and the Y-axis can be perpendicular to each other. In addition, the relationship between the X-axis and the Y-axis can also be adjusted according to actual conditions.

[0095] Therefore, multiple convolution kernels in a first direction can be used to perform convolution calculations on the image data to obtain first temporal features of the vehicle at multiple moments. The first temporal features can refer to the features of the vehicle on the X-axis. Multiple convolution kernels in a second direction can also be used to perform convolution calculations on the image data to obtain second temporal features of the vehicle at multiple moments. The second temporal features can refer to the features of the vehicle on the Y-axis. In addition, multiple convolution kernels in the first direction and multiple convolution kernels in the second direction can also be used to perform convolution calculations on the image data simultaneously to obtain features of the vehicle on the X-axis and features on the Y-axis.

[0096] S103: Determine first operating state information of the vehicle based on temporal characteristics of the vehicle at multiple moments.

[0097] In this process, the first operating state of the vehicle may be determined according to the X-axis characteristics and / or the Y-axis characteristics obtained in S102 .

[0098] In one embodiment, when only X-axis features of the vehicle are obtained at multiple moments, these X-axis features can be superimposed. Based on the superimposed features, the vehicle's operating state on the X-axis can be determined. In this case, the X-axis operating state can be referred to as the first operating state. In one embodiment, the X-axis operating state can refer to a left turn or a right turn.

[0099] Figure 2This is a schematic diagram of the superposition of X-axis features at multiple moments. I0, I1, and I2 represent the elements in the image data sequence to be convolved. The area indicated by the scissor 1 represents multiple convolution kernels (Conv). The area indicated by the scissor 2 represents the results of convolution calculations on different elements. The area indicated by the scissor 3 represents the laminated layer. The area indicated by the scissor 4 represents the features of the X-axis features after the layers are superimposed. The area indicated by the scissor 5 represents the features generated by multiple subsequent superpositions of the X-axis features.

[0100] In one embodiment, the features generated by multiple subsequent superposition of X-axis features can be as shown in formula (5).

[0101] C hori =|C0 C1 C2 ... C n | (5)

[0102] Among them, C hori Represents the set of superimposed X-axis features, C0, C1, C2…C n Represents I0, I1, I2…I n The corresponding result is calculated by convolution.

[0103] Similarly, in one embodiment, when only Y-axis features of the vehicle at multiple moments are obtained, these Y-axis features can be superimposed and the vehicle's operating state on the Y-axis determined based on the superimposed features. In this case, the Y-axis operating state can be referred to as the first operating state. In one embodiment, the Y-axis operating state can refer to a forward or reverse operating state, where forward can also refer to acceleration and reverse can also refer to braking.

[0104] Figure 3 This is a schematic diagram of the superposition of Y-axis features at multiple moments. Here, I0, I1, and I2 also represent the elements in the image data sequence to be convolved. The area indicated by the 6th scissor represents multiple convolution kernels (Conv), the area indicated by the 7th scissor represents the calculation results of convolution calculations on different elements, the area indicated by the 8th scissor represents the laminated frag layer, the area indicated by the 9th scissor represents the features of the Y-axis features after the frag layer is superimposed, and the area indicated by the 10th scissor represents the features generated by multiple subsequent superpositions of the Y-axis features.

[0105] In one embodiment, the features generated by multiple subsequent superposition of Y-axis features can be as shown in formula (6).

[0106]

[0107] Among them, C vertRepresents the set of superimposed Y-axis features, C0, C1, C2…C n Represents I0, I1, I2…I n The corresponding result is calculated by convolution.

[0108] In one embodiment, after simultaneously obtaining the vehicle's X-axis and Y-axis features at multiple moments, these features can be superimposed to determine the vehicle's operating states along the X and Y axes. The vehicle's omnidirectional operating state can then be determined based on the vehicle's operating states along the X and Y axes. This omnidirectional operating state can be referred to as the first operating state.

[0109] S104: updating the parameters of the initial vehicle operating state detection model according to the first operating state information and pre-stored second operating state information corresponding to the image data to obtain a trained vehicle operating state detection model.

[0110] Specifically, in this process, the loss value can be determined based on the first operating status information determined in S103 and the pre-stored second operating status information corresponding to the image data obtained in S101, and then the parameters of multiple convolution kernels in the initial vehicle operating status detection model are updated based on the loss value; when the loss value meets the preset conditions, the vehicle operating status detection model corresponding to the loss value that meets the preset conditions is used as the trained vehicle operating status detection model.

[0111] An embodiment of the present invention provides a method for training a vehicle operating state detection model. The method involves acquiring image data of a vehicle at multiple moments; then performing convolution calculations on the image data using multiple convolution kernels included in a preset initial vehicle operating state detection model to obtain time-series features of the vehicle at multiple moments. Because features are obtained by directly performing convolution calculations on the image data, the target positioning step is skipped, thereby improving detection efficiency. Feature superposition is then performed based on the time-series features of the vehicle at multiple moments to determine first operating state information of the vehicle; parameters of the initial vehicle operating state detection model are updated based on the first operating state information and pre-stored second operating state information corresponding to the image data, thereby obtaining a trained vehicle operating state detection model for detecting the vehicle's operating state. Furthermore, because the first operating state information of the vehicle is determined by superposition based on the obtained time-series features of the vehicle at multiple moments, and the parameters of the trained vehicle operating state detection model are updated based on pre-stored second operating state information corresponding to the image data, detection accuracy is also improved, resolving the problem of the inability to accurately identify the vehicle's driving direction in current technical solutions and improving recognition efficiency and accuracy.

[0112] In one embodiment, in order to enhance the features of the image data obtained in S101 and reduce the impact of natural conditions such as weather changes, during the execution of S102 on the vehicle side or the cloud server, the image data can be grayscale balanced to obtain first image data; then, the first image data is convolved using multiple convolution kernels included in the initial vehicle detection model to obtain temporal features of the vehicle at multiple moments.

[0113] The process of performing grayscale value balancing on the image data to obtain the first image data can be shown as formula (7):

[0114]

[0115] Where cdf() is the cumulative distribution function, cdf min is the preset minimum grayscale value, M and N are the width and height of the image data, L is the preset grayscale value range, and v is the current grayscale value. In addition, it should be noted that cdf min And L can be adjusted according to actual conditions to adapt to different situations.

[0116] In one example, the grayscale histogram of the image data before grayscale value balancing is as follows: Figure 4 As shown by Figure 4 It can be seen that the grayscale values ​​of the image data are concentrated together, and the grayscale dynamic range and contrast are relatively small, which is not conducive to the subsequent vehicle motion state recognition. The grayscale histogram of the image data after grayscale value balancing calculation processing using formula (7) is as follows: Figure 5 shown.

[0117] As Figure 5 As shown in the figure, the grayscale histogram of the image data almost covers the entire grayscale value range, and except for the number of individual grayscale values ​​that are more prominent, the entire grayscale value distribution is approximately uniform. Therefore, this image has a larger grayscale dynamic range and higher contrast, and the image details are richer, so it is possible to enhance the characteristics of the image data and reduce the influence of natural conditions such as weather changes, which is beneficial to the later recognition of vehicle motion status.

[0118] In one embodiment, in order to eliminate the influence of part of the background in the image data on the target, during the execution of S102 on the vehicle side or the cloud server, the background feature data in the image data can be removed according to the grayscale value of each pixel in the image data at each moment, the average grayscale value of the image data and the preset background feature threshold to obtain second image data, wherein the average grayscale value is determined based on the grayscale value of the pixel whose grayscale value change rate between the image data at two moments is less than the threshold; then, the second image data is convolved using the multiple convolution kernels included in the initial vehicle operation status detection model to obtain the temporal features of the vehicle at multiple moments.

[0119] The process of removing background feature data from the image data according to the grayscale value of each pixel in the image data at each moment, the average grayscale value of the image data, and the preset background feature threshold to obtain the second image data can be shown as formula (8):

[0120]

[0121] Among them, σ represents the calculated standard deviation, N represents the number of pixels of the image data at a certain moment, and x i It represents the gray value of the i-th pixel in the image data at a certain moment, and μ is the average gray value.

[0122] A threshold T can then be preset. If σ<the threshold T, the image data is considered to be a background feature, and the grayscale value of the image data is set to zero to eliminate the influence of the background on the target.

[0123] The training method of the vehicle operation status detection model provided by the embodiment of the present invention is achieved by acquiring image data of the vehicle at multiple moments; then the image data is subjected to gray value balancing processing and background feature data removal processing to enhance the features of the image data, reduce the influence of natural conditions such as weather changes, and eliminate the influence of background features on the later detection of the vehicle's operation status. Then, the multiple convolution kernels included in the preset initial vehicle operation status detection model are used to perform convolution calculation on the processed image data to obtain the time-series features of the vehicle at multiple moments. Because the features are obtained by directly performing convolution calculation on the image data, the target positioning step is skipped, so that the efficiency of detection is improved. Then, the first operation status information of the vehicle is determined by performing feature superposition based on the time-series features of the vehicle at multiple moments; based on the first operation status information and the pre-stored second operation status information corresponding to the image data, the parameters of the initial vehicle operation status detection model are updated to obtain the trained vehicle operation status detection model for detecting the vehicle's operation status. In addition, because the first operating status information of the vehicle is determined by feature superposition based on the temporal features of the vehicle at multiple moments, and the parameters are updated based on the second operating status information corresponding to the pre-stored image data to obtain a trained vehicle operating status detection model for detecting the operating status of the vehicle, the detection accuracy is also improved, which solves the problem of the current technical solution that cannot accurately identify the vehicle's driving direction, and improves the recognition efficiency and accuracy.

[0124] Figure 6 It is a flow chart of a method for detecting a vehicle operating status provided by an embodiment of the present invention.

[0125] like Figure 6 As shown, the method for detecting the vehicle operating status may include:

[0126] S201: Acquire image data of the vehicle at multiple moments.

[0127] S201: Inputting the image data into a vehicle operation status detection model to obtain the vehicle operation status.

[0128] Among them, the vehicle running state detection model is based on Figure 1 The training method of the vehicle operation status detection model is obtained.

[0129] The vehicle running state detection method provided by the embodiment of the present invention is to obtain the image data of the vehicle at multiple moments and then input the image data into the vehicle running state detection method according to the vehicle running state detection method. Figure 1 In the vehicle operation status detection model obtained by the training method of the vehicle operation status detection model in , convolution calculation is performed to obtain the time series features of the vehicle at multiple moments. Figure 1The vehicle operation status detection model in this paper directly performs convolution calculations on image data to obtain features, skipping the target positioning step, thereby improving detection efficiency. Furthermore, by superimposing the vehicle's time-series features at multiple moments to determine the vehicle's operation status, the problem of current technical solutions that cannot accurately identify the vehicle's driving direction is solved, improving recognition efficiency and accuracy.

[0130] Corresponding to the embodiment of the vehicle running state detection model training method, the embodiment of the present invention also provides a vehicle running state detection model training device, such as Figure 7 shown. Figure 7 4 is a schematic diagram of a training device for a vehicle operation status detection model provided by an embodiment of the present invention.

[0131] Figure 7 The training device of the vehicle running state detection model shown may include an acquisition module 701 and a processing module 702.

[0132] The acquisition module 701 can be used to acquire image data of the vehicle at multiple moments.

[0133] The processing module 702 may be configured to perform convolution calculations on the image data using multiple convolution kernels included in a preset initial vehicle operating state detection model to obtain temporal features of the vehicle at multiple moments.

[0134] The processing module 702 may also be configured to determine first operating state information of the vehicle based on temporal characteristics of the vehicle at multiple moments.

[0135] The processing module 702 can also be used to update the parameters of the initial vehicle operating state detection model according to the first operating state information and the pre-stored second operating state information corresponding to the image data to obtain a trained vehicle operating state detection model.

[0136] The processing module 702 may also be configured to perform feature superposition on the time-series features of the vehicle at multiple moments to obtain superposed features; and determine the first operating state information of the vehicle based on the superposed features.

[0137] In one embodiment, the preset initial vehicle operating status detection model includes multiple convolution kernels in a first direction and / or multiple convolution kernels in a second direction; the first direction and the second direction are perpendicular to each other; the timing features include a first timing feature and a second timing feature.

[0138] Therefore, the processing module 702 can also be used to perform convolution calculations on the image data using multiple convolution kernels in a first direction to obtain first temporal features of the vehicle at multiple moments, and / or to perform convolution calculations on the image data using multiple convolution kernels in a second direction to obtain second temporal features of the vehicle at multiple moments; and determine the first operating status information of the vehicle based on the first temporal features and / or second temporal features of the vehicle at multiple moments.

[0139] The processing module 702 can also be used to determine the loss value between the first operating status information and the second operating status information; update the parameters of the convolution kernel according to the loss value; when the loss value meets the preset conditions, the vehicle operating status detection model corresponding to the loss value that meets the preset conditions is used as the trained vehicle operating status detection model.

[0140] An embodiment of the present invention provides a training device for a vehicle operating state detection model. Image data of a vehicle at multiple moments is acquired; then, convolution calculations are performed on the image data using multiple convolution kernels included in a preset initial vehicle operating state detection model to obtain time-series features of the vehicle at multiple moments. Because features are obtained by directly performing convolution calculations on the image data, the target positioning step is skipped, thereby improving detection efficiency. First operating state information of the vehicle is then determined based on the time-series features at multiple moments; parameters of the initial vehicle operating state detection model are updated based on the first operating state information and pre-stored second operating state information corresponding to the image data, thereby obtaining a trained vehicle operating state detection model for detecting the vehicle's operating state. Furthermore, because the first operating state information of the vehicle is determined by superimposing the obtained time-series features at multiple moments, and the parameters of the trained vehicle operating state detection model are updated based on pre-stored second operating state information corresponding to the image data, thereby improving detection accuracy, addressing the problem of the inability to accurately identify the vehicle's driving direction in current technical solutions and improving recognition efficiency and accuracy.

[0141] In one embodiment, the processing module 702 can also be used to perform grayscale value balancing processing on the image data to obtain first image data, and use multiple convolution kernels included in the initial vehicle detection model to perform convolution calculation on the first image data to obtain temporal features of the vehicle at multiple moments.

[0142] In one embodiment, the processing module 702 can also be used to remove background feature data in the image data based on the grayscale value of each pixel in the image data at each moment, the average grayscale value of the image data, and a preset background feature threshold to obtain second image data, wherein the average grayscale value is determined based on the grayscale value of the pixel whose grayscale value change rate between the image data at two moments is less than the threshold; and use multiple convolution kernels included in the initial vehicle operation status detection model to perform convolution calculation on the second image data to obtain the temporal characteristics of the vehicle at multiple moments.

[0143] It is understandable that Figure 7 Each module in the vehicle running state detection device shown has the function of realizing Figure 1 The functions of each step in the embodiment can achieve the corresponding technical effects, which will not be described in detail here for the sake of brevity.

[0144] The training device of the vehicle operation status detection model provided by the embodiment of the present invention. By acquiring image data of the vehicle at multiple moments; then performing grayscale value balancing processing on the image data and removing background feature data processing, the features of the image data are enhanced, the influence of natural conditions such as weather changes is weakened, and the influence of background features on the later detection of the vehicle's operation status is eliminated. Then, the multiple convolution kernels included in the preset initial vehicle operation status detection model are used to perform convolution calculation on the processed image data to obtain the time series features of the vehicle at multiple moments. Because the features are obtained by directly performing convolution calculation on the image data, the target positioning step is skipped, so that the efficiency of detection is improved. Then, the first operation status information of the vehicle is determined based on the time series features of the vehicle at multiple moments; based on the first operation status information and the pre-stored second operation status information corresponding to the image data, the parameters of the initial vehicle operation status detection model are updated to obtain the trained vehicle operation status detection model for detecting the vehicle's operation status. In addition, because the first operating status information of the vehicle is determined by feature superposition based on the temporal features of the vehicle at multiple moments, and the parameters are updated based on the second operating status information corresponding to the pre-stored image data to obtain a trained vehicle operating status detection model for detecting the operating status of the vehicle, the detection accuracy is also improved, which solves the problem of the current technical solution that cannot accurately identify the vehicle's driving direction, and improves the recognition efficiency and accuracy.

[0145] Corresponding to the embodiment of the method for detecting the vehicle running state, the embodiment of the present invention further provides a device for detecting the vehicle running state, such as Figure 8 shown. Figure 8 Schematic diagram of a vehicle operating status detection device provided by an embodiment of the present invention.

[0146] Figure 8The vehicle running state detection device shown may include an acquisition module 801 and a processing module 802.

[0147] The acquisition module 801 can be used to acquire image data of the vehicle at multiple moments.

[0148] The processing module 802 may be configured to input the image data into a vehicle operation status detection model to obtain the vehicle operation status.

[0149] Among them, the vehicle running state detection model is based on Figure 1 The training method of the vehicle operation status detection model is obtained.

[0150] It is understandable that Figure 8 Each module in the vehicle running state detection device shown has the function of realizing Figure 6 The functions of each step in the embodiment can achieve the corresponding technical effects, which will not be described in detail here for the sake of brevity.

[0151] The vehicle running state detection device provided by the embodiment of the present invention obtains the image data of the vehicle at multiple moments; then inputs the image data into the vehicle running state detection device according to the vehicle running state detection device. Figure 1 In the vehicle operation status detection model obtained by the training method in

[15] , convolution calculations are performed to obtain time-series features of the vehicle at multiple moments. Because the features are directly obtained by convolution calculations on the image data, the target positioning step is skipped, thereby improving detection efficiency. The vehicle's operation status information is then determined by superimposing the time-series features from multiple moments. This solves the problem of accurate vehicle direction identification in current technical solutions and improves recognition efficiency and accuracy.

[0152] Figure 9 This is a structural diagram of the hardware architecture of a computing device provided by an embodiment of the present invention. Figure 9 As shown, computing device 900 includes an input device 901, an input interface 902, a central processing unit 903, a memory 904, an output interface 905, and an output device 906. The input interface 902, the central processing unit 903, the memory 904, and the output interface 905 are interconnected via a bus 910. The input device 901 and the output device 906 are connected to the bus 910 via the input interface 902 and the output interface 905, respectively, and are further connected to other components of computing device 900.

[0153] Specifically, the input device 901 receives input information from the outside and transmits the input information to the central processing unit 903 through the input interface 902; the central processing unit 903 processes the input information based on the computer-executable instructions stored in the memory 904 to generate output information, temporarily or permanently stores the output information in the memory 904, and then transmits the output information to the output device 906 through the output interface 905; the output device 906 outputs the output information to the outside of the computing device 900 for user use.

[0154] That is to say, Figure 9 The computing device shown can also be implemented as a training device for a vehicle operating status detection model, or a vehicle operating status detection device. The training device or the detection device may include: a memory storing computer-executable instructions, and a processor. When executing the computer-executable instructions, the processor can implement the training method for the vehicle operating status detection model provided in an embodiment of the present invention, or the vehicle operating status detection method.

[0155] An embodiment of the present invention also provides a computer-readable storage medium having computer program instructions stored thereon; when the computer program instructions are executed by a processor, the training method of the vehicle operation status detection model provided in an embodiment of the present invention, or the vehicle operation status detection method, is implemented.

[0156] It should be understood that the present invention is not limited to the specific configurations and processes described above and illustrated in the figures. For the sake of brevity, a detailed description of known methods is omitted. In the above embodiments, several specific steps are described and illustrated as examples. However, the method of the present invention is not limited to the specific steps described and illustrated. Those skilled in the art may make various changes, modifications, and additions, or change the order of the steps after understanding the spirit of the present invention.

[0157] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of the present invention are programs or code segments used to perform the required tasks. The programs or code segments can be stored in a machine-readable medium, or transmitted on a transmission medium or communication link via a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, read-only memories (ROMs), flash memories, erasable read-only memories (EROMs), floppy disks, compact disc read-only memories (CD-ROMs), optical discs, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segments can be downloaded via computer networks such as the Internet and intranets.

[0158] It should also be noted that the exemplary embodiments described herein describe methods or systems based on a series of steps or devices. However, the present invention is not limited to the order of the steps described above. In other words, the steps may be performed in the order described in the embodiments, or in a different order, or several steps may be performed simultaneously.

[0159] Aspects of the present disclosure have been described above with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor or a field programmable logic circuit. It is also understood that each box in the block diagram and / or flowchart and the combination of the boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0160] The above description is only a specific embodiment of the present invention. Those skilled in the art will clearly understand that for the convenience and brevity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present invention is not limited to this. Any person skilled in the art can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention.

Claims

1. A training method for a vehicle driving direction detection model, characterized in that: The training method comprises: Acquire image data of the vehicle at multiple times; Performing convolution calculations on the image data using multiple convolution kernels included in a preset initial vehicle driving direction detection model to obtain temporal features of the vehicle at multiple moments; Determining first operating state information of the vehicle according to the time series characteristics of the vehicle at multiple moments; updating the parameters of the initial vehicle driving direction detection model according to the first operating state information and pre-stored second operating state information corresponding to the image data to obtain a trained vehicle driving direction detection model; The preset initial vehicle driving direction detection model includes multiple convolution kernels in a first direction and / or multiple convolution kernels in a second direction; the first direction and the second direction are perpendicular to each other; the temporal features include a first temporal feature and a second temporal feature; and the training method further includes: Performing convolution calculation on the image data using multiple convolution kernels in the first direction to obtain first temporal features of the vehicle at multiple moments, and / or performing convolution calculation on the image data using multiple convolution kernels in the second direction to obtain second temporal features of the vehicle at multiple moments; The first operating state information of the vehicle is determined according to the first time-series characteristics and / or the second time-series characteristics of the vehicle at multiple moments.

2. The training method according to claim 1, characterized in that The training method further comprises: performing grayscale value balancing processing on the image data to obtain first image data; The first image data is convolved using multiple convolution kernels included in the initial vehicle driving direction detection model to obtain temporal features of the vehicle at multiple moments.

3. The training method according to claim 1 or 2, characterized in that: The method further comprises: removing background feature data from the image data based on the grayscale value of each pixel in the image data at each moment, the average grayscale value of the image data, and a preset background feature threshold, to obtain second image data, wherein the average grayscale value is determined based on the grayscale values ​​of pixels whose grayscale value change rate between the image data at two moments is less than the threshold; The second image data is convolutionally calculated using multiple convolution kernels included in the initial vehicle driving direction detection model to obtain temporal features of the vehicle at multiple moments.

4. The training method according to claim 1, characterized in that The determining the first operating state information of the vehicle according to the time-series characteristics of the vehicle at multiple moments includes: Superimposing the time-series features of the vehicle at multiple moments to obtain superimposed features; First operating state information of the vehicle is determined according to the superimposed features.

5. The training method according to claim 1, characterized in that The updating of the parameters of the initial vehicle driving direction detection model according to the first operating state information and pre-stored second operating state information corresponding to the image data to obtain a trained vehicle detection model includes: determining a loss value between the first operating status information and the second operating status information; Updating the parameters of the convolution kernel according to the loss value; When the loss value meets the preset conditions, the vehicle driving direction detection model corresponding to the loss value meeting the preset conditions is used as the trained vehicle driving direction detection model.

6. A method for detecting a vehicle's operating status, characterized in that: The detection method comprises: Acquire image data of the vehicle at multiple times; The image data is input into a vehicle driving direction detection model to obtain the vehicle's running status, wherein the vehicle driving direction detection model is obtained based on the training method of the vehicle driving direction detection model according to any one of claims 1 to 5.

7. A training device for a vehicle driving direction detection model, characterized in that: The training device comprises: An acquisition module, used to acquire image data of the vehicle at multiple times; a processing module, configured to perform convolution calculations on the image data using a plurality of convolution kernels included in a preset initial vehicle driving direction detection model to obtain temporal features of the vehicle at multiple moments; The processing module is further configured to determine first operating state information of the vehicle based on the time series characteristics of the vehicle at multiple moments; The processing module is further configured to update the parameters of the initial vehicle driving direction detection model according to the first operating state information and pre-stored second operating state information corresponding to the image data, to obtain a trained vehicle driving direction detection model; The preset initial vehicle running state detection model includes multiple convolution kernels in a first direction and / or multiple convolution kernels in a second direction; the first direction and the second direction are perpendicular to each other; the time series feature includes a first time series feature and a second time series feature; the processing module is further used to: Performing convolution calculation on the image data using multiple convolution kernels in the first direction to obtain first temporal features of the vehicle at multiple moments, and / or performing convolution calculation on the image data using multiple convolution kernels in the second direction to obtain second temporal features of the vehicle at multiple moments; The first operating state information of the vehicle is determined according to the first time-series characteristics and / or the second time-series characteristics of the vehicle at multiple moments.

8. A vehicle running status detection device, characterized in that: The detection device comprises: An acquisition module, used to acquire image data of the vehicle at multiple times; A processing module is used to input the image data into a vehicle driving direction detection model to obtain the vehicle's operating status, wherein the vehicle driving direction detection model is obtained based on the training method of the vehicle driving direction detection model described in any one of claims 1 to 5.

9. An electronic device, characterized in that: The device includes: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, it implements the training method of the vehicle driving direction detection model as described in any one of claims 1 to 5, or implements the vehicle running state detection method as described in claim 6.

10. A computer storage medium, characterized in that The computer storage medium stores computer program instructions, which, when executed by the processor, implement the training method for the vehicle driving direction detection model as described in any one of claims 1 to 5, or implement the vehicle operating status detection method as described in claim 6.

Images