Driver standard behavior detection and early warning method and system based on machine vision

Abstract

The invention relates to the technical field of video monitoring, in particular to a driver standard behavior detection and early warning method and system based on machine vision, and the system comprises an image collection module, an image processing module, an image data clustering module, a standard feature image data collection module, an image matching module and a driver driving behavior judgment module. According to the method, driver eye feature image data and driver body feature image data are clustered through an agglomerated hierarchical clustering algorithm, any image data in each class in a clustering result is selected to be matched with standard image data, the calculated amount of the matching process is reduced through the clustering algorithm, the image matching speed is increased, and the matching efficiency is improved. Meanwhile, the accuracy of a matching result is ensured; and judging whether the driving behavior of the driver in the current driving state is standard or not according to the calculated eye standard weight factor and body standard weight factor, thereby realizing scientific detection of the standard behavior of the driver and completing reasonable early warning of the abnormal behavior at the same time.

Description

Technical Field

[0001] The present invention relates to the field of video surveillance technology, and in particular to a driver's standardized behavior detection and early warning method and system based on machine vision. Background Art

[0002] With the rapid development of science and technology and the national economy, people's demand for transportation is getting higher and higher. Therefore, with the rapid popularization of vehicles, the incidence of traffic accidents has also increased, and people's attention to traffic safety has also increased.

[0003] During the driving process of a vehicle, many types of traffic accidents may occur, and there are always many traffic accidents caused by improper operation of the driver; however, the existing driver behavior detection technology cannot accurately detect the driver's driving behavior, and the speed of the detection process is relatively low.

[0004] The Chinese invention patent with announcement number CN113033239B introduces a behavior detection method and device, which obtains and analyzes the monitoring video of the monitoring scene when the vehicle is in a stopped state, obtains the actions performed in sequence by the driver in the cab and the actions performed in sequence by the driver outside the cab, and the corresponding driver position in the monitoring scene; according to the order of the actions performed in sequence by the driver in the cab and the order of the actions performed in sequence outside the cab, according to the obtained actions and positions, detects whether the driver's behavior is standard behavior; but only considers whether the behavior of the vehicle in the stopped state is standard, and does not detect the driver's driving behavior during the vehicle driving process. Summary of the invention

[0005] In order to solve the deficiencies in the background technology, the present invention proposes a method and system for detecting and warning driver's standardized behavior based on machine vision, which can realize the scientific detection of driver's standardized behavior and complete the reasonable warning of abnormal behavior.

[0006] A driver's standardized behavior detection and early warning method based on machine vision includes the following steps: S1, collecting driver's eye images and driver's body images in driving state; S2, processing the driver's eye image and the driver's body image respectively to obtain the driver's eye feature image data and the driver's body feature image data; S3, clustering the driver's eye feature image data and the driver's body feature image data respectively to obtain an eye feature image classification data set and a body feature image classification data set; S4, establishing an eye standard feature image dataset and a body standard feature image dataset; S5, performing image matching on the eye feature image classification dataset and the eye standard feature image dataset, and on the body feature image classification dataset and the body standard feature image dataset, and calculating the driver's eye standard weight factor and body standard weight factor according to the matching results; S6. Determine whether the driver's driving behavior is standard in the current driving state based on the eye standard weight factor and the body standard weight factor.

[0007] By performing grayscale processing and filtering noise reduction processing on the collected driver's eye images and driver's body images respectively, the influence of noise in the image is reduced; the driver's eye feature image data and the driver's body feature image data are clustered respectively through an agglomerative hierarchical clustering algorithm, and any image data in each category in the clustering results is selected to perform image matching with the standard image data, and the clustering algorithm is used to reduce the calculation amount of the matching process, improve the speed of image matching, and ensure the accuracy of the matching results; the driver's eye standard weight factor and body standard weight factor are calculated according to the matching results to ensure the accuracy of the calculation results; based on the eye standard weight factor and the body standard weight factor, it is determined whether the driver's driving behavior in the current driving state is standard, so as to realize scientific detection of the driver's standard behavior and complete reasonable early warning of abnormal behavior.

[0008] Preferably, the specific steps of collecting the driver's eye image and the driver's body image in the driving state are as follows: S11, setting the image acquisition cycle; S12, regularly collecting the driver's eye image and the driver's body image in the driving state by using an image capturing device installed in the vehicle to obtain a driver's eye image set and a driver's body image set; The time interval between two adjacent image acquisitions is an image acquisition cycle, and three groups of eye images and body images of the driver in the driving state are continuously acquired each time.

[0009] Preferably, the specific steps of processing the driver's eye image and the driver's body image respectively to obtain the driver's eye feature image data and the driver's body feature image data are as follows: S21, performing grayscale processing on the driver eye images in the driver eye image set and the driver body images in the driver body image set respectively, to obtain driver eye image data and driver body image data after grayscale processing; the grayscale processing process is as follows: S211, extracting pixel values ​​of red, green and blue corresponding color components of each pixel in the image; S212, setting the weight factors of the red, green and blue color components to be ω respectively R ,ω G And ω B ; S213, using a weighted average method to calculate the pixel mean of the red, green and blue color components of each pixel, and using the calculated pixel mean as the target grayscale value of the corresponding pixel; the calculation formula is as follows: ; Among them, Gray1 represents the pixel mean corresponding to each pixel point, R, G and B represent the pixel values ​​of the red, green and blue color components of each pixel point respectively; S22, using the median filtering method to perform noise reduction on the grayscale processed driver eye image data and driver body image data, and obtain the driver eye feature image data set A={a1, a2, …, a i ,…,a k} and the driver's body feature image data B={b1,b2,…,b i ,…,b k}, where a i represents the driver's eye feature image data obtained after processing the i-th driver's eye image, b i represents the driver's body feature image data obtained after the i-th driver's body image is processed, and k represents the total number of driver's eye feature image data; the denoising process is as follows: S221, setting an m×m sliding window, and aligning the center position of the sliding window with each pixel point in sequence; S222, whenever the center position of the sliding window is aligned with a pixel, the target grayscale values ​​of all the pixels in the sliding window are obtained, and the target grayscale values ​​are arranged from large to small according to the value size to obtain a grayscale value sequence S={s1,s2,…,s i ,…,s n}, where s i represents the i-th gray value in the gray value sequence, and n represents the total number of gray values ​​in the gray value sequence; S223, calculating the median of the gray value sequence, and using the calculated median as the target gray value of the pixel corresponding to the center position of the current sliding window; the calculation formula is as follows: ; Among them, Gray2 represents the median of the gray value sequence, Represents the gray value sequence Gray value, Represents the gray value sequence Gray value, Represents the gray value sequence Gray value.

[0010] The influence of noise in the image is reduced by performing grayscale processing and filtering noise reduction processing on the collected driver's eye image and driver's body image respectively.

[0011] Preferably, the specific steps of clustering the driver's eye feature image data and the driver's body feature image data to obtain the eye feature image classification data set and the body feature image classification data set are as follows: S31, respectively, for the driver eye feature image dataset A={a1, a2, …, a i ,…,a k} in the driver's eye feature image data and the driver's body feature image data B={b1,b2,…,b i ,…,b k} to cluster the driver body feature image data and obtain the eye feature image classification dataset and body feature image classification dataset ,in, An image dataset representing the eye features of the i-th driver. A dataset of images representing the physical features of the i-th driver, represents the total number of driver eye feature categories, Represents the total number of driver physical feature categories; the clustering operation is as follows: S311, each image data in the image data set is regarded as a cluster, and the Euclidean distance between clusters is calculated to obtain an initial distance matrix; the Euclidean distance calculation formula is as follows: ; Where d(x,y) represents the Euclidean distance between x and y, x and y represent the n-dimensional feature vectors of any two cluster centers, and x i Represents the value of vector x in the i-th dimension, y i Represents the value of vector y in the i-th dimension; S312, selecting a minimum value from the initial distance matrix, merging two clusters corresponding to the minimum value into a new cluster, and recalculating the Euclidean distance between the new cluster and other clusters; S313, repeating the operations from S311 to S312 until the shortest Euclidean distance between clusters is greater than the set Euclidean distance threshold, stopping the clustering operation, and obtaining clustered image data.

[0012] The driver's eye feature image data and the driver's body feature image data are clustered separately through the agglomerative hierarchical clustering algorithm to ensure the accuracy of the clustering results.

[0013] Preferably, the specific operations of establishing the eye standard feature image dataset and the body standard feature image dataset are as follows: S41. Collecting the driver's eye standard feature image data in the driving state and establishing the eye standard feature image data set ,in, represents the collected eye standard feature image data of the driver in the i-th driving state, l represents the total number of collected eye standard feature image data of the driver in the driving state; S42. Collecting the driver's body standard feature image data in the driving state and establishing a body standard feature image data set ,in, represents the collected body normative feature image data of the driver in the i-th driving state, and p represents the total number of the collected body normative feature image data of the driver in the driving state.

[0014] Preferably, the specific steps of performing image matching on the eye feature image classification data set and the eye standard feature image data set, and on the body feature image classification data set and the body standard feature image data set, and calculating the driver's eye standard weight factor and body standard weight factor according to the matching results are as follows: S51, randomly selecting one image data from the image data sets of eye features of various types of drivers in the eye feature image classification data set, to obtain all the eye feature image data to be matched; When any eye feature image data to be matched successfully matches the eye standard feature image data in the eye standard feature image data set, it means that all image data in the image data set of the eye features of such driver in the eye feature image classification data set successfully matches the eye standard feature image data in the eye standard feature image data set; Selecting one image data at random from the image data sets of various types of drivers' body features in the body feature image classification data set, respectively, to obtain all body feature image data to be matched; When any body feature image data to be matched successfully matches the body standard feature image data in the body standard feature image data set, it means that all image data in the image data set of the body features of such driver in the body feature image classification data set successfully matches the body standard feature image data in the body standard feature image data set; S52, sequentially matching each eye feature image data to be matched with the eye standard feature image data in the eye standard feature image data set, and selecting the eye feature image data with successful matching, and calculating the eye standard weight factor , where c irepresents the total number of driver eye feature image data of the corresponding class of the i-th successfully matched eye feature image data in the eye feature image classification data set, o represents the total number of successfully matched eye feature image data to be matched, and k represents the total number of driver eye feature image data; the image matching process is as follows: S521. Construct a crow population, set the population size to N, the flight step length to fl, the detection probability to AP, the current number of iterations to t, and the maximum number of iterations to t max and the search space dimension of the eye normative feature image data is P; In the eye standard feature image data search space, N crow individuals are randomly generated, each crow individual corresponds to an eye standard feature image data, and the initial position set of the crow population is initialized to X={X1,X2,…,X i ,…,X N}, where X i represents the initial position of the i-th crow in the crow population; S522, taking the initial position of each crow individual in the crow population as the initial food hiding position of each crow individual, and calculating the fitness function value of the food hiding position of each crow individual in the crow population; the fitness function formula is as follows: ; Among them, Fit(x i ) represents the fitness function value of the food hiding location of the i-th crow in the crow population, x i represents the degree of deviation between the eye standard feature image data corresponding to the food hiding position of the i-th crow and the eye feature image data to be matched, Indicates the correction value; S523, in order to steal food from other crows, individual crows will follow other crows in the eye canonical feature image data search space; When the random number rand is generated between [0,1] i When it is greater than or equal to the detection probability AP, it is determined that the crow individual has not been discovered in the process of following other crow individuals in the eye standard feature image data search space. At this time, the crow individual will update its position according to the food hiding position of other crow individuals; the position update formula is as follows: ; in, and They represent the positions of the i-th crow in the crow population during the t+1th and tth iterations, respectively. represents the food hiding location of the jth crow in the crow population; When the random number rand is generated between [0,1]i When it is greater than or equal to the detection probability AP, it is determined that the crow individual is discovered in the process of following other crow individuals in the eye standard feature image data search space. At this time, the crow individual will be brought to a random position in the eye standard feature image data search space by other crow individuals; S524, calculating the fitness function value of the crow individual after the position is updated in the eye standard feature image data search space, when the fitness function value of the crow individual after the position is updated is greater than the fitness function value before the position is updated, the position is used as the food hiding position of the crow individual; otherwise, the original food hiding position of the crow individual is retained; S525: Determine whether the current number of iterations t is less than the maximum number of iterations t max , if the current number of iterations is t and less than the maximum number of iterations is t max , then the current iteration number is t plus 1 and returns to S523; otherwise, the fitness function value of each food hiding position of each crow in the crow population is calculated, and the food hiding position of the crow with the highest fitness function value is taken as the global optimal solution, and the fitness function threshold is set. When the fitness function value of the global optimal solution is greater than the fitness function threshold, it is determined that the eye standard feature image data corresponding to the global optimal solution matches the eye feature image data to be matched successfully; otherwise, it is determined that the eye feature image data to be matched and all the eye standard feature image data in the eye standard feature image data set fail to match; S53, using the operations in S521 to S525, image matching is performed on each of the body feature image data to be matched with the body standard feature image data in the body standard feature image data set, and the body feature image data to be matched that successfully matches is screened out, and the body standard weight factor is calculated. , where d i It represents the total number of driver body feature image data of the corresponding class of the i-th successfully matched body feature image data in the body feature image classification data set, u represents the total number of successfully matched body feature image data to be matched, and k represents the total number of driver body feature image data.

[0015] Any image data in each category in the clustering results is selected to perform image matching with the standard image data, and the clustering algorithm is used to reduce the calculation amount of the matching process, improve the speed of image matching, and ensure the accuracy of the matching results. Compared with the traditional image similarity matching method, the present invention realizes the matching process of driver feature image data and standard feature image data through the crow algorithm, which improves the speed and efficiency of the matching process and ensures the accuracy of the matching results. At the same time, by simulating the behavior of crows stealing food from other crows in nature, a mathematical model is constructed to effectively avoid the matching results from falling into the local optimal solution.

[0016] Preferably, the specific steps of determining whether the driver's driving behavior is standard in the current driving state according to the eye standard weight factor and the body standard weight factor are as follows: S61, setting the eye norm weight threshold to α and the body norm weight threshold to β; S62, respectively comparing the eye standard weight factor λ with the eye standard weight threshold α, the body standard weight factor η with the body standard weight threshold β, and determining whether the driver's driving behavior is standard in the current driving state according to the comparison results; S621, if λ≥α and η≥β, it is determined that the driver's driving behavior in the current driving state is a standard behavior, and no operation is performed; If λ<α and η≥β or λ≥α and η<β, the driver's driving behavior in the current driving state is determined to be abnormal, and the determination result is fed back to the main control unit, which issues an abnormal behavior signal warning; If λ<α and η<β, the driver's driving behavior in the current driving state is determined to be a dangerous behavior, and the determination result is fed back to the main control unit, which issues a dangerous behavior signal warning and a voice prompt.

[0017] Whether the driver's driving behavior is standard in the current driving state is determined based on the eye standard weight factor and the body standard weight factor, thereby achieving scientific detection of the driver's standard behavior and completing reasonable early warning of abnormal behavior.

[0018] The present invention also discloses a system for detecting and warning driver's standard behavior based on machine vision, comprising an image acquisition module, an image processing module, an image data clustering module, a standard feature image data collection module, an image matching module and a driver's driving behavior determination module; The image acquisition module regularly acquires the driver's eye image and the driver's body image in the driving state through the image capture device installed in the vehicle; The image processing module performs grayscale processing and filtering noise reduction processing on the driver's eye image and the driver's body image respectively to obtain the driver's eye feature image data and the driver's body feature image data; The image data clustering module clusters the driver's eye feature image data and the driver's body feature image data respectively through an agglomerative hierarchical clustering algorithm to obtain an eye feature image classification data set and a body feature image classification data set; The standard feature image data collection module collects the standard feature image data of the eyes and the standard feature image data of the body of the driver in the driving state to establish the standard feature image data set of the eyes and the standard feature image data set of the body; The image matching module performs image matching on the eye feature image classification data set and the eye standard feature image data set, and on the body feature image classification data set and the body standard feature image data set, and calculates the driver's eye standard weight factor and body standard weight factor according to the matching results; The driver driving behavior determination module compares the eye standard weight factor with the set eye standard weight threshold, the body standard weight factor with the set body standard weight threshold, and determines whether the driver's driving behavior is standard in the current driving state according to the numerical comparison results.

[0019] 1. The present invention is provided with an image acquisition module, an image processing module, an image data clustering module, a standard feature image data collection module, an image matching module and a driver driving behavior determination module; the influence of noise in the image is reduced by performing grayscale processing and filtering noise reduction processing on the collected driver's eye image and driver's body image respectively; the driver's eye feature image data and the driver's body feature image data are clustered respectively by an agglomerative hierarchical clustering algorithm, and any image data in each category in the clustering result is selected to perform image matching with the standard image data, and the clustering algorithm is used to reduce the amount of calculation in the matching process, improve the speed of image matching, and ensure the accuracy of the matching result; the driver's eye standard weight factor and body standard weight factor are calculated according to the matching result to ensure the accuracy of the calculation result; whether the driver's driving behavior in the current driving state is standard according to the eye standard weight factor and the body standard weight factor, so as to realize the scientific detection of the driver's standard behavior and complete the reasonable early warning of abnormal behavior; 2. Compared with the traditional image similarity matching method, the present invention realizes the matching process of driver feature image data and standard feature image data through the crow algorithm, which improves the speed and efficiency of the matching process and ensures the accuracy of the matching results. At the same time, by simulating the behavior of crows stealing food from other crows in nature, a mathematical model is constructed to effectively avoid the matching results falling into the local optimal solution. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the invention, the drawings required for describing the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the invention. For ordinary technicians in this field, they can also obtain drawings based on these drawings without paying creative work.

[0021] Figure 1 A flowchart of a driver's standardized behavior detection and early warning method based on machine vision provided by the present invention; Figure 2A module schematic diagram of a system for a driver's standardized behavior detection and early warning method based on machine vision provided by the present invention. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] In the description of the present invention, it is necessary to understand that the terms "opening", "upper", "lower", "top", "middle", "inside" and the like indicating orientation or positional relationship are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the components or elements referred to must have a specific orientation, be constructed and operate in a specific orientation, and therefore cannot be understood as limiting the invention.

[0024] The first embodiment of the driver's standard behavior detection and early warning method based on machine vision is as follows: See also Figure 1 , a driver's standardized behavior detection and early warning method based on machine vision, comprising the following steps: S1, collecting driver's eye images and driver's body images in driving state; S11, setting the image acquisition cycle; S12, regularly collecting the driver's eye image and the driver's body image in the driving state by using an image capturing device installed in the vehicle to obtain a driver's eye image set and a driver's body image set; The time interval between two adjacent image acquisitions is an image acquisition cycle, and three groups of eye images and body images of the driver in the driving state are continuously acquired each time.

[0025] S2, processing the driver's eye image and the driver's body image respectively to obtain the driver's eye feature image data and the driver's body feature image data; S21, performing grayscale processing on the driver eye images in the driver eye image set and the driver body images in the driver body image set respectively, to obtain driver eye image data and driver body image data after grayscale processing; the grayscale processing process is as follows: S211, extracting pixel values ​​of red, green and blue corresponding color components of each pixel in the image; S212, setting the weight factors of the red, green and blue color components to be ω respectively R ,ω G And ω B ; S213, using a weighted average method to calculate the pixel mean of the red, green and blue color components of each pixel, and using the calculated pixel mean as the target grayscale value of the corresponding pixel; the calculation formula is as follows: ; Among them, Gray1 represents the pixel mean corresponding to each pixel point, R, G and B represent the pixel values ​​of the red, green and blue color components of each pixel point respectively; S22, using the median filtering method to perform noise reduction on the grayscale processed driver eye image data and driver body image data, and obtain the driver eye feature image data set A={a1, a2, …, a i ,…,a k} and the driver's body feature image data B={b1,b2,…,b i ,…,b k}, where a i represents the driver's eye feature image data obtained after processing the i-th driver's eye image, b i represents the driver's body feature image data obtained after the i-th driver's body image is processed, and k represents the total number of driver's eye feature image data; the denoising process is as follows: S221, setting an m×m sliding window, and aligning the center position of the sliding window with each pixel point in sequence; S222, whenever the center position of the sliding window is aligned with a pixel, the target grayscale values ​​of all the pixels in the sliding window are obtained, and the target grayscale values ​​are arranged from large to small according to the value size to obtain a grayscale value sequence S={s1,s2,…,s i ,…,s n}, where s i represents the i-th gray value in the gray value sequence, and n represents the total number of gray values ​​in the gray value sequence; S223, calculating the median of the gray value sequence, and using the calculated median as the target gray value of the pixel corresponding to the center position of the current sliding window; the calculation formula is as follows: ; Among them, Gray2 represents the median of the gray value sequence, Represents the gray value sequence Gray value, Represents the gray value sequence Gray value, Represents the gray value sequence Gray value.

[0026] S3, clustering the driver's eye feature image data and the driver's body feature image data respectively to obtain an eye feature image classification data set and a body feature image classification data set; S31, respectively, for the driver eye feature image dataset A={a1, a2, …, a i ,…,a k} in the driver's eye feature image data and the driver's body feature image data B={b1,b2,…,b i ,…,b k} to cluster the driver body feature image data and obtain the eye feature image classification dataset and body feature image classification dataset ,in, An image dataset representing the eye features of the i-th driver. A dataset of images representing the physical features of the i-th driver, represents the total number of driver eye feature categories, Represents the total number of driver physical feature categories; the clustering operation is as follows: S311, each image data in the image data set is regarded as a cluster, and the Euclidean distance between clusters is calculated to obtain an initial distance matrix; the Euclidean distance calculation formula is as follows: ; Where d(x,y) represents the Euclidean distance between x and y, x and y represent the n-dimensional feature vectors of any two cluster centers, and x i Represents the value of vector x in the i-th dimension, y i Represents the value of vector y in the i-th dimension; S312, selecting a minimum value from the initial distance matrix, merging two clusters corresponding to the minimum value into a new cluster, and recalculating the Euclidean distance between the new cluster and other clusters; S313, repeating the operations from S311 to S312 until the shortest Euclidean distance between clusters is greater than the set Euclidean distance threshold, stopping the clustering operation, and obtaining clustered image data.

[0027] S4, establishing an eye standard feature image dataset and a body standard feature image dataset; S41. Collecting the driver's eye standard feature image data in the driving state and establishing the eye standard feature image data set ,in, represents the collected eye standard feature image data of the driver in the i-th driving state, l represents the total number of collected eye standard feature image data of the driver in the driving state; S42. Collecting the driver's body standard feature image data in the driving state and establishing a body standard feature image data set ,in, represents the collected body normative feature image data of the driver in the i-th driving state, and p represents the total number of the collected body normative feature image data of the driver in the driving state.

[0028] S5, performing image matching on the eye feature image classification dataset and the eye standard feature image dataset, and on the body feature image classification dataset and the body standard feature image dataset, and calculating the driver's eye standard weight factor and body standard weight factor according to the matching results; S51, randomly selecting one image data from the image data sets of eye features of various types of drivers in the eye feature image classification data set, to obtain all the eye feature image data to be matched; When any eye feature image data to be matched successfully matches the eye standard feature image data in the eye standard feature image data set, it means that all image data in the image data set of the eye features of such driver in the eye feature image classification data set successfully matches the eye standard feature image data in the eye standard feature image data set; Selecting one image data at random from the image data sets of various types of drivers' body features in the body feature image classification data set, respectively, to obtain all body feature image data to be matched; When any body feature image data to be matched successfully matches the body standard feature image data in the body standard feature image data set, it means that all image data in the image data set of the body features of such driver in the body feature image classification data set successfully matches the body standard feature image data in the body standard feature image data set; S52, sequentially matching each eye feature image data to be matched with the eye standard feature image data in the eye standard feature image data set, and selecting the eye feature image data with successful matching, and calculating the eye standard weight factor , where c i represents the total number of driver eye feature image data of the corresponding class of the i-th successfully matched eye feature image data in the eye feature image classification data set, o represents the total number of successfully matched eye feature image data to be matched, and k represents the total number of driver eye feature image data; the image matching process is as follows: S521. Construct a crow population, set the population size to N, the flight step length to fl, the detection probability to AP, the current number of iterations to t, and the maximum number of iterations to t max and the search space dimension of the eye normative feature image data is P; In the eye standard feature image data search space, N crow individuals are randomly generated, each crow individual corresponds to an eye standard feature image data, and the initial position set of the crow population is initialized as X={X1,X2,…,X i ,…,X N}, where X i represents the initial position of the i-th crow in the crow population; S522, taking the initial position of each crow individual in the crow population as the initial food hiding position of each crow individual, and calculating the fitness function value of the food hiding position of each crow individual in the crow population; ; Among them, Fit(x i ) represents the fitness function value of the food hiding location of the i-th crow in the crow population, x i represents the degree of deviation between the eye standard feature image data corresponding to the food hiding position of the i-th crow and the eye feature image data to be matched, Indicates the correction value; S523, in order to steal food from other crows, individual crows will follow other crows in the eye canonical feature image data search space; When the random number rand is generated between [0,1] i When it is greater than or equal to the detection probability AP, it is determined that the crow individual has not been discovered in the process of following other crow individuals in the eye standard feature image data search space. At this time, the crow individual will update its position according to the food hiding position of other crow individuals; the position update formula is as follows: ; in, and They represent the positions of the i-th crow in the crow population during the t+1th and tth iterations, respectively. represents the food hiding location of the jth crow in the crow population; When the random number rand is generated between [0,1] i When it is greater than or equal to the detection probability AP, it is determined that the crow individual is discovered in the process of following other crow individuals in the eye standard feature image data search space. At this time, the crow individual will be brought to a random position in the eye standard feature image data search space by other crow individuals; S524, calculating the fitness function value of the crow individual after the position is updated in the eye standard feature image data search space, when the fitness function value of the crow individual after the position is updated is greater than the fitness function value before the position is updated, the position is used as the food hiding position of the crow individual; otherwise, the original food hiding position of the crow individual is retained; S525: Determine whether the current number of iterations t is less than the maximum number of iterations t max , if the current number of iterations is t and less than the maximum number of iterations is t max , then the current iteration number is t plus 1 and returns to S523; otherwise, the fitness function value of each food hiding position of each crow in the crow population is calculated, and the food hiding position of the crow with the highest fitness function value is taken as the global optimal solution, and the fitness function threshold is set. When the fitness function value of the global optimal solution is greater than the fitness function threshold, it is determined that the eye standard feature image data corresponding to the global optimal solution matches the eye feature image data to be matched successfully; otherwise, it is determined that the eye feature image data to be matched and all the eye standard feature image data in the eye standard feature image data set fail to match; S53, using the operations in S521 to S525, image matching is performed on each of the body feature image data to be matched with the body standard feature image data in the body standard feature image data set, and the body feature image data to be matched that successfully matches is screened out, and the body standard weight factor is calculated. , where d i It represents the total number of driver body feature image data of the corresponding class of the i-th successfully matched body feature image data in the body feature image classification data set, u represents the total number of successfully matched body feature image data to be matched, and k represents the total number of driver body feature image data.

[0029] S6, determining whether the driver's driving behavior is standard in the current driving state according to the eye standard weight factor and the body standard weight factor; S61, setting the eye norm weight threshold to α and the body norm weight threshold to β; S62, respectively comparing the eye standard weight factor λ with the eye standard weight threshold α, the body standard weight factor η with the body standard weight threshold β, and determining whether the driver's driving behavior is standard in the current driving state according to the comparison results; S621, if λ≥α and η≥β, it is determined that the driver's driving behavior in the current driving state is a standard behavior, and no operation is performed; If λ<α and η≥β or λ≥α and η<β, the driver's driving behavior in the current driving state is determined to be abnormal, and the determination result is fed back to the main control unit, which issues an abnormal behavior signal warning; If λ<α and η<β, the driver's driving behavior in the current driving state is determined to be a dangerous behavior, and the determination result is fed back to the main control unit, which issues a dangerous behavior signal warning and a voice prompt.

[0030] The second embodiment of the system of the driver's standard behavior detection and early warning method based on machine vision is as follows: See also Figure 2 , a system for detecting and warning driver's standard behavior based on machine vision, including an image acquisition module, an image processing module, an image data clustering module, a standard feature image data collection module, an image matching module and a driver's driving behavior determination module; The image acquisition module regularly acquires the driver's eye image and the driver's body image in the driving state through the image capture device installed in the vehicle; The image processing module performs grayscale processing and filtering noise reduction processing on the driver's eye image and the driver's body image respectively to obtain the driver's eye feature image data and the driver's body feature image data; The image data clustering module clusters the driver's eye feature image data and the driver's body feature image data respectively through an agglomerative hierarchical clustering algorithm to obtain an eye feature image classification data set and a body feature image classification data set; The standard feature image data collection module collects the standard feature image data of the eyes and the standard feature image data of the body of the driver in the driving state to establish the standard feature image data set of the eyes and the standard feature image data set of the body; The image matching module performs image matching on the eye feature image classification data set and the eye standard feature image data set, and on the body feature image classification data set and the body standard feature image data set, and calculates the driver's eye standard weight factor and body standard weight factor according to the matching results; The driver driving behavior determination module compares the eye standard weight factor with the set eye standard weight threshold, the body standard weight factor with the set body standard weight threshold, and determines whether the driver's driving behavior is standard in the current driving state according to the numerical comparison results.

[0031] In the description of this specification, the description with reference to the terms "one embodiment", "example", "specific example", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0032] The preferred embodiments of the invention disclosed above are only used to help explain the invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the invention, so that those skilled in the art can understand and use the invention well.

Claims

1. A driver's standard behavior detection and early warning method based on machine vision, characterized in that: The steps include: S1, collecting driver's eye images and driver's body images in driving state; S2, processing the driver's eye image and the driver's body image respectively to obtain the driver's eye feature image data and the driver's body feature image data; S3, clustering the driver's eye feature image data and the driver's body feature image data respectively to obtain an eye feature image classification data set and a body feature image classification data set; S4, establishing an eye standard feature image dataset and a body standard feature image dataset; S5, performing image matching on the eye feature image classification dataset and the eye standard feature image dataset, and on the body feature image classification dataset and the body standard feature image dataset, and calculating the driver's eye standard weight factor and body standard weight factor according to the matching results; S6. Determine whether the driver's driving behavior is standard in the current driving state based on the eye standard weight factor and the body standard weight factor.

2. The method for detecting and warning driver's standardized behavior based on machine vision according to claim 1, characterized in that: The S1 comprises the following steps: S11, setting the image acquisition cycle; S12, regularly collecting the driver's eye images and the driver's body images in the driving state through an image capturing device installed in the vehicle.

3. The method for detecting and warning driver's standardized behavior based on machine vision according to claim 2 is characterized in that: The S2 comprises the following steps: S21, performing grayscale processing on the driver's eye image and the driver's body image respectively to obtain grayscale processed driver's eye image data and driver's body image data; S22, using a median filtering method to perform noise reduction processing on the driver's eye image data and the driver's body image data after grayscale processing, to obtain a driver's eye feature image data set and a driver's body feature image data set.

4. The method for detecting and warning driver's standard behavior based on machine vision according to claim 3 is characterized in that: The grayscale processing of the image in S21 comprises the following steps: S211, extracting pixel values ​​of red, green and blue corresponding color components of each pixel in the image; S212, setting weight factors of red, green and blue corresponding color components; S213, using a weighted average method to calculate the pixel mean of the red, green and blue color components corresponding to each pixel point, and using the calculated pixel mean as the target grayscale value of the corresponding pixel point.

5. The method for detecting and warning driver's standard behavior based on machine vision according to claim 4 is characterized in that: The noise reduction processing of the image data in S22 comprises the following steps: S221, setting an m×m sliding window, and aligning the center position of the sliding window with each pixel point in sequence; S222, whenever the center position of the sliding window is aligned with a pixel, the target grayscale values ​​of all the pixels in the sliding window are obtained, and the target grayscale values ​​are arranged from large to small according to the numerical values ​​to obtain a grayscale value sequence; S223, calculating the median of the gray value sequence, and using the calculated median as the target gray value of the pixel point corresponding to the center position of the current sliding window.

6. The method for detecting and warning driver's standard behavior based on machine vision according to claim 5 is characterized in that: The S3 comprises the following steps: S31, clustering the driver's eye feature image data in the driver's eye feature image data set and the driver's body feature image data in the driver's body feature image data set to obtain an eye feature image classification data set and a body feature image classification data set; the clustering operation is as follows: S311, treating each image data in the image data set as a cluster, calculating the Euclidean distance between clusters, and obtaining an initial distance matrix; S312, selecting a minimum value from the initial distance matrix, merging two clusters corresponding to the minimum value into a new cluster, and recalculating the Euclidean distance between the new cluster and other clusters; S313, repeating the operations from S311 to S312 until the shortest Euclidean distance between clusters is greater than the set Euclidean distance threshold, stopping the clustering operation, and obtaining clustered image data.

7. The method for detecting and warning driver's standard behavior based on machine vision according to claim 6 is characterized in that: The S4 comprises the following steps: S41, collecting the standard eye feature image data of the driver in the driving state, and establishing a standard eye feature image data set; S42. Collect the standard body feature image data of the driver in the driving state, and establish a standard body feature image data set.

8. The method for detecting and warning driver's standard behavior based on machine vision according to claim 7 is characterized in that: The S5 comprises the following steps: S51, randomly selecting one image data from the image data sets of eye features of various types of drivers in the eye feature image classification data set, to obtain all the eye feature image data to be matched; When any eye feature image data to be matched successfully matches the eye standard feature image data in the eye standard feature image data set, it means that all image data in the image data set of the eye features of such driver in the eye feature image classification data set successfully matches the eye standard feature image data in the eye standard feature image data set; Selecting one image data at random from the image data sets of various types of driver body features in the body feature image classification data set, respectively, to obtain all body feature image data to be matched; When any body feature image data to be matched successfully matches the body standard feature image data in the body standard feature image data set, it means that all image data in the image data set of the body features of such driver in the body feature image classification data set successfully matches the body standard feature image data in the body standard feature image data set; S52, sequentially matching each eye feature image data to be matched with the eye standard feature image data in the eye standard feature image data set, and selecting the eye feature image data with successful matching, and calculating the eye standard weight factor λ; the image matching process is as follows: S521. Construct a crow population, set the population size to N, the detection probability to AP, the current number of iterations to t, and the maximum number of iterations to t max and the search space dimension of the eye normative feature image data is P; Randomly generate N crow individuals in the eye standard feature image data search space, each crow individual corresponds to one eye standard feature image data, and initialize the initial position set of the crow population; S522, taking the initial position of each crow individual in the crow population as the initial food hiding position of each crow individual, and calculating the fitness function value of the food hiding position of each crow individual in the crow population; S523, in order to steal food from other crows, individual crows will follow other crows in the eye canonical feature image data search space to update their positions; S524, calculating the fitness function value of the crow individual after the position is updated in the eye standard feature image data search space, when the fitness function value of the crow individual after the position is updated is greater than the fitness function value before the position is updated, the position is used as the food hiding position of the crow individual; otherwise, the original food hiding position of the crow individual is retained; S525: Determine whether the current number of iterations t is less than the maximum number of iterations t max , if the current number of iterations is t and less than the maximum number of iterations is t max , then the current iteration number is t plus 1 and returns to S523; otherwise, the fitness function value of each food hiding position of each crow in the crow population is calculated, and the food hiding position of the crow with the highest fitness function value is taken as the global optimal solution, and the fitness function threshold is set. When the fitness function value of the global optimal solution is greater than the fitness function threshold, it is determined that the eye standard feature image data corresponding to the global optimal solution matches the eye feature image data to be matched successfully; otherwise, it is determined that the eye feature image data to be matched and all the eye standard feature image data in the eye standard feature image data set fail to match; S53, using the operations in S521 to S525, image matching is performed on each body feature image data to be matched with the body standard feature image data in the body standard feature image data set in turn, and the body feature image data to be matched that matches successfully is screened out, and the body standard weight factor η is calculated.

9. The method for detecting and warning driver's standard behavior based on machine vision according to claim 8, characterized in that: The S6 comprises the following steps: S61, setting the eye norm weight threshold to α and the body norm weight threshold to β; S62, respectively comparing the eye standard weight factor λ with the eye standard weight threshold α, the body standard weight factor η with the body standard weight threshold β, and determining whether the driver's driving behavior is standard in the current driving state according to the comparison results; S621, if λ≥α and η≥β, it is determined that the driver's driving behavior in the current driving state is a standard behavior, and no operation is performed; If λ<α and η≥β or λ≥α and η<β, the driver's driving behavior in the current driving state is determined to be abnormal, and the determination result is fed back to the main control unit, which issues an abnormal behavior signal warning; If λ<α and η<β, the driver's driving behavior in the current driving state is determined to be a dangerous behavior, and the determination result is fed back to the main control unit, which issues a dangerous behavior signal warning and a voice prompt.

10. A system for implementing the driver's standardized behavior detection and early warning method based on machine vision as described in any one of claims 1 to 9.

Images