Secondary monitoring method of driver behavior based on model quantification

Abstract

The present invention provides a secondary monitoring method of driver behavior based on model quantization, the method comprising: obtaining offline video data; the offline video data is the part of the online video data where the driver behavior event occurs; running a first video understanding network, and using the offline video data as the input of the first video understanding network; the first video understanding network is obtained by quantization of the second video understanding network; and obtaining the secondary monitoring result of the driver behavior according to the event monitoring task score output by the first video understanding network. The present invention effectively solves the problem of high false alarm rate of driver behavior event monitoring caused by insufficient computing resources on the vehicle equipment side; at the same time, it can take into account the efficiency of offline video feature extraction and the accuracy of target prediction, so that the secondary monitoring of offline video data uploaded by multiple vehicle-mounted equipment sides is more robust.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to a driver behavior secondary monitoring method based on model quantization. Background Art

[0002] Driver behavior recognition in the cockpit is the key to ensuring driver safety. The algorithm needs to monitor and identify the driver's behavior in real time and issue timely alarms to reduce unsafe driving behaviors such as smoking, making phone calls, closing eyes due to fatigue, yawning, camera obstruction, distraction, direct sunlight, whether the camera angle is correct, whether the driver is in the camera, etc.

[0003] In order to achieve the purpose of monitoring the behavior of the pilot in the cabin, the existing technology provides some algorithms that use facial key points, smoking detection / classification or mobile phone detection / classification, combined with post-processing logic to judge and identify the behavior.

[0004] However, due to the limited computing resources of vehicle mobile devices, the accuracy of monitoring and identification results of existing technologies is low and the false alarm rate is high.

[0005] Therefore, how to provide a driver behavior monitoring method with high accuracy and low false alarm rate has become an urgent problem to be solved in the industry. Summary of the invention

[0006] The present invention provides a secondary monitoring method for driver behavior based on model quantization, which is used to solve the defects of low accuracy and high false alarm rate of monitoring and recognition results in the prior art due to limited computing resources of vehicle mobile terminal devices, and realize accurate driver behavior monitoring.

[0007] The present invention provides a driver behavior secondary monitoring method based on model quantification, comprising:

[0008] Obtaining offline video data; the offline video data is the portion of the online video data where the driver behavior event occurs, and is determined by a driver behavior monitoring system deployed on the vehicle equipment end based on the online video data in the vehicle cabin;

[0009] Running a first video understanding network, and using the offline video data as input of the first video understanding network; the first video understanding network is obtained by quantizing the second video understanding network, and the number of parameter bits of the first video understanding network is smaller than the number of parameter bits of the second video understanding network; the second video understanding network is obtained by training with decoded data samples;

[0010] According to the event monitoring task score output by the first video understanding network, a secondary monitoring result of the driver's behavior is obtained.

[0011] According to a driver behavior secondary monitoring method based on model quantization provided by the present invention, the step of running a first video understanding network and using the offline video data as input of the first video understanding network includes:

[0012] Decoding the offline video data in parallel based on the CPU to obtain decoded data;

[0013] The first video understanding network is concurrently run based on a GPU, and the decoded data is used as an input of the first video understanding network.

[0014] According to a driver behavior secondary monitoring method based on model quantization provided by the present invention, the quantification refers to:

[0015] Transforming the parameters of the second video understanding network by a set mapping to obtain a third video understanding network having a smaller number of parameter bits than the second video understanding network;

[0016] Determining that the accuracy of the third video understanding network is less than the accuracy of the second video understanding network, and the accuracy difference between the third video understanding network and the second video understanding network is greater than a threshold, then using the third video understanding network after supplementary training as the first video understanding network; the number of parameter bits of the supplementary training is the same as the number of parameter bits of the third video understanding network;

[0017] If it is determined that the accuracy of the third video understanding network is not less than the accuracy of the second video understanding network, or the accuracy difference between the third video understanding network and the second video understanding network is not greater than a threshold, the third video understanding network is used as the first video understanding network.

[0018] According to a driver behavior secondary monitoring method based on model quantization provided by the present invention, the set mapping refers to:

[0019] T 2 =sf×T 1 +bias

[0020] Where, T 2 is the parameter of the second video understanding network; T 1 are the parameters of the first video understanding network; bias and sf are respectively obtained by T 2 The value of T 1 The number of bits determines the bias value and scale factor.

[0021] According to a driver behavior secondary monitoring method based on model quantization provided by the present invention, the CPU resources of the CPU and the GPU resources of the GPU meet the following conditions:

[0022] The time consumed by using the CPU resources to decode the offline video data in parallel is the same as the time consumed by using the GPU resources to concurrently run the first video understanding network to obtain the event monitoring task score;

[0023] The CPU resources refer to the total computing resources used for secondary monitoring of driver behavior in one or more CPUs; the GPU resources refer to the total computing resources used for secondary monitoring of driver behavior in one or more GPUs.

[0024] According to a driver behavior secondary monitoring method based on model quantization provided by the present invention, the parameters of the first video understanding network are 8-bit integers; and the parameters of the second video understanding network are 32-bit floating point numbers.

[0025] The present invention also provides a driver behavior secondary monitoring system based on model quantification, comprising:

[0026] A data acquisition module, used to acquire offline video data; the offline video data is the part of the online video data where the driver behavior event occurs, and is determined by the driver behavior monitoring system deployed on the vehicle equipment end based on the online video data in the vehicle cabin;

[0027] A network inference module, used for running a first video understanding network, and taking the offline video data as input of the first video understanding network; the first video understanding network is obtained by quantizing the second video understanding network, and the number of parameter bits of the first video understanding network is smaller than the number of parameter bits of the second video understanding network; the second video understanding network is obtained by training with decoded data samples;

[0028] The monitoring result module is used to obtain a secondary monitoring result of the driver's behavior according to the event monitoring task score output by the first video understanding network.

[0029] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of any of the above-mentioned methods for secondary monitoring of driver behavior based on model quantification are implemented.

[0030] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the above-described methods for secondary monitoring of driver behavior based on model quantification.

[0031] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the steps of any of the above-mentioned methods for secondary monitoring of driver behavior based on model quantification are implemented.

[0032] The driver behavior secondary monitoring method based on model quantization provided by the present invention performs secondary monitoring on the part of the driver behavior event monitoring results on the vehicle equipment side that determines the occurrence of the event, thereby effectively solving the problem of high false alarm rate of driver behavior event monitoring caused by insufficient computing resources on the vehicle equipment side; at the same time, based on the quantized first video understanding network, network operations (such as convolution) are performed with a lower number of parameter bits, which can take into account both the efficiency of offline video feature extraction and the accuracy of target prediction, thereby making the secondary monitoring of offline video data uploaded by multiple vehicle-mounted equipment sides more robust. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0034] Figure 1 It is a flow chart of the driver behavior secondary monitoring method based on model quantification provided by the present invention;

[0035] Figure 2 is a schematic diagram of parameter quantization provided by an embodiment of the present invention;

[0036] Figure 3 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] Combine the following Figure 1 , Figure 2 The present invention describes the driver behavior secondary monitoring method based on model quantization.

[0039] like Figure 1 As shown, an embodiment of the present invention provides a driver behavior secondary monitoring method based on model quantification, comprising:

[0040] Step 101, obtaining offline video data; the offline video data is the part of the online video data where the driver behavior event occurs, and is determined by the driver behavior monitoring system deployed on the vehicle equipment end according to the online video data in the vehicle cabin;

[0041] Step 103, running a first video understanding network, and using the offline video data as input of the first video understanding network; the first video understanding network is obtained by quantizing the second video understanding network, and the number of parameter bits of the first video understanding network is smaller than the number of parameter bits of the second video understanding network; the second video understanding network is obtained by training with decoded data samples;

[0042] Step 105: Obtain a secondary monitoring result of the driver's behavior according to the event monitoring task score output by the first video understanding network.

[0043] The execution subject of this embodiment may be a secondary driver behavior monitoring system based on an offline video understanding network deployed on the cloud or platform side.

[0044] The secondary monitoring method of this embodiment is compatible with vehicle device-side monitoring systems of different types and algorithms, such as a lightweight driver behavior monitoring network based on facial key points, a human eye closure determination network, an online video understanding network, etc. deployed locally in the vehicle. The offline video data in step 101 can be obtained by the vehicle device side through network transmission.

[0045] The method of this embodiment can be applied to a monitoring platform, and the monitoring platform can have different vehicles belonging to multiple vehicle-mounted monitoring equipment companies.

[0046] A typical application scenario is a driver behavior monitoring platform for trucks and other operating vehicles. The monitoring platform has multiple different vehicle operating companies, and multiple operating vehicles under different vehicle operating companies use different vehicle local monitoring devices. Although different vehicle local monitoring devices use different primary monitoring methods, the offline video data of driver behavior events reported to the monitoring platform can be monitored secondary through the monitoring platform. Therefore, even if the vehicle local monitoring device has a high false alarm rate, the monitoring platform can still give more accurate monitoring results.

[0047] In this application scenario, the vehicle's local monitoring equipment can focus more on improving the recall rate of monitoring results and reducing the accuracy requirements of monitoring results, thereby maximizing the utilization efficiency of limited computing resources on the vehicle's local equipment.

[0048] Furthermore, since the monitoring equipment on the vehicle is more focused on improving the recall rate of monitoring results, the monitoring platform also receives a huge amount of offline video data. Even if the computing resources on the platform / cloud are relatively abundant compared to the local vehicle, when facing a high number of vehicles that need secondary monitoring, the secondary monitoring method on the platform side still needs to reduce the computing resource requirements and improve the efficiency of secondary monitoring while ensuring accuracy. This is also the reason why all online videos are not uploaded to the cloud / platform side for monitoring.

[0049] In order to ensure the accuracy of the secondary monitoring results, the feature extraction network in the offline video understanding network of this embodiment usually uses a deep network with a large number of convolutional layers. The network calculation (such as convolution calculation) process is time-consuming. At the same time, although the decoding process of offline video data occupies few computing resources, it is time-consuming (in traditional algorithms, the process link of video decoding, input network, and obtaining target prediction results is single-threaded sequential, so the network reasoning operation process needs to wait for the video decoding to be completed. During the video decoding period, some computing resources are idle, which makes the overall process of driver behavior monitoring time-consuming).

[0050] In summary, how to improve feature extraction efficiency and target prediction efficiency while ensuring accuracy is another technical problem solved by this embodiment.

[0051] When training and using deep learning networks, 32-bit floating point parameters are usually used for calculations. Calculations of high-bit parameters require more computing resources and are more time-consuming, and the improvement in network accuracy is not proportional to the increase in computing resource requirements (or computing time). Therefore, this embodiment provides a video understanding network that quantizes high-bit parameters into low-bit parameters, which can effectively reduce the demand for computing resources and computing time with little loss in accuracy. Taking 32-bit networks and 8-bit networks as examples, the latter loses about 1% of its accuracy relative to the former, while the amount of calculation can be reduced by nearly three quarters.

[0052] The beneficial effects of this embodiment are:

[0053] By conducting secondary monitoring of the part of the driver behavior event monitoring results on the vehicle equipment side that determines the occurrence of the event, the problem of high false alarm rate in driver behavior event monitoring caused by insufficient computing resources on the vehicle equipment side is effectively solved; at the same time, based on the quantized first video understanding network, network operations (such as convolution) are performed with a lower number of parameter bits, which can take into account both the efficiency of offline video feature extraction and the accuracy of target prediction, making the secondary monitoring of offline video data uploaded by multiple on-board devices more robust.

[0054] According to the above embodiment, in this embodiment:

[0055] The step of running the first video understanding network and using the offline video data as input of the first video understanding network includes:

[0056] Decoding the offline video data in parallel based on the CPU to obtain decoded data;

[0057] The first video understanding network is concurrently run based on a GPU, and the decoded data is used as an input of the first video understanding network.

[0058] The CPU resources of the CPU and the GPU resources of the GPU satisfy:

[0059] The time consumed by using the CPU resources to decode the offline video data in parallel is the same as the time consumed by using the GPU resources to concurrently run the first video understanding network to obtain the event monitoring task score;

[0060] The CPU resources refer to the total computing resources used for secondary monitoring of driver behavior in one or more CPUs; the GPU resources refer to the total computing resources used for secondary monitoring of driver behavior in one or more GPUs.

[0061] This embodiment can be understood as decoupling the execution subject's program SDK, extracting video decoding (i.e. processing the video into pictures for each frame) from the program link, and then using the CPU to perform the video decoding part and the GPU to perform network reasoning. Since the CPU is better at decoding and the GPU is better at network reasoning, this decoupling setting can further speed up the running speed of the SDK.

[0062] The two decoupled parts, namely video decoding and network inference, are performed on the CPU and GPU respectively. Therefore, these two parts of work are carried out simultaneously and computing resources can be utilized to the maximum extent.

[0063] At the same time, for the CPU, its computing resources are sufficient to process the decoding of multiple offline videos in parallel, and the GPU's computing resources are also sufficient to process multiple network inference processes in parallel. Therefore, the multiple network inferences run by the GPU at the current moment are based on the multiple decoded data output by the CPU at the previous moment (each decoded data corresponds to a network for inference), and the video data decoded by the CPU at the current moment is used by the GPU at the next moment.

[0064] The beneficial effects of this embodiment are:

[0065] Driver behavior recognition in the cockpit is the key to ensuring driver safety. The device-side algorithm returns a large amount of videos every day, and the platform-side algorithm needs to do a lot of processing and judgment on these videos. Therefore, it is crucial to improve the inference speed of the platform-side model. The existing algorithm has a very slow inference speed and processes very few videos concurrently, with a single GPU processing less than 10,000 videos per day. In this embodiment:

[0066] 1. Greatly improved the speed of model reasoning and reduced the hardware cost of GPU;

[0067] 2. The number of concurrently processed videos has been greatly improved, which can meet the daily concurrent processing capacity of more than 50,000 videos on a single GPU.

[0068] According to any of the above embodiments, in this embodiment:

[0069] The quantification refers to:

[0070] Transforming the parameters of the second video understanding network by a set mapping to obtain a third video understanding network having a smaller number of parameter bits than the second video understanding network;

[0071] Determining that the accuracy of the third video understanding network is less than the accuracy of the second video understanding network, and the accuracy difference between the third video understanding network and the second video understanding network is greater than a threshold, then using the third video understanding network after supplementary training as the first video understanding network; the number of parameter bits of the supplementary training is the same as the number of parameter bits of the third video understanding network;

[0072] If it is determined that the accuracy of the third video understanding network is not less than the accuracy of the second video understanding network, or the accuracy difference between the third video understanding network and the second video understanding network is not greater than a threshold, the third video understanding network is used as the first video understanding network.

[0073] The mapping of the settings refers to:

[0074] T 2 =sf×T 1 +bias

[0075] Where, T 2 is the parameter of the second video understanding network; T 1 are the parameters of the first video understanding network; bias and sf are respectively obtained by T 2 The value of T 1 The number of bits determines the bias value and scale factor.

[0076] In the above embodiment, although the process of quantizing the network effectively reduces the computation time, it still causes a slight loss in accuracy.

[0077] This embodiment adds a verification process after network quantization to further reduce the accuracy loss. This embodiment can be specifically implemented through the following process:

[0078] Running the second video understanding network with the driver behavior secondary monitoring program (or other execution entities applicable to the present embodiment), quantizing the parameters of the second video understanding network into a lower number of bits with a set mapping, and forming a third video understanding network;

[0079] Verify and determine the accuracy of the third video understanding network relative to the second video understanding network; for example, the second video understanding network and the third video understanding network may be respectively input with the same test set, and the outputs of the two networks may be compared with the true value (label) of the test set, thereby obtaining the accuracy data of the two networks.

[0080] If the accuracy of the third video understanding network is higher than that of the second video understanding network (in some cases, due to the mapping rules, the accuracy of the network after quantization may be higher than that of the network before quantization), the third video network is directly used as the network for the final application operation, that is, the first video understanding network;

[0081] If the accuracy of the third video understanding network is not higher than that of the second video understanding network, the set accuracy difference threshold is used for judgment:

[0082] If the accuracy difference between the third video understanding network and the second video understanding network is not greater than the threshold, the third video network is still directly used as the network for the final application operation, that is, the first video understanding network;

[0083] If the accuracy difference between the third video understanding network and the second video understanding network is greater than a threshold, supplementary training is performed on the third video understanding network, and the supplementary trained third video network is used as the network for final application operation, namely, the first video understanding network.

[0084] During the supplementary training, the number of parameter bits is the same as the number of parameter bits in the third video understanding network. That is to say, in this case, the third video understanding network has undergone two trainings with different numbers of bits. The stopping condition of the supplementary training can be set to a difference in accuracy with the second video understanding network that is less than or equal to a threshold.

[0085] Generally speaking, the threshold should be set to a small value, such as 1×10 -4 .

[0086] In a preferred solution of this embodiment, the parameters of the first video understanding network are 8-bit integers; and the parameters of the second video understanding network are 32-bit floating point numbers.

[0087] like Figure 2As shown in the figure, the mapping of the quantization process is explained by taking the quantization of a 32-bit floating point number to an 8-bit integer as an example:

[0088] First, for each layer in the second video understanding network, the activation function value range is determined;

[0089] Second, according to the activation function value range, determine the mapping range -max to +max as T 2 The value range of

[0090] Third, according to T 2 =sf×T 1 +bias, determines each parameter T in the first video understanding network shown 1 The value of .

[0091] In some preferred implementations, after the above mapping process is completed, a quantized third video understanding network can be obtained. For the third video understanding network, it is determined that the accuracy of the third video understanding network is less than the accuracy of the second video understanding network, and the accuracy difference between the third video understanding network and the second video understanding network is greater than a threshold. Then, the third video understanding network is further optimized and used as the first video understanding network.

[0092] Regarding the optimization process of the third video understanding network, in addition to the supplementary training described above, the accuracy of the network model can be made higher by adjusting the value of max.

[0093] It is worth noting that since the range of activation function values ​​may be different in each layer of the video understanding network, the max value of each layer may also be different, and the quantization mapping of each layer may also be different.

[0094] The beneficial effects of this embodiment are:

[0095] On the basis of the above embodiment, by using the verified third video understanding network as the first video understanding network actually running, the accuracy loss of the network is further reduced while maintaining the reduced calculation time.

[0096] The following is a description of the secondary driver behavior monitoring device based on model quantization provided by the present invention. The secondary driver behavior monitoring device based on model quantization described below and the secondary driver behavior monitoring method based on model quantization described above can refer to each other.

[0097] The embodiment of the present invention further provides a driver behavior secondary monitoring system based on model quantification, comprising:

[0098] A data acquisition module, used to acquire offline video data; the offline video data is the part of the online video data where the driver behavior event occurs, and is determined by the driver behavior monitoring system deployed on the vehicle equipment end based on the online video data in the vehicle cabin;

[0099] A network inference module, used for running a first video understanding network, and taking the offline video data as input of the first video understanding network; the first video understanding network is obtained by quantizing the second video understanding network, and the number of parameter bits of the first video understanding network is smaller than the number of parameter bits of the second video understanding network; the second video understanding network is obtained by training with decoded data samples;

[0100] The monitoring result module is used to obtain a secondary monitoring result of the driver's behavior according to the event monitoring task score output by the first video understanding network.

[0101] Furthermore, the network reasoning module includes:

[0102] A decoding submodule, used for decoding the offline video data in parallel based on a CPU to obtain decoded data;

[0103] A running submodule is used to concurrently run the first video understanding network based on a GPU, and use the decoded data as an input of the first video understanding network.

[0104] The quantification refers to:

[0105] Transforming the parameters of the second video understanding network by a set mapping to obtain a third video understanding network having a smaller number of parameter bits than the second video understanding network;

[0106] Determining that the accuracy of the third video understanding network is less than the accuracy of the second video understanding network, and the accuracy difference between the third video understanding network and the second video understanding network is greater than a threshold, then using the third video understanding network after supplementary training as the first video understanding network; the number of parameter bits of the supplementary training is the same as the number of parameter bits of the third video understanding network;

[0107] If it is determined that the accuracy of the third video understanding network is not less than the accuracy of the second video understanding network, or the accuracy difference between the third video understanding network and the second video understanding network is not greater than a threshold, the third video understanding network is used as the first video understanding network.

[0108] The mapping of the settings refers to:

[0109] T 2 =sf×T 1 +bias

[0110] Where, T2 is the parameter of the second video understanding network; T 1 are the parameters of the first video understanding network; bias and sf are respectively obtained by T 2 The value of T 1 The number of bits determines the bias value and scale factor.

[0111] The CPU resources of the CPU and the GPU resources of the GPU satisfy:

[0112] The time consumed by using the CPU resources to decode the offline video data in parallel is the same as the time consumed by using the GPU resources to concurrently run the first video understanding network to obtain the event monitoring task score;

[0113] The CPU resources refer to the total computing resources used for secondary monitoring of driver behavior in one or more CPUs; the GPU resources refer to the total computing resources used for secondary monitoring of driver behavior in one or more GPUs.

[0114] The parameters of the first video understanding network are 8-bit integers; the parameters of the second video understanding network are 32-bit floating point numbers.

[0115] Figure 3 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 3 As shown, the electronic device may include: a processor 310, a communication interface 320, a memory 330 and a communication bus 340, wherein the processor 310, the communication interface 320 and the memory 330 communicate with each other through the communication bus 340. The processor 310 may call the logic instructions in the memory 330 to execute the secondary monitoring method of driver behavior based on model quantization, the method comprising: obtaining offline video data; the offline video data is the part of the online video data where the driver behavior event occurs, and is determined by the driver behavior monitoring system deployed at the vehicle equipment end according to the online video data in the vehicle cabin; running a first video understanding network, and using the offline video data as the input of the first video understanding network; the first video understanding network is obtained by quantizing the second video understanding network, and the parameter bit number of the first video understanding network is less than the parameter bit number of the second video understanding network; the second video understanding network is obtained by training the decoded data sample; according to the event monitoring task score output by the first video understanding network, the secondary monitoring result of the driver behavior is obtained.

[0116] In addition, the logic instructions in the above-mentioned memory 330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0117] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the model-quantized secondary driver behavior monitoring method provided by the above-mentioned methods, the method including: obtaining offline video data; the offline video data is the part of the online video data where the driver behavior event occurs, which is obtained by a driver behavior monitoring system deployed on a vehicle equipment end according to the online video data in the vehicle cabin; running a first video understanding network, and using the offline video data as the input of the first video understanding network; the first video understanding network is quantized by the second video understanding network, and the number of parameter bits of the first video understanding network is less than the number of parameter bits of the second video understanding network; the second video understanding network is trained by decoding data samples; and according to the event monitoring task score output by the first video understanding network, a secondary monitoring result of the driver behavior is obtained.

[0118] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the model-quantized secondary driver behavior monitoring method provided by the above-mentioned methods, the method comprising: obtaining offline video data; the offline video data is the part of the online video data where the driver behavior event occurs, and is determined by a driver behavior monitoring system deployed at the vehicle equipment end based on the online video data in the vehicle cabin; running a first video understanding network, and using the offline video data as the input of the first video understanding network; the first video understanding network is quantized from the second video understanding network, and the number of parameter bits of the first video understanding network is smaller than the number of parameter bits of the second video understanding network; the second video understanding network is trained from decoded data samples; and according to the event monitoring task score output by the first video understanding network, a secondary monitoring result of the driver behavior is obtained.

[0119] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0120] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0121] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A secondary monitoring method of driver behavior based on model quantification, characterized in that: include: Get offline video data; The offline video data is the part of the online video data where the driver behavior event occurs, and is determined by the driver behavior monitoring system deployed on the vehicle equipment end based on the online video data in the vehicle cabin; Running a first video understanding network, and using the offline video data as input of the first video understanding network; The first video understanding network is obtained by quantizing the second video understanding network, and the number of parameter bits of the first video understanding network is smaller than the number of parameter bits of the second video understanding network; The second video understanding network is trained by decoding data samples; The step of running the first video understanding network and using the offline video data as input of the first video understanding network includes: Decoding the offline video data in parallel based on the CPU to obtain decoded data; Concurrently running the first video understanding network based on the GPU, and using the decoded data as input of the first video understanding network; The parameters of the first video understanding network are 8-bit integers; the parameters of the second video understanding network are 32-bit floating point numbers; The quantification refers to: Transforming the parameters of the second video understanding network by a set mapping to obtain a third video understanding network having a smaller number of parameter bits than the second video understanding network; Determining that the accuracy of the third video understanding network is less than the accuracy of the second video understanding network, and the accuracy difference between the third video understanding network and the second video understanding network is greater than a threshold, then using the third video understanding network after supplementary training as the first video understanding network; the number of parameter bits of the supplementary training is the same as the number of parameter bits of the third video understanding network; Determining that the accuracy of the third video understanding network is not less than the accuracy of the second video understanding network, or the accuracy difference between the third video understanding network and the second video understanding network is not greater than a threshold, then using the third video understanding network as the first video understanding network; According to the event monitoring task score output by the first video understanding network, a secondary monitoring result of the driver's behavior is obtained.

2. The driver behavior secondary monitoring method based on model quantification according to claim 1 is characterized in that: The mapping of the settings refers to: T2=sf×T1+bias In the formula, T2 is a parameter of the second video understanding network; T1 is a parameter of the first video understanding network; bias and sf are a bias value and a proportional factor respectively determined by the value of T2 and the number of bits of T1.

3. The driver behavior secondary monitoring method based on model quantification according to claim 1 is characterized in that: The CPU resources of the CPU and the GPU resources of the GPU satisfy: The time consumed by using the CPU resources to decode the offline video data in parallel is the same as the time consumed by using the GPU resources to concurrently run the first video understanding network to obtain the event monitoring task score; The CPU resources refer to the total computing resources used for secondary monitoring of driver behavior in one or more CPUs; the GPU resources refer to the total computing resources used for secondary monitoring of driver behavior in one or more GPUs.

4. A driver behavior secondary monitoring system based on model quantification, characterized in that: include: A data acquisition module, used to acquire offline video data; The offline video data is the part of the online video data where the driver behavior event occurs, and is determined by the driver behavior monitoring system deployed on the vehicle equipment end based on the online video data in the vehicle cabin; A network inference module, configured to run a first video understanding network and use the offline video data as an input of the first video understanding network; The first video understanding network is obtained by quantizing the second video understanding network, and the number of parameter bits of the first video understanding network is smaller than the number of parameter bits of the second video understanding network; The second video understanding network is trained by decoding data samples; The step of running the first video understanding network and using the offline video data as input of the first video understanding network includes: Decoding the offline video data in parallel based on the CPU to obtain decoded data; Concurrently running the first video understanding network based on the GPU, and using the decoded data as input of the first video understanding network; The parameters of the first video understanding network are 8-bit integers; the parameters of the second video understanding network are 32-bit floating point numbers; The quantification refers to: Transforming the parameters of the second video understanding network by a set mapping to obtain a third video understanding network having a smaller number of parameter bits than the second video understanding network; Determining that the accuracy of the third video understanding network is less than the accuracy of the second video understanding network, and the accuracy difference between the third video understanding network and the second video understanding network is greater than a threshold, then using the third video understanding network after supplementary training as the first video understanding network; the number of parameter bits of the supplementary training is the same as the number of parameter bits of the third video understanding network; Determining that the accuracy of the third video understanding network is not less than the accuracy of the second video understanding network, or the accuracy difference between the third video understanding network and the second video understanding network is not greater than a threshold, then using the third video understanding network as the first video understanding network; The monitoring result module is used to obtain a secondary monitoring result of the driver's behavior according to the event monitoring task score output by the first video understanding network.

5. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the driver behavior secondary monitoring method based on model quantification as described in any one of claims 1 to 3 are implemented.

6. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the driver behavior secondary monitoring method based on model quantification as claimed in any one of claims 1 to 3 are implemented.

7. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the driver behavior secondary monitoring method based on model quantification as claimed in any one of claims 1 to 3 are implemented.

Images